Technology & AI

Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M: CPU-Fast Bidirectional Encoders for 8K Content

Photo of admin admin Send an email 2 hours ago
0 0 5 minutes read
Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M: CPU-Fast Bidirectional Encoders for 8K Content

Liquid AI has released two open source bidirectional encoders, the LFM2.5-Encoder-230M and the LFM2.5-Encoder-350M. Both are latent language models built on a hybrid LFM2 core. Both hold a core of 8,192 tokens.

Encoders live under separators, purpose routers, security filters, and PII finders. Those tasks run continuously, often without the GPU, and are increasingly long-term installations. BERT invented the class. ModernBERT pushed its accuracy, speed, and context. Liquid AI’s argument is that LFM2’s design continues along that line, because its cost increases slowly as the input takes longer.

How does a decoder become an encoder

Coders are not trained from scratch. Launched from the LFM2.5-230M and LFM2.5-350M decoder backbones, then it is changed by three changes:

  • First, the causal attention mask is replaced by a bidirectional one, so every token looks in both directions.
  • Second, LFM2 short convolutions are arbitrary using symmetric center padding, so the transformation of each token converges to neighbors on both sides.
  • Third, the model is trained for the hidden language matching target with a mask value of 30%. That’s tighter than 15% BERT, following evidence Liquid AI shows that a higher mask rate helps at this scale.

The training is divided into two phases:

  • The first stage establishes a common language ability for the purpose of short MLM content in the large web corpus of 1,024 tokens.
  • The second phase expands the context to 8,192 tokens in the complete integration of the data, strengthening the capabilities of authenticity, law, and multilingualism.

Architecturally, the core combines modified blocks with a group attention gate, the same design described in the LFM2 technical report. Both checkpoints use a hash size of 1024 and a vocabulary of 65,536 tokens, and support 15 languages. The license is LFM Open License v1.0.

The embed below breaks down conversions, the full ranking table, individual task scores, and CPU latency statistics.

Benchmark results

Liquid AI tested 14 models on 17 tasks extracted from GLUE, SuperGLUE, and multilingual classification. Every model is fully fine-tuned for each function, and the reported result is the result of the fine-tuned model.

The LFM2.5-Encoder-350M posts a 17-task mean of 81.02 (±1.00), ranking fourth. Three models are ahead of all majors. They are XLM-R XL at 3.5B (83.06), ModernBERT-large at 395M (81.68), and XLM-R large at 560M (81.34). The top model is about 10 times its size.

LFM2.5-Encoder-230M posts 79.29 (±1.02), ranked sixth. It beats ModernBERT-base at 78.19 and every EuroBERT model on the table, including EuroBERT-610M (75.87) and EuroBERT-2.1B (72.19). Both new encoders are also above their Liquid AI siblings, the LFM2.5-ColBERT-350M (76.18) and the LFM2.5-Embedding-350M (75.68). That gap is the stated reason that Liquid AI built a general-purpose encoder instead of reusing detectors.

The mechanism is the most interesting part, and it is open under Apache-2.0. Every model is loaded with fp32 and bf16 autocast core weights, so the table compares models rather than a numerical format. All models use the same AdamW recipe, taken from the EuroBERT card. The learning rate is selected for each model and is performed on all 10 values ​​and 3 seeds. Scores are then reported as the average of more than 5 seeds that have not affected the selection. I transformers version is pinned to 4.56.2 so dependency drift is not an uncontrollable difference.

Use Cases and Delivery Locations

The release consists of three settings. Edge and embedded devices come first. A car computer or industrial controller doesn’t have a spare GPU, and can’t afford a round trip. Managed and on-premises systems in finance, healthcare, and legal, where documents are long, sensitive, and cannot leave internal infrastructure. And pipelines are sensitive to high-volume costs, where a small encoder acts as a cheap first pass in front of a larger model.

Liquid AI also puts a useful number in the context window: 8,192 tokens on about 13 to 15 pages. One pass forward includes a full contract or a complete patient record.

To show what a properly configured encoder looks like, the research team sent five demos. Each runs on CPU-only Hugging Face Space. They cover the zero shot fast track, zero-shot policy coverage, and spell checking. The PII detector handles 40 types of information in 16 languages. The demo of the hidden bonus distribution uses the coder as a chatbot that generates by repeatedly unpacking.

Making them run

Both encoders load transformers. The body is expressed as Lfm2BidirectionalModel and the use of hidden loading-LM Lfm2BidirectionalForMaskedLM. Both are connected by cables auto_mapso trust_remote_code=True required for all phone uploads.

A basic encoder produces general-purpose representations, not function outputs, so fine-tuning is necessary. Liquid AI’s fine-tuning tutorial goes through lengthy legal documents on 8k content configurations. The model selection guideline is straightforward: 350M where accuracy is most important, 230M for hard hardware or high throughput.