LFM2.5-Encoders for Fast Long-Context Inference on CPU

Two new models from Liquid AI, weighing 230M and 350M parameters, process 8,192-token inputs on a CPU in under 30 seconds. They match or exceed larger encoders on standard benchmarks while running significantly faster than ModernBERT-base for long documents.

Developers can now deploy intent routers, policy linters, and PII detectors that handle full contracts or transcripts without needing high-end hardware. Live demos are available below.

Why a general-purpose encoder matters

Last month, Liquid AI released LFM2.5-Retrievers for multilingual search. These new encoders follow the same architecture but serve a wider range of needs. Pre-trained with a masked-language objective, they support fine-tuning for classification, token-level tasks, and search. Search is only one function an encoder can enable, so the team built a general-purpose model rather than reusing the retrievers.

Modern production NLP relies on encoders for classifiers, intent routers, and safety filters. These jobs run constantly, usually on CPU, with inputs getting longer every year. BERT established this category, and ModernBERT recently pushed accuracy and speed further. LFM2.5-Encoders take the next step on the LFM2 architecture, where processing cost grows slowly as inputs expand.

Construction details

The team initialises the encoders from their respective LFM2 decoder backbones: LFM2.5-230M and LFM2.5-350M. They then convert each causal decoder into a bidirectional encoder with specific adjustments:

• Bidirectional attention mask: each token now sees tokens on both sides, not just the ones preceding it.
• Non-causal short convolutions: padding is symmetric so each token’s convolution mixes in neighbours on both sides.
• Masked language modeling: 30% of tokens are masked during training.

Both models undergo two training stages:

• General language competence: a short-context masked-language objective on a large web corpus at a 1,024-token context.
• Long-context adaptation: extending context to 8,192 tokens on the full data mix, strengthening factual, legal, and multilingual competence.

Benchmark Results

The team fine-tunes each model fully on every task and reports the resulting score. Across the table, that covers 14 models on 17 tasks pulled from GLUE, SuperGLUE, and multilingual classification.

Results are the mean across five held-out seeds, ensuring numbers are stable run to run. The full framework and raw results are open-sourced.

LFM2.5-Encoder-350M ranks fourth of the 14 models. The three ahead of it are all larger, including a 3.5B model nearly 10 times its size. LFM2.5-Encoder-230M beats ModernBERT-base and every EuroBERT model, while being smaller than most of them. Both also score well above their own LFM2.5-Retrievers here.

Inference speed on CPU and GPU

The encoders inherit the LFM2 backbone’s fast inference. Since both models and ModernBERT support an 8,192-token context, speed is measured across the full range.

The biggest edge appears on CPU. Here, LFM2.5-Encoder-230M is the fastest at every sequence length, even outperforming the smaller ModernBERT-base for short inputs. As input length increases, throughput drops sharply for ModernBERT, while the LFM2.5-Encoders rise into the mid-range before tapering. At 8,192 tokens, ModernBERT-base takes over a minute and a half per forward pass versus about 28s for LFM2.5-Encoder-230M. This is roughly 3.7x faster. For developers, that means scanning or classifying a full contract, transcript, or long support thread in under 30 seconds on a laptop CPU.

On GPU, a similar pattern holds with a smaller margin: ModernBERT-base leads below ~1K tokens on the Apple GPU. The encoders take the lead from about 2K tokens. This shows that for long inputs, LFM2.5-Encoders are the faster choice, and if you are running on CPU, dramatically so.

LFM2.5-Encoder demos

The demos below use fine-tuned LFM2.5-Encoders. Each one runs in a CPU-only Hugging Face space:

• Zero-shot prompt routing: define your own routing lanes as free text. The model scores the whole prompt against every lane in one pass.
• Zero-shot policy linting: check text against your company’s rules, written as free text. It scores every token against every rule in one pass.
• Spell checking: correct misspellings token by token.
• PII detection: spot and remove 40 kinds of personal information across 16 languages.
• Masked-diffusion text generation (bonus): run the encoder as a chatbot that generates text by iteratively unmasking instead of left to right.

How to use and fine-tune LFM2.5-Encoders

Choose an LFM2.5-Encoder for high-volume understanding tasks, such as classification, routing, extraction, or scoring, that run constantly and must stay cheap and fast. For these jobs, a fine-tuned encoder is smaller, faster, and far cheaper to run than a generative LLM, and it fits on the CPUs you already have.

Between the two encoder sizes:

• LFM2.5-Encoder-350M: choose it when accuracy matters most.
• LFM2.5-Encoder-230M: choose it for tighter hardware or higher throughput.

You can start in a few lines. Load a model with

transformers

. Then run it directly for masked-token prediction, or attach your own head and fine-tune it for your task.

Load and run the model

Install the latest version of

transformers

:

pip install -U transformers
Run masked-token prediction:
from transformers import AutoModelForMaskedLM, AutoTokenizer
import torch

model_id = "LiquidAI/LFM2.5-Encoder-230M"  # or "LiquidAI/LFM2.5-Encoder-350M"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
mlm = AutoModelForMaskedLM.from_pretrained(model_id, trust_remote_code=True)

text = f"The capital of France is {tok.mask_token}."
enc = tok(text, return_tensors="pt")
with torch.no_grad():
    logits = mlm(**enc).logits
pos = (enc["input_ids"][0] == tok.mask_token_id).nonzero()[0].item()
print([tok.decode([t]).strip() for t in logits[0, pos].topk(5).indices.tolist()])
# -> ['Paris', 'Strasbourg', 'Paris', 'Lyon', 'Versailles']
For downstream tasks, load the encoder body and attach your own head (classification, token classification, regression, retrieval):
from transformers import AutoModel

body = AutoModel.from_pretrained(model_id, trust_remote_code=True)
If your GPU supports it, use Flash Attention 2 for the highest efficiency:
pip install flash-attn
Fine-tuning for your task
A base encoder gives you general-purpose representations, not task outputs. So you fine-tune it for each task. The team’s fine-tuning tutorial walks through fine-tuning on long legal documents with an 8k context.
Get started with LFM2.5-Encoders
Both encoders are open-weight and available on Hugging Face today:
Download: LFM2.5-Encoder-230M and LFM2.5-Encoder-350M on Hugging Face.
Try: run the demos above in your browser, no setup needed.
Fine-tune: adapt an encoder to your task with the team’s fine-tuning tutorial.
We can’t wait to see what you build.
Citation
If you use this work, please cite the release blog:
@article{liquidAI2026Encoders,
  author = {Liquid AI},
  title = {LFM2.5-Encoders: Fast at Long Context, Even on CPU},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/lfm2-5-encoders},
}

Source Read original →
Related reading
Meituan Releases LongCat-2.0: A 1.6T-Parameter Open MoE Model with Native 1M Context and LongCat Sparse Attention
GLM-5.2 OpenAI-Compatible API: A Hands-On Guide to Reasoning Effort, Function Calling, and Long-Context Retrieval
Liquid AI Introduces LFM2.5-Embedding-350M and LFM2.5-ColBERT-350M: Dense Bi-Encoder and Late-Interaction Models for Fast Multilingual Search Across 11 Languages