DeepSeek AI has released DeepSeek-V4.1-Flash, a multimodal model capable of handling 1 million tokens. The system features a 552 billion parameter backbone and 196 billion Engram parameters. During prefill, it activates 8 billion parameters per token; during decoding, that rises to 16 billion. The primary improvement is a global key-value cache footprint of 890 bytes per token. This is roughly one-quarter the size of the previous DeepSeek-V4-Flash and approximately 437 times smaller than DeepSeek-V1.
The weights are open source under an MIT license. They are available for deployment via vLLM, SGLang, and Transformers on Hugging Face. DeepSeek also describes a public API offering low, high, and maximum reasoning tiers.
Causal Encoder-Decoder architecture
The 40-layer backbone splits into a 20-layer causal encoder and a 20-layer decoder. Following the YOCO approach, the decoder does not calculate its own global key-value cache. Instead, projection weights at each layer derive the cache from the final encoder hidden state. Prompt tokens pass only through the encoder, which nearly halves the compute required for prefill.
Sliding-window attention runs in every layer with a 128-token window. The research team calls this Decoder SWA Bounded Replay. The decoder rebuilds sliding-window states by replaying only the last 128 prompt tokens.
Compressed Sparse Attention 2
DeepSeek-V4.1-Flash uses pure Compressed Sparse Attention 2 to reduce cache size along the layer axis. Each layer is statically assigned one of three modes:
- Full: Computes its own main key-value cache, projects the indexer K from it, and selects fresh Top-512 indices.
- Reindex: Reuses the main key-value cache and indexer K from the last Full layer but rescores them with its own indexer Q.
- Reuse: Reuses both the main key-value cache and the latest Top-K indices, skipping the indexer entirely.
Every layer retains its own main Q and sliding-window key-value cache. The 18 CSA2 encoder layers use a compression ratio of 2 in three groups of six, consisting of one Full layer and five Reuse layers. The 20 decoder layers use a ratio of 1 in five groups of four: the first group contains one Full layer plus three Reuse layers, while the remaining groups contain Reindex layers plus three Reuse layers.
A Hierarchical Sparse Indexer in the decoder allows the Full layer to build a candidate pool of up to 16,384 positions. This comprises 2,048 blocks of eight tokens. Later Reindex layers score a bounded set instead of the entire context.
FP4 KV, Bounded Replay, and other extensions
The main key-value cache quantises to E2M1 with one E4M3 scale per 16 channels. This follows NVFP4 without its global scale. The team introduced this through quantization-aware training in post-training. It nearly halves storage against V4’s FP8 cache.
At the deployment level, sliding-window key-value caches no longer persist to SSD. They live in a distributed pool carved from 10% of host DRAM with a time-to-live of minutes. The global key-value cache maintains a guaranteed 72-hour lifetime. On a miss, Encoder SWA Bounded Replay recomputes only 128 tokens instead of layers times the window.
Other changes include Single-Pass mHC, which shifts input-mixing coefficients by one block so a fused Mega-mHC kernel can halve activation memory traffic. The Engram conditional memory module operates at layers 1 and 14. DSpark speculative decoding trains after pre-training with the backbone frozen. Head-wise Muon is also present. Single-token decode FLOPs rise by only one-quarter when context grows from 4K to 1M.
Training and results
Pre-training covers 45 trillion multimodal tokens at a 7:1 text-to-multimodal ratio. Sparse attention trains from scratch at 64K sequence length with no dense warmup, extending context to 1M at 34 trillion tokens. The base model matches DeepSeek-V4-Pro-Base on world knowledge and coding while using one-third of the total parameters and one-quarter of the activated parameters.
Post-training introduces no new algorithms. Gains come from large-scale synthesis of verifiable agent tasks. RL operates across heterogeneous scaffolds including Claude Code, Codex, OpenCode, Pi, mini-SWE, and DeepSeek Harness. On-policy distillation uses over 40 teachers.
Selected max-effort results:
| Benchmark | DS-V4.1-Flash | DS-V4-Flash | Opus-5 | GPT-5.6 Sol |
|---|---|---|---|---|
| Terminal-Bench 2.1 | 90.6 | 82.7 | 89.1 | 88.8 |
| DeepSWE v1.1 | 74.2 | 54.4 | 74.0 | 73.0 |
| Terminal-Bench 4.0 | 31.2 | 7.0 | 51.8 | 39.9 |
| Automation-Bench | 54.8 | 37.7 | 50.3 | 45.8 |
| GPQA Diamond | 90.9 | 89.9 | 93.4 | 94.1 |
| Codeforces (rating) | 3471 | 3289 | n/a | n/a |
What it means
Developers can run long-context tasks without the previous memory pressure. The drop in cache size reduces the need for expensive high-bandwidth memory. Agents that previously hit limits on context length can now operate within standard hardware constraints.
