Welcome Beam: a 501B open model you can own, customize, and scale
Reflection has released Beam, a 501-billion parameter model designed for coding and autonomous agent tasks. Teams can run this model on their own hardware or cloud infrastructure. The release includes a dedicated Rust-based harness to maximise performance.
Beam features native support for transformers, vLLM, and SGLang. The Reflection team has provided recipes for inference, supervised fine-tuning, and reinforcement learning. This post covers running the model, post-training it, and the initial results.
TL;DR
- 501B-parameter MoE LLM with 23B active parameters per token. 52 layers, 256 experts per MoE layer, 8 routed per token.
- Text in, text out with a context window of 1M tokens.
- Checkpoint formats: BF16, FP8, and NVFP4.
- Reasoning that stays in context across turns (preserve_thinking), tool calling with a dedicated grammar. The model supports five reasoning values: “low”, “medium” (default), “high”, “xhigh”, and “max”.
- Response template in the tokenizer.
- Speculative decoding drafter model with DFlash2.
- Day-0 libraries: transformers, SGLang, vLLM. Full-parameter RL with Miles.
- Ships with a Rust-based custom harness.
- Apache 2.0.
Build with Beam
Reflection is offering $10,000 in Hub credits to the best projects using Beam. You must build something and share it on the Hub.
- Build a demo using the Beam model on Spaces.
- Quantize the model for your preferred hardware.
- Generate a synthetic dataset for a specific problem.
- Create an RL environment with or for Beam.
- Benchmark Beam and share the results.
- Surprise the team.
If you build something with Beam, share the space, model, or dataset repo on the model’s page. The team will collect the links, select the best entries, and send credits to your Hub accounts.
What is Beam
Beam is a text-only model built for long-context reasoning and agentic work. It uses a sparse MoE architecture. Each token is computed by a small fraction of the network, allowing it to run at the speed of a smaller model while keeping the capacity of a large one.
Attention
The decoder has 52 layers with a hidden size of 5,120. Most layers use sliding-window attention over the previous 512 tokens. Every sixth layer and the final layer attend to the full context. This gives 43 sliding-window layers and 9 global ones.
Inside the attention block (both layer types)
The two kinds of layers handle position differently. Sliding-window layers use RoPE, since relative position matters most within a short window. Global layers use no positional encoding. Instead, each query is multiplied by a scaler that grows slowly with the number of visible tokens,
(1 + (pos + 1) / 2048) ** 0.1
. It acts as a temperature: the softmax keeps its precision as the context lengthens instead of flattening out. The implementation calls this scalable-softmax query scaling.
Attention uses grouped-query attention with 48 query heads and 8 key-value heads, with RMSNorm on queries and keys. A learned sigmoid gate, computed from the layer input, scales every channel of every head before the output projection.
Mixture of Experts
The first 3 layers use a dense SwiGLU MLP. The other 49 replace it with a Mixture-of-Experts block of 256 experts, each a SwiGLU MLP with an intermediate size of 2,560, or about 39M parameters. The experts hold 98.4% of the model’s parameters.
The router follows DeepSeek-V3’s group-limited scheme. It scores all 256 experts with a sigmoid, splits them into 8 groups of 32, keeps the 4 best groups (ranked by the sum of each group’s top two scores), then picks the top 8 experts inside them. Beam uses the same group sizes as DeepSeek-V3 but no shared expert. Expert outputs are combined with normalized router weights and scaled by 2.5.
Only 8 experts run per token, so the model computes each token with about 23B parameters. All 256 experts stay in memory, so deployment needs the memory of a 501B model but runs at the speed of a 23B one. The BF16 checkpoint is about 1 TB in safetensors format. The NVFP4 checkpoint stores most parameters in 4-bit precision for NVIDIA Blackwell GPUs, some parameters are kept in BF16 for accuracy. There are also FP8 checkpoints on the hub which are ideal for Hopper and Ampere series chips.
If you want to deploy Beam yourself, check out the vLLM or SGLang recipes below.
Residual Stream
The residual connections are the most distinctive part. A standard transformer adds each sublayer output to the stream with weight 1, so the stream grows with depth.
Beam counts the input embedding as step 1. At step
k
it adds the sublayer output with weight
1/sqrt(k)
and keeps the existing stream with weight
sqrt((k-1)/k)
. The squares of the two weights sum to one, so the stream stays at a constant scale from the first layer to the last. After all 104 additions, every sublayer output and the input embedding carry the same weight in the final hidden state.
Every sublayer is also normalized before and after, and the final logits pass through a tanh softcap at ±30, computed in FP32. The implementation fixes where it computes in FP32: the router, the SwiGLU product, the norms, RoPE, the query scaling and the softcap.
Long Context
Hybrid attention works well for long contexts. Each layer caches 2 x 8 KV heads x 128 dims = 2,048 values per token. If all 52 layers kept every token, the cache would grow by 208 KiB per token. Only the 9 global layers keep the whole sequence. The 43 sliding-window layers hold a fixed 512-token window each and stop growing. So the cache grows by 36 KiB per token, roughly a sixth of the naive rate.
| Context | If all 52 layers were global | Beam |
|---|---|---|
| 32K tokens | 6.5 GiB | 1.2 GiB |
| 128K tokens | 26 GiB | 4.6 GiB |
| Full context (1M) | 208 GiB | 36.08 GiB |
These numbers are per sequence, in BF16, with the transformers dynamic cache.
Benchmark Results
Here are the benchmark results across different precision levels, as provided by the Reflection team.
| Beam (BF16) | Beam (FP8) | Beam (NVFP4) | |
|---|---|---|---|
| Agentic Coding / Terminal | |||
| DeepSWE v1.1 | 43.3% | 43.0% | 42.7% |
| SWE Bench Pro v2-Hard | 76.5% | 78.4% | 78.6% |
| SWE Bench Pro v1 | 66.1% | 65.9% | 65.6% |
| Terminal Bench v2.1 | 80.1% | 78.9% | 78.2% |
| SWE Atlas Codebase QnA | 31.8% | 32.2% | 31.4% |
| SWEBench Multilingual | 78.3% | 78.1% | 77.5% |
| SWEBench Verified | 81.6% | 80.2% | 80.7% |
| Reasoning | |||
| AIME 2026 | 97.7% | 98.0% | 97.5% |
| HLE (no tools) | 36.0% | 36.0% | 34.8% |
| HLE (with tools) | 51.1% | 51.1% | 51.1% |
| SciCode | 49.2% | 48.3% | 46.9% |
| CriPT AA | 16.3% | 17.4% | 15.4% |
| GPQA Diamond | 90.6% | 90.3% | 89.3% |
| LiveCodeBench | 88.3% | 88.1% | 87.4% |
| Tool Calling / Search | |||
| AutomationBench – public split only |
