Thinking Machines Lab has released Inkling, a 975-billion parameter open-weights multimodal model with 41 billion active parameters. The release includes a version called Inkling-Small, which uses 276 billion parameters with 12 billion active. Both models support a context window of up to 1 million tokens.
The architecture is a Mixture-of-Experts transformer. It uses a 66-layer decoder-only structure with a sparse MoE feed-forward backbone. Each MoE layer contains 256 routed experts plus 2 shared experts. Six routed experts activate per token, while both shared experts activate on every token. A sigmoid-based router handles selection using an auxiliary-loss-free load-balancing bias. Routed and shared scores are normalised jointly before weighting combined outputs. The design largely follows DeepSeek-V3.
Attention mechanisms differ from standard conventions. Sliding-window and global layers interleave at a 5:1 ratio with 8 KV heads. Positioning uses a relative positional embedding rather than RoPE, which the lab reports extrapolates better. Short convolutions apply after key and value projections, and on residual branch outputs.
Multimodality operates without an encoder. Audio enters as dMel spectrograms, and images become 40×40 pixel patches through a four-layer hMLP. A lightweight embedding layer projects both inputs, then the decoder processes them jointly with text tokens.
Training utilised Muon for large matrix weights and Adam for other parameters on NVIDIA GB300 NVL72 systems. Post-training bootstrapped from supervised fine-tuning on synthetic data, including data generated by Kimi K2.5. Most compute went to asynchronous RL, scaled past 30 million rollouts, improving log-linearly throughout. That RL run also produced the model’s main control surface.
During RL, the research team set effort by changing the system message and adjusting per-token cost. The model learned to spend different token budgets on different rollouts. The release post sweeps effort from 0.2 to 0.99, and harnesses can set it directly. In the transformers library, the same control is exposed as a reasoning_effort argument with named levels.
Efficiency data is specific. Inkling spends one third as many tokens as Nemotron 3 Ultra for equal Terminal Bench 2.1 performance. Cost and latency become tunable per call, not fixed per model.
Alongside effort, the research team targeted trustworthiness directly.
Performance
All Inkling evaluations run at effort=0.99 and temperature 1.0, with a 256K trajectory limit for coding. Several scores are externally reported by Artificial Analysis. Against open-weights peers, the picture is quite competitive.
| Benchmark | Inkling | Nemotron 3 Ultra | Kimi K2.6 | GLM 5.2 | DeepSeek V4 Pro |
|---|---|---|---|---|---|
| HLE (text only) | 29.7% | 26.6% | 35.9% | 40.1% | 35.9% |
| AIME 2026 | 97.1% | 94.2% | 96.4% | 99.2% | 96.7% |
| GPQA Diamond | 87.2% | 86.7% | 91.1% | 89.5% | 88.8% |
| SWEBench Verified | 77.6% | 70.7% | 80.2% | 80.0% | 80.6% |
| Terminal Bench 2.1 | 63.8% | 56.4% | 71.3% | 82.7% | 64% |
| MCP Atlas | 74.1% | 44.7% | 68.1% | 77.8% | 73.2% |
| SimpleQA Verified | 43.9% | 32.4% | 38.7% | 38.1% | 57.0% |
| IFBench | 79.8% | 81.4% | 76.0% | 73.3% | 76.5% |
| FORTRESS (Adversarial) | 78.0% | 77.6% | 65.6% | 71.3% | 36.0% |
Inkling leads this open-weights group on FORTRESS Adversarial at 78.0%. It trails GLM 5.2 on Terminal Bench 2.1 by 18.9 points. It reports 73.5% on MMMU Pro and 91.4% on VoiceBench. It posts 1257 on Design Arena’s Agentic Web Dev leaderboard, a blinded human evaluation.
With numbers established, deployment becomes the practical question.
Running and Fine-Tuning Inkling
Two checkpoints ship. BF16 needs at least 2 TB aggregated VRAM: 8x NVIDIA B300 or 16x H200. NVFP4 drops that to at least 600 GB, running W4A4 on 4x B300 or W4A16 on 8x H200. Runtimes include SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face transformers.
# pip install -U transformers (5.14.0 or later)
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "thinkingmachines/Inkling" # BF16, Hopper or later
# model_id = "thinkingmachines/Inkling-NVFP4" # NVFP4, Blackwell
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
model_id, dtype="auto", device_map="auto",
)
messages = [
{"role": "user", "content": [
{"type": "audio", "audio": "support_call.wav"}, # 16kHz WAV
{"type": "text", "text": "Transcribe, then list every billing complaint."},
]},
]
inputs = processor.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
# none | minimal | low | medium | high | xhigh | max
reasoning_effort="medium",
).to(model.device)
# use_mtp enables the shipped multi-token-prediction drafter.
outputs = model.generate(**inputs, max_new_tokens=2000, use_mtp=True)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))
OpenAI-compatible serving takes one command:
vllm serve thinkingmachines/Inkling --tensor-parallel-size 8 --served-model-name inkling
For fine-tuning, Inkling is live on Tinker with 64K and 256K context options. The research team also released tml-renderers for post-training with tool calls and multimodal inputs. Hosted APIs exist via TogetherAI, Fireworks, Modal, Databricks, and Baseten.
Given those constraints, three deployment patterns follow.
Where Inkling Fits: Use Cases
- Voice-and-vision agents: A major design goal was backing the lab’s interaction models system. A support agent could ingest a 16kHz WAV callSource Read original →
