Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

In this articleArchitecture detailsBenchmark ResultsInference Support Thinking Machines Lab releases Inkling 2 Mini, a 3B active-parameter model for local use Thinking Machines…

By Vane November 20, 2026 6 min read


Thinking Machines Lab releases Inkling 2 Mini, a 3B active-parameter model for local use

Thinking Machines Lab has released Inkling 2 Mini, a multimodal model designed to run on local hardware. The system uses a looped transformer architecture with approximately 26 billion physical parameters. It activates around 3 billion parameters per token. The design visits certain layers three times in a loop, effectively increasing capacity without expanding the physical model size. The model supports a context window of 1 million tokens and accepts both images and text. Weights are distributed in mixed-precision formats, specifically NVFP4 and MXFP8. A DSpark drafter is included to improve speed with minimal memory overhead.

Inkling 2 Mini is significantly smaller than the Inkling 2 Small variant. The release targets local AI setups. Initial testing suggests performance comparable to Qwen 3.8 27B. Further validation is required to confirm these results. The distribution includes support for transformers, Llama App (llama.cpp), and MLX. SGLang offers day-zero inference support for production deployments on datacenter hardware.

Architecture details

The model contains 24 decoder layers. Layers 6 through 17 execute three times per token, creating the effect of 48 logical layers within the 24 physical layers. Layers 0 to 5 and 18 to 23 run once. All three passes through the middle layers reuse the same weights.

Like Inkling 2 Small, Inkling 2 Mini routes tokens to eight experts per layer visit. The hidden size is 2,304, compared to 4,608 in the Small variant. The Mini selects experts from a pool of 128 per layer, whereas Small selects from 384.

Previous architectures, such as Nanbeige, have explored looping. Inkling 2 Mini uses a distinct approach. Every pass through the looped core begins with the output of the first six layers. Information transfers between passes via cross-pass messages. Each looped layer reads a message from the previous pass’s outputs using learned query weights. A gating controller examines both the current input and the message to decide how much to forget and how much to add. Each pass refines the output of the previous one.

  • Each six-layer block contains five sliding-window layers and one full-attention layer. The local window holds up to 1,024 tokens.
  • Attention uses 24 query heads and eight KV heads. It employs learned relative position logits, a learned sink, and an output gate. Full-attention layers use dynamic key and value convolutions.
  • The router selects eight of 128 SwiGLU experts for each token. Scores are normalised and weights scaled by 8.
  • During passes 2 and 3, each core layer attends over two residual blocks from the previous pass. Four rank-64 gates mix separate messages into the query, key, value, and MoE inputs.

Text decoder width is 2,304, and the intermediate width of each expert is 1,152. Reusing core weights reduces the number of distinct layers stored in the checkpoint. The model executes all 48 visits, and each visit maintains its own KV and short-convolution state.

A research artifact shares the same architecture but was trained to accept a variable number of passes through the loop. Setting one or two passes instead of the default three makes inference faster, though generation quality may drop slightly. We tested this variant and found it functional, but we have not evaluated it in depth.

Benchmark Results

Inkling 2 Mini ranks as state-of-the-art in its class on the Artificial Analysis Intelligence Index v5 (AAv5) and across many benchmarks. Compared with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL, Inkling 2 Mini leads in 32 benchmarks out of 48. These cover coding and autoresearch, agentic search and tool use for knowledge work, vision interpretation and creation, reasoning and knowledge, and safety and alignment. See the Inkling 2 Mini model card for the full list.

The model performs well as a local coding companion. It leads in FrontierCode v2.0, ties with Qwen3.8-27B on Terminal-Bench 4.0, and trails Qwen on AutomationBench-AA v1.0.6. Note that AA Briefcase uses an ELO rating, where higher is better.

Inference Support

Transformers

Support exists for both Inkling 2 Mini (fixed loop) and the

TML-d24-varloop-research

(variable loop) research variant.

Fixed loop with Inkling 2 Mini

You can use the

AutoModelForMultimodalLM

class.

import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "thinkingmachines/Inkling-2-Mini"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
)

messages = [{"role": "user", "content": [{"type": "text", "text": "Why is the sky blue?"}]}]

inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",
).to(model.device)
input_len = inputs["input_ids"].shape[-1]

outputs = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))

Variable loop

To set the number of loops in the research checkpoint, modify the

loop_num_passes

in

text_config

. The example below uses two loops.

import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "thinkingmachines/TML-d24-varloop-research"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    text_config={"loop_num_passes": 2},
)

messages = [{"role": "user", "content": [{"type": "text", "text": "Why is the sky blue?"}]}]

inputs = processor.apply_chat_template(
    messages, reasoning_effort="medium", tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt", 
).to(model.device)
input_len = inputs["input_ids"].shape[-1]

outputs = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))

Images and text

For an image prompt, replace messages in either example with the list below and infer as usual.

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "/path/to/image.jpg"},
            {"type": "text", "text": "What is in this image?"},
        ],
    }
]

Reasoning Effort

Pass

reasoning_effort="medium"

to

processor.apply_chat_template(...)

to set the effort for a request. The chat template adds the numeric value to the prompt as a system message. The default is

high

(0.9).

inputs = processor.apply_chat_template(
    messages, reasoning_effort="medium", tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt", 
).to(model.device)

Available levels are

none

(0.0),

minimal

(0.1),

low

(0.2),

medium

(0.7),

high

(0.9; default),

xhigh

(0.99), and

max

(0.99). You can also pass a number from 0.0 to 0.99, such as

reasoning_effort=0.35

.

Tool calling

In this example we ask our model about the current temperature at a given location. With the prompt, we also provide the tool (

get_current_temperature

) to do it. The model asks for a tool call, we call the tool on its behalf and then add the output to the next instruction set. It uses the output of the tool and provides us with the final result.

If you want to know more about tool calling, we recommend going to our official docs.

import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "thinkingmachines/Inkling-2-Mini"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

def get_current_temperature(location: str, unit: str):
"""
Get the current temperature at a location.

Args:
location: The location to get the temperature for, in the format "City, Country"
unit: The unit to return the temperature in. (choices: ["celsius", "fahrenheit"])
"""
return 22.0 # A real function would call a weather API here

def ask(messages):
inputs = processor.apply_chat_template(
messages, tools=[get_current_temperature], reasoning_effort="low", tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
output = model.generate(**inputs, max_new_tokens=1024)[0][inputs["input_ids"].shape[-1] :]
parsed = processor.parse_response(output, prefix=inputs["input_ids"][0])
return parsed

messages = [{"role": "user", "content": "Hey, what's the temperature in Paris right now?"}]

reply = ask(messages)
call = reply["tool_calls"][0]["function"]
print("Model calls:", call["name"], call["arguments"]) # the model fills in the arguments itself

temperature = get_current_temperature(**call["arguments"])
messages += [reply, {"role": "tool", "name": call["name"], "content": str(temperature)}]
print(ask(messages)["content"])

# Model calls: get_current_temperature {'location': 'Paris, France', 'unit': 'celsius'}
# Right now in Paris it's about **22°C** (rough

Scroll to Top