Mistral Large 4 is now available on Hugging Face

Mistral Large 4, nicknamed Le Chonk, is a multimodal, multilingual Mixture-of-Experts model from Mistral. It accepts text and images as input and outputs text. The model has approximately 1 trillion total parameters and 49 billion active parameters, with a context window of 1 million tokens.

The release includes two official variants. GGUF quantisations created by the Unsloth and llama.cpp teams are also available. The model is distributed under the Mistral Ecosystem Open-Weight License, an MIT derivative that sets limits on model-as-a-service offerings.

Mistral Large 4 performs particularly well in cybersecurity, coding agents, and multimodal tasks. This post covers the model’s architecture, capabilities, and technical details.

What is new with Mistral Large 4

Mistral Large 4 combines a large pool of experts with sparse attention and a Pixtral vision encoder. The language model selects a small group of experts for each token. Its attention indexer selects which earlier token positions to attend to. These are two different forms of sparsity, where one selects feed-forward networks and the other selects context positions.

You can find all the checkpoints in the Mistral Large 4 collection.

Cybersecurity Capabilities

Mistral Large 4 scores 82% on the Artificial Analysis Cyber Index test. This test asks a model to reproduce a real vulnerability in open-source software and then patch the software. Mistral reports that several closed frontier models score near zero on the same test because they refuse to attempt the work. This is important for defenders because fixing a vulnerability usually requires reproducing the vulnerability. In Mistral’s internal testing, Mistral Large 4 was also useful for malware analysis, vulnerability prioritisation and writing detection rules, which the model was not explicitly trained for.

Mistral mainly positions the model for defensive security work. Vulnerability research and incident response often appear like attacks in model input, so closed source filters can block them. A team that depends on an API can also lose access in the middle of an incident. Crucially, open weight models put both the usage policy and the deployment under the user’s control. The model still declines harmful requests: on cyber prompts from JailbreakBench, StrongREJECT and AgentHarm, its average refusal rate is higher than any other open weight model.

For community builders, an open weight model like Mistral Large 4 enables a load of downstream use cases. You could build your own cyber system around your code base, crash logs, and malware samples. For example, with TRL you could fine-tune smaller models from Mistral Large 4 traces patching your code base. In fact, reproduce-then-patch tasks use a verifiable reward, so it fits with RL environments that could be used to further RL smaller security models. Open weights also let researchers measure the model’s refusal behaviour directly instead of relying on the provider’s description.

In short, cyber capabilities like ML4’s in the open are a huge boost for the community.

We have interviewed Pierre Stock about the cyber capabilities of the model, you can watch it here.

Architecture

Mistral Large 4’s text decoder has 72 layers, a hidden dimension of 6,144, and 64 attention heads. Its attention blocks combine Multi-head Latent Attention (MLA) with a DeepSeek Sparse Attention (DSA) indexer, a direction found in other models such as GLM 5.3. MLA keeps a compressed KV representation, and DSA selects up to 2,048 key positions for each query.

For each decoder block, there is a 3:1 RoPE to NoPE pattern. Three decoder layers employ RoPE and every fourth layer uses NoPE. The model also shares KV state across layers. The first decoder layer uses a dense feed-forward network. The other 71 layers use MoE blocks, in which a softmax router selects four of the 128 experts for each token. One shared expert always runs alongside the selected experts, for a total of 5 active experts per token.

Regarding RoPE, Mistral uses LeRope, a YaRN-scaled RoPE with one learned multiplier per frequency.

Vision encoder

Mistral Large 4 employs a 1.6B Pixtral encoder. Pixtral has 30 layers, a hidden dimension of 2,048, and 16 attention heads. It uses 14 x 14 patches and a patch-merge projector with spatial merge size 2. You can see the encoder pipeline below.

Benchmark Results

Mistral Large 4 delivers strong benchmark performance across coding, cybersecurity, multimodal understanding, and agentic workflows, positioning it as a competitive addition to the open-weight ecosystem. As of October 29, 2026, the model is the fifth best open weights model on Artificial Analysis Intelligence index.

On coding, ML4 achieves 61.7% on DeepSWE v1.1, 59.4% on SWE-Atlas-QnA, and 28.3% on Terminal-Bench 4. Its combined Coding Agent Index score of 49.8% places it ahead of DeepSeek V4 Pro 0813 and Qwen3.8 Max.

Coding capabilities of ML4 extend to knowledge work, with 59.9% on AutomationBench and 1,393 Elo on AA-Briefcase, which evaluate business workflows and long-horizon professional tasks.

Cybersecurity is one of Mistral Large 4’s standout capabilities, with strong results across real-world vulnerability assessment and security competition tasks. Three benchmarks highlight the model’s cyber performance:

• Artificial Analysis Cyber Index: ML4 ranks among the top five models globally on this independent evaluation of models’ ability to find and fix software vulnerabilities.
• Vulnerability reproduction and patching: It achieves 82%, the highest reported score on this test within the index. The task requires reproducing a real vulnerability in open-source software and then patching it, testing both investigation and remediation.
• Cybench: ML4 solves 93% of a suite of 40 challenges drawn from security competitions, demonstrating strong performance on hands-on security problems.

Mistral Large 4 also demonstrates 93.3% attack resistance on Lakera’s B3 benchmark, adding evidence of robustness against prompt-injection attacks. Beyond benchmarks, internal testing showed promising results in malware analysis, vulnerability prioritization, and detection-rule generation.

Inference Support

Mistral Large 4 is shipped with day-0 support for transformers, vLLM and llama.cpp. GGUF quants are available from GGUF and the llama.cpp team. We provide options to deploy remotely with Inference Endpoints, and use serverless inference through Inference Providers.

Transformers

The Transformers implementation provides AutoModelForMultimodalLM, with AutoProcessor preparing the conversation and image inputs. Mistral Large 4 can take low, medium, high as reasoning effort. The default is high, and note that reasoning cannot be disabled, so using reasoning_effort: none would still use high.

To run the model, upgrade the transformers package

pip install --upgrade transformers

import os
from transformers import AutoProcessor, AutoModelForMultimodalLM, DistributedConfig

distributed_config = DistributedConfig(
    tp_size=int(os.environ["WORLD_SIZE"]),
    ep_size=int(os.environ["WORLD_SIZE"]),
)

model_id = "mistralai/Mistral-Large-4-1T-A52B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    distributed_config=distributed_config,
)

# only text
text_messages = [
    {
        "role": "user",
        "content": "Who are you?"
    }
]

# text + image
multimodal_messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg",
            },
            {
                "type": "text",
                "text": "How many cats are in the image?",
            },
        ],
    },
]

for messages in [text_messages, multimodal_messages]:
    inputs = processor.apply_chat_template(
        messages,
        add_generation_prompt=True,
        tokenize=True,
        return_dict=True,
        return_tensors="pt",
        reasoning_effort="medium",
    ).to(model.device).to(model.dtype)
    input_len = inputs["input_ids"].shape[-1]

    outputs = model.generate(**inputs, max_new_tokens=100)
    if int(os.environ["RANK"]) == 0:
        print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))

You can run the script above by saving it in a python file script.py and then run

torchrun --nproc-per-node 8 script.py

vLLM

vLLM supports the high-quality NVFP4 Mistral Large 4 checkpoint, which has a smaller memory footprint and requires NVIDIA Blackwell GPUs. The model also ships with a DSpark drafter for additional speed-ups with vLLM. Launch the server like this:

vllm serve mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
  --config-format mistral \
  --load-format mistral \
  --tensor-parallel-size 8 \
  --decode-context-parallel-size 8 \
  --enable-expert-parallel \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.85 \
  --enable-prefix-caching \
  --enforce-eager \
  --mm_processor_cache_gb 0.0

Once the server is up and running, you can run the following.

curl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"What is 17*23?"}],"max_tokens":512}'

Llama.cpp

Llama.cpp and Llama App already support Mistral Large 4.Source Read original →