Kyutai has released CADRE, a family of open causal diffusion autoencoders designed to reduce action lag in interactive world models to under 230 ms. The system runs at 20 fps on a single H100 GPU, addressing a bottleneck where standard text-to-video autoencoders pack too many frames into one latent representation.
Specs and performance
- Architecture: The suite includes seven autoencoders, each with approximately 476M trainable parameters, paired with a frozen 86M DINOv3 ViT-B/16 backbone. The total world model size is 1.2B parameters.
- Resolution: Video is processed at 288×512.
- Hardware: It operates on one H100 using bf16 with a batch size of 1. Quantized versions are not yet available.
- Latency: CADRE sits on the latency-quality frontier. It outperformed MIRA, Wan 2.2, Cosmos, and LTX-2 in 42 of 48 comparisons.
- Driving quality: The FVD score reached 162.6 (CADRE-2×16×16, 1 step), compared to 599.7 for Wan 2.2.
- Robotics: Action following (ARR) scored 0.45 for CADRE-2×32×32, behind Wan 2.2’s 0.85.
How the system works
The encoder adapts the MIRA recipe. A frozen DINOv3 model extracts features from each frame patch. A trained 3D convolution then compresses these into a 64-channel latent. Only this bottleneck layer differs between the seven variants.
The decoder is a causal diffusion transformer with 24 blocks. It processes data chunk by chunk. Progressive self-distillation allows the model to decode in one or two steps during a single training phase, without adversarial loss.
Training data comes from 23 public sources, totaling roughly 76,000 hours of web, robot, driving, and game video.
Evaluation methodology
Kyutai trained the same 1.2B causal diffusion transformer on each autoencoder. Testing covered four environments: robot manipulation (ABC), human manipulation (EgoDex), driving (PhysicalAI AV), and indoor navigation (RealEstate10K). Each model observed two seconds of real video before generating 4.8 seconds.
Metrics include image quality (FID), motion quality (FVD), and action following (ARR). ARR uses a frozen inverse dynamics probe where a score of 1 matches real video perfectly.
Benchmark results
On a single H100, CADRE-2×32×32 reacts in 185 ms and sustains 23.4 fps with one decoding step. Wan 2.2 requires 317 ms, LTX-2 needs 492 ms, and Cosmos takes 605 ms. DC-AE is faster at 118 ms but operates frame-by-frame.
The delay gap widens with moving cameras. In driving, Wan 2.2’s FVD reached 599.7 while LTX-2 hit 913.3. In indoor navigation, LTX-2 ARR dropped to 0.04. Against a retrained MIRA decoder, the 1-step diffusion decoder reduced FVD by 1.4 to 2.5 times.
Four key findings
- Diffusion decoding: At high compression, perceptual regression blurs the image. Diffusion sampling restores sharpness.
- Temporal compression: Compressing time within the world model loses detail. At eight frames, ARR drops from 0.72 to 0.56.
- Camera motion: Still cameras tolerate temporal compression. Moving cameras require fewer frames and coarser spatial tokens.
- Spatial compression: Coarse tokens hurt motion first. In robot manipulation, 32× spatial compression multiplied FVD by 2.4 over 16×, while FID barely changed.
Comparison table
| Feature | CADRE-2×32×32 | CADRE-2×16×16 | MIRA recipe | Wan 2.2 VAE | Cosmos CV8x8x8 | LTX-2 VAE |
|---|---|---|---|---|---|---|
| Compression (τ×s×s) | 2×32×32 | 2×16×16 | 2×32×32 | 4×16×16 | 8×8×8 | 8×32×32 |
| Tokens per 16 frames | 1152 | 4608 | 1152 | 2304 | 4608 | 288 |
| Decoder | Causal diffusion, 1 to 2 steps | Causal diffusion, 1 to 2 steps | Perceptual regression | Causal | Causal | Non-causal, windowed |
| Action latency, 1 H100 | 185 ms | 216 ms | 175 ms | 317 ms | 605 ms | 492 ms |
| Max fps, 1 H100 | 23.4 | 17.3 | 26.6 | 34.2 | 39.0 | 87.4 |
| FVD, driving (lower is better) | 171.3 | 162.6 | 311.6 | 599.7 | 452.5 | 913.3 |
| FVD, robot manipulation | 105.0 | 47.6 | 267.1 | 52.9 | 39.5 | 69.6 |
| ARR, robot manipulation (higher is better) | 0.45 | 0.73 | 0.39 | 0.85 | 0.78 | 0.25 |
| Reconstruction PSNR (OpenVid-HD) | 22.86 | 25.12 | 24.44 | 35.02 | 31.39 | 28.24 |
| Weights license | CC BY-NC 4.0 + DINOv3 License | CC BY-NC 4.0 + DINOv3 License | Not disclosed | Apache 2.0 | NVIDIA Open Model License | LTX-2 Community License |
Latency, fps, FVD, ARR and PSNR figures come from the paper, tables 4, 6 and 7. CADRE uses one decoding step. All figures are self-reported by Kyutai.
Limitations
CADRE does not win every metric. In robot manipulation, Cosmos posted an FVD of 39.5 against 47.6 for the best 1-step CADRE. DC-AE beats CADRE-1×48×48 at the fast end for manipulation tasks. Wan 2.2 reconstructs held-out video far better, achieving 35.02 dB PSNR.
The Kyutai team argues reconstruction quality does not predict rollouts. Wan 2.2 ranked last on indoor rollout FVD.
Long latents leave visible seams between chunks. At τ = 8, boundary excess reaches 1.6 to 1.8 across environments. CADRE-4×24×24 is an outlier in navigation, with FVD 403 in driving. Each world model covers one environment at 288×512. Whether findings transfer to generalist models remains open.
Availability
The code is MIT licensed. Weights on Hugging Face are CC BY-NC 4.0, so commercial use is excluded. The release covers seven autoencoders, four longer-trained multi-resolution versions, and four world models on CADRE-2×16×16. A live demo allows users to drive the models by keyboard.
What it means
For developers building interactive simulations, CADRE removes the need to choose between speed and quality. By isolating compression as the variable, the system proves that causal diffusion decoding can handle high compression without perceptual blurring. This allows world models to react to user input within the 230 ms window humans notice, making driving and robotics applications significantly more responsive.
