LeRobot v0.6.0: Imagine, Evaluate, Improve
The release adds world model policies, a new wave of vision-language-action models, and a unified API for reward functions, alongside faster data loading and six new simulation benchmarks.
World models: policies that imagine
The robotics community is testing whether world models actually help robot policies. Version 0.6.0 introduces three policies to answer that question. Each one learns to imagine the future during training, using different methods to keep the cost low.
VLA-JEPA
VLA-JEPA trains a compact VLA on Qwen3-VL-2B to predict the future in latent space while it learns to act. During training, a JEPA world model anticipates upcoming frames from the model’s own actions. The system removes the world model at inference time, so you get world-model supervision at zero extra cost. Three ready-to-use checkpoints are on the Hub, including a DROID-pretrained base for fine-tuning:
lerobot-train \
--policy.path=lerobot/VLA-JEPA-Pretrain \
--dataset.repo_id=${HF_USER}/my_dataset \
--policy.repo_id=${HF_USER}/my_finetuned_policyCheck the VLA-JEPA documentation and the paper for more details.
LingBot-VA
LingBot-VA is an autoregressive video-action model that predicts future video and actions together, chunk by chunk. It feeds real observations back in to keep its imagination grounded. You can save what the robot imagined using --policy.save_predicted_video=true and compare it with what actually happened. Inference runs on a single 24–32 GB GPU. Read the documentation and paper for technical details.
FastWAM
FastWAM pairs a ~5B video-generation expert with a compact action expert in a single network. The model learns to dream its own rollouts. At inference it skips the dreaming entirely and directly denoises action chunks. Fine-tune it from lerobot/fastwam_base, and read the documentation for more information.
VLAs: the model zoo keeps growing
GR00T N1.7
We upgraded the NVIDIA GR00T integration to GR00T N1.7, the newest open generation of NVIDIA’s cross-embodiment foundation model. N1.7 swaps the previous VLM for Cosmos-Reason2-2B, built on Qwen3-VL, feeding a flow-matching action head. The integration is parity-tested against NVIDIA’s original Isaac-GR00T implementation, using the same inputs and outputs. Flash-attention is now optional, so pip install 'lerobot[groot]' works out of the box, and you can load NVIDIA’s published checkpoints directly.
GR00T N1.7 replaces N1.5 in LeRobot. If you need N1.5, pin
lerobot==0.5.1.
MolmoAct2
MolmoAct2, the Allen Institute for AI’s vision-language-action model, is now available in LeRobot with the full lifecycle covered: fine-tuning, evaluation, and real-robot deployment. Ready-made checkpoints with calibration correction baked in allow you to run it zero-shot on an SO-100/101:
lerobot-rollout \
--policy.path=lerobot/MolmoAct2-SO100_101-LeRobot \
--robot.type=so100_follower \
--robot.port=/dev/ttyACM0 \
--robot.cameras='{cam0: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, cam1: {type: opencv, index_or_path: 2, width: 640, height: 480, fps: 30}}' \
--task="pick up the red cube" --duration=30Inference fits in ~12 GB at bf16, and LoRA fine-tuning fits on a single 24 GB GPU. See the MolmoAct2 documentation for the full deployment guide.
EO-1
EO-1, a VLA pretrained upstream on interleaved vision-text-action data, joins LeRobot. It uses a Qwen2.5-VL-3B backbone with a flow-matching action head, contributed by one of the paper’s authors. Train it with the standard lerobot-train workflow using --policy.type=eo1. Read the documentation and paper for details.
Multitask DiT
The Multitask Diffusion Transformer policy brings the TRI Large Behavior Models recipe to LeRobot. It is a ~450M-parameter diffusion transformer conditioned on CLIP vision and language embeddings, so one model learns many tasks selected via natural language. It supports both diffusion and flow-matching objectives and is small enough to train yourself. See the documentation.
EVO1
VLAs do not have to be huge. EVO1 packs its policy into 0.77B parameters, using an InternVL3-1B backbone with a flow-matching action head. It is small enough to fine-tune and run in real time on modest GPUs. It ships with two-stage fine-tuning and Real-Time Chunking support out of the box. See the EVO1 documentation and the paper.
Reward models: knowing when your robot succeeds
Success detection and progress estimation were missing pieces in the robot learning loop. Version 0.6.0 gives them a home. LeRobot now has a unified reward models API (lerobot.rewards), mirroring the policies API, with four reward models behind one interface: the HIL-SERL reward classifier, SARM, and two new additions.
Robometer
Robometer is a pretrained, general-purpose reward model. Point lerobot/Robometer-4B at any LeRobot dataset and it scores task progress and success from raw video plus a language instruction, with no task-specific training required. It is built on Qwen3-VL-4B and trained via trajectory comparisons over a dataset of more than one million robot trajectories (RSS 2026 paper).
TOPReward
TOPReward goes fully zero-shot with no reward weights. It wraps an off-the-shelf VLM (Qwen3-VL) and reads the log-probability of the token “True” given the trajectory video and the task instruction. Any capable VLM becomes a reward function.
Both ship with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning (RA-BC), dataset quality inspection, and progress-overlay videos. Check the Robometer and TOPReward docs.
Datasets: faster loading, richer data
Your codec, your rules
Recording is no longer stuck with one hard-coded codec. The new --dataset.rgb_encoder.* options expose the full encoding surface (codec, quality, pixel format, GOP, presets). The vcodec=auto option probes for hardware encoders like NVENC, VideoToolbox, VAAPI, and QSV before falling back to the default software AV1 encoder. For existing datasets, one command re-encodes everything:
lerobot-edit-dataset \
--repo_id ${HF_USER}/my_dataset \
--operation.type reencode_videos \
--operation.rgb_encoder.vcodec h264 \
--operation.rgb_encoder.crf 23Full details are in the video encoding documentation.
Depth support, end to end
Plug in an Intel RealSense, set use_depth: true, and LeRobot records depth maps end to end. It captures them in millimeters, compresses them as compact 12-bit depth video streams alongside your RGB cameras, and decodes them back to physical units at training time. Depth renders live during recording and in lerobot-dataset-viz. It works across SO-100/101, Koch, OpenArm, reBot, Unitree G1 and more.
Language annotations at scale
Your dataset stops being one task string per episode. LeRobot datasets now natively store rich language annotations (timestamped subtasks, plans, memory, corrections, speech, and per-camera VQA pairs). The new lerobot-annotate CLI fills them in automatically using a VLM that watches your episodes:
lerobot-annotate \
--repo_id=${HF_USER}/my_dataset \
--new_repo_id=${HF_USER}/my_dataset_annotated \
--vlm.model_id=Qwen/Qwen2.5-VL-7B-Instruct \
--push_to_hub=trueA YAML recipe layer then renders these annotations into chat-style training messages at sample time. Scale it up with HF Jobs, and read the annotation pipeline docs to learn more.
Up to 2x faster data loading
Training on video datasets is now up to ~2x faster out of the box. Multi-camera frames decode in parallel, dataloader workers ship compact uint8 frames (4x less memory between processes), and persistent workers keep decoder caches alive across epochs. Loading a subset of a large dataset (episodes=[...]) went from minutes to milliseconds (275 s down to 0.06 s in our benchmark). Sampling is also deterministic and resumable now, so interrupted trainings restart sample-exact.
Benchmarks: one CLI to evaluate them all
Version 0.5.0 planted the flag on LeRobot as an evaluation hub for VLAs; version 0.6.0 makes it the real deal with six new simulation benchmarks. All are runnable through the same lerobot-eval CLI, each with a docs page, a Docker image, and a SmolVLA baseline checkpoint smoke-tested in CI:
- LIBERO-plus stress-tests VLAs with roughly 10,000 perturbed variants of LIBERO across seven axes, from lighting and camera viewpoints to rewritten instructions. It tells you when a policy breaks.
- RoboTwin 2.0 covers 50 bimanual manipulation tasks on SAPIEN with heavy domain randomization, and comes with more than 100k ready-to-train trajectories on the Hub.
- RoboCasa3Source Read original →
