Nvidia Cosmos 3 Edge is now on Hugging Face

Edge devices in factories, hospitals and warehouses require models that match data centre performance while running on limited memory. Nvidia Cosmos 3 Edge is a 4-billion-parameter open world model designed for robots and vision AI agents to understand surroundings, reason in real time and generate actions directly on edge hardware.

Developers can download the model at https://huggingface.co/nvidia/Cosmos3-Edge

The release supports high-throughput inference on Nvidia RTX PRO GPUs, Nvidia DGX systems, Nvidia GeForce RTX GPUs and Nvidia Jetson modules, including the newly announced Jetson T2000 and T3000.

As a compact vision language model, it delivers best-in-class throughput and accuracy with real-time inference.

Operating as a post-trained world action model, Cosmos 3 Edge processes robot-control resolution inputs of 640×360 observations. On Nvidia Jetson Thor, it generates 32 actions per inference while maintaining real-time control at 15 Hz.

Among models with similar size, it ranks first on VANTAGE-Bench for vision analytics and achieves state-of-the-art results for robot policy learning, setting new benchmarks for smart infrastructure and robotics.

What is world modeling

A world model tracks how an environment changes over time. It represents objects, motion, spatial relationships and the effects of actions.

When a robot reaches for an object, recognising it is only the start. The system must also understand the object’s location, gripper movement, potential contact outcomes and the most likely action to complete the task.

Cosmos 3 Edge unifies these capabilities in one on-device model. Its shared representation allows physical AI systems to understand the current world state, simulate possible futures and link those futures to specific actions.

Two transformer towers, one shared representation

The architecture combines two transformer towers:

• Autoregressive tower: Handles vision and text tokens for understanding and reasoning.
• Diffusion tower: Processes vision, audio and action tokens for prediction, generation and neural simulation.

Each tower maintains separate normalization layers and multilayer perceptrons. They share multimodal attention layers to align information across language, video, audio and action.

This design lets the model reason about a scene before generating an output. Depending on the task, Cosmos 3 produces reasoning tokens from the autoregressive tower or denoised video and action tokens from the diffusion tower.

Attention patterns adapt to each information type. Language uses causal attention, where each token attends only to previous tokens. Diffusion tokens attend more broadly to available context, supporting coherent prediction and generation.

Together, these components create a common representation of current events, potential futures and how actions influence outcomes.

A common representation for action

Physical systems describe actions differently. A vehicle may use ego pose and movement. A camera relies on motion. A robot arm tracks end effector pose, while a hand or gripper represents grasp state.

Cosmos 3 maps these different embodiments into a single action representation.

Actions are encoded as compact geometric vectors that capture:

• Translation
• Rotation
• Manipulation state

This creates a direct link between control and the visual structure of the world. The model associates changes in pixels with physical motion, spatial relationships and control inputs.

Generated video becomes more than a visual prediction. It shows how the world is expected to change in response to an action. This gives developers training data grounded in motion, control and cause and effect.

Policy mode

As a policy, Cosmos 3 predicts an action together with its expected visual consequence.

The model generates what the system should do and simulates what is likely to happen next. This connects world modeling directly to robot policy training and evaluation.

The workflow takes current state as input, processes the action, and outputs the visual consequence.

These modes enable one model to learn cause, effect and policy. Action can flow through the model in both directions, allowing Cosmos 3 Edge to predict the effects of an action or infer an action from its effects.

Prompt: Pick up the banana and put it in the plate.

The release also includes Cosmos 3 Edge Policy (DROID): A robot manipulation policy post-trained on the DROID dataset for pick-and-place tasks, with accompanying post-training scripts.

Developers can use a small cluster of H100 or Nvidia DGX Station to efficiently fine-tune Cosmos 3 Edge for target workloads before deploying to Nvidia edge and accelerated computing platforms.

Post-trained samples for improved performance

Alongside base models, the release includes reference post-trained checkpoints and training recipes to help developers adapt and optimise Cosmos 3 for their applications.

Cosmos 3 is an open world foundation model platform. As models are post-trained with high-quality, domain-specific data, their accuracy improves for specialised applications. Cosmos serves as a starting point for building domain-adapted world models using open frameworks. Post-training lets developers improve performance while preserving output quality.

To demonstrate this, the release includes the Cosmos 3 Super 4-Step Distillation checkpoint and a post-training script. This provides a faster reference implementation for a 64B model, helping developers adapt Cosmos to their domains and achieve better downstream performance.

Cosmos 3 Super 4-Step (Text-to-Image and Image-to-Video): These are distribution-matching distilled checkpoints and scripts that reduce diffusion from 35–50 denoising steps to just 4. This delivers up to 25× faster inference while preserving image and video quality and fidelity.

These examples serve as reference implementations. Using the open training scripts, developers can post-train Cosmos 3 Edge for custom robot policies or distil Cosmos 3 Super for faster image and video generation tailored to their application.

Try Cosmos 3 Edge

Download the model: https://huggingface.co/nvidia/Cosmos3-Edge

Cosmos 3 Edge connects understanding, prediction, simulation and action through a shared world representation. Developers can deploy it closer to sensors on-device. The model works as a reasoner or as an action generator.

Use open Cosmos Frameworks to customise and specialise the models.

Checkout Cosmos 3: https://huggingface.co/collections/nvidia/cosmos3

What’s ahead

Work continues on Cosmos 3 for Physical AI, with upcoming improvements in interactive world generation, driving scenario simulation and robotics policies.

Investment is also directed toward faster inference and more efficient post-training across a broader range of hardware, reducing the time and compute required to build downstream models.

This includes optimising Cosmos 3 checkpoints with open inference and optimisation frameworks such as vLLM, with more optimisations and developer tooling coming soon.