NVIDIA announced Cosmos 3 Edge, a 4-billion-parameter omni-model in the Cosmos 3 family designed for on-device robotic manipulation. Cosmos 3 Edge was pretrained on the same physical-world data as Cosmos 3 Nano and Cosmos 3 Super, so it starts with grounded knowledge of how objects move and interact but is small enough to run on NVIDIA Jetson AGX Thor.
Why this matters
World foundation models provide internal representations that help predict and reason about physical interactions (how objects fall, slide, or respond to contact). Large world models are often too big for direct onboard deployment. Cosmos 3 Edge addresses that gap by offering a compact model that can be post-trained into a robot action policy and run fully on the robot without offloading inference to a data-center GPU.
What the tutorial delivers
The supplied tutorial walks through post-training Cosmos 3 Edge into a DROID manipulation policy that:
- runs on Jetson Thor,
- can be evaluated in closed-loop simulation (RoboLab),
- performs real-time replanning by generating action chunks on-device.
All steps are reproducible from the open cosmos-framework repository, and the released checkpoints are available on HuggingFace.
Device memory and control latency
On-robot deployment has two main constraints: device memory and control latency. The post-trained Edge policy fits in Thor’s memory (weights are roughly 9 GB in BF16), so inference runs locally on the Jetson AGX Thor. On that platform, the DROID action policy generates each action chunk in about 1.53 seconds (at 640×540 resolution and 15 Hz), while a chunk covers roughly 2.13 seconds of robot motion. Because the next chunk is ready before the current one finishes, the arm moves continuously; the policy generates chunks and replans after each inference cycle rather than after every observation.
Closed-loop benchmark results
In RoboLab closed-loop evaluation (the Isaac Lab-Arena benchmark behind the RoboLab leaderboard), the post-trained Edge policy achieves 22.9% success across the 120-task suite. By comparison, Cosmos 3 Nano reaches 36.8% on the same benchmark; the Edge trade-off is lower inference compute and full on-device operation in exchange for a lower success rate.
Training data used for the policy
The release example trains on the nvidia/Cosmos3-DROID dataset. That dataset contains 76k successful teleoperated trajectories — about 350 hours across 86 tasks and 564 scenes — collected with a Franka Panda arm and Robotiq gripper. The data is packaged in LeRobotDataset v3.0 format at 640×360 resolution. Preparation steps include filtering idle/non-task frames, selecting successful demonstrations, and applying random cropping, rescaling and color jitter during training.
If you bring your own robot data, convert it to LeRobot Dataset v3 (per-frame camera video, joint states, gripper state, actions, task instruction). Different embodiments require their own experiment configuration specifying action space, dimensionality, camera layout and normalization. Cosmos 3 supports several embodiments (dual-arm Franka, UR, WidowX 250, LeRobot SO101); see the Cosmos 3 Edge model card for the full list.
Preconditions and compute requirements
Required items before starting include the latest Cosmos framework release, validated training hardware (NVIDIA DGX Station with GB200 or GB300 Grace Blackwell Superchip), supported CUDA/container versions (CUDA 13.0 cu130, NGC 26.06-py3), access to the Cosmos 3 Edge base checkpoint and the Cosmos3-DROID dataset, and a Hugging Face access token.
This is foundation model post-training, not a single-GPU fine-tune: the validated run uses 64 nodes of 4× GB200 for 60K iterations (about 68 hours, ~17.4K GB200-hours). The article provides a full post-training configuration (action space: 8-D absolute joint_pos, 7 joints + gripper; observation canvas: wrist 360×640 plus two exterior 180×320 views → 540×640; action chunk: 32 future actions at 15 Hz; learning rate 2e-4; global batch 8192; etc.).
Post-training workflow (overview)
High-level steps: download the dataset, convert the base checkpoint to distributed-checkpoint (DCP) format, apply the curation filter, and launch post-training. The repository includes launcher scripts; to train Edge instead of Nano you substitute EDGE_MODEL_CONFIG for NANO_MODEL_CONFIG and point BASE_CHECKPOINT_PATH to the converted Edge DCP checkpoint, then run the Edge launch script.
Deploying and running the policy on Jetson Thor
The policy server communicates via WebSocket using the OpenPI protocol: the client sends an observation dictionary, the server returns an action chunk. For Edge, the server runs natively on Jetson Thor, so host="localhost" and requests never leave the robot. Weights (≈9 GB BF16) fit onboard, allowing the policy server and the control client to run entirely on the robot without a data-center GPU in the loop.
A smoke test demonstrates the server returns a well-formed action chunk even with placeholder zeros for joint/gripper inputs. In real operation, the client reads cameras and measured joint/gripper positions, executes a prefix of the returned chunk, and then re-asks the server for the next chunk — standard practice to allow state-conditioned replanning that starts from the arm’s actual state. Starting the server with video decoding enabled also returns the policy’s imagined rollout alongside the actions for inspection.
Closed-loop simulation with RoboLab
You can validate policies in simulation before running them on hardware. RoboLab connects to the same policy server and executes each action chunk in physics across 120 language-conditioned manipulation tasks, streaming rendered observations back for a true closed loop. The tutorial shows how to clone RoboLab, build the sim image, and run tasks (single or headless multi-environment runs). Each run produces videos and a success log for analysis.
On RoboLab, the post-trained Edge policy reaches 22.9% success across the suite, demonstrating that a 4B world foundation model can serve as a practical, real-time on-device policy backbone.
Beyond robot control
Post-training methods that improve robot policies also apply to related use cases like synthetic data generation. The article cites Aigen as an example: they post-trained Cosmos to generate diverse synthetic crop and weed variations, enabling an autonomous weeding system to perform well using only 1% real-world training data. Cosmos’s open weights and framework under the OpenMDW1.1 license make post-training a flexible path to creating specialized, high-performance physical AI models.
Getting started
- Download Cosmos Edge policy weights and the Cosmos3-DROID dataset.
- Join the Cosmos Labs livestream on August 20.
- Explore Cosmos Edge on Jetson, download Cosmos 3 checkpoints, and get the code on GitHub.
The repository, model card and DROID post-training reproduction guide are maintained alongside the code and contain the latest end-to-end instructions for checkpoint conversion, environment configuration and curation-filter setup.
