NVIDIA outlines a complete workflow that links AI agents, asset creation, physics simulation, and cloud‑scale reinforcement learning to train the Microduck robot to navigate an obstacle course in simulation. Hermes and NVIDIA Nemotron 3.5 Bolt assist with planning, Astra generates assets, Newton provides physics, Isaac Lab runs large‑scale RL training, and RTX Spark with NVIDIA Isaac Sim is used for local evaluation. An adaptation run described in the text used 4,096 parallel environments on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU.
Why more than a walking policy is needed
A robot that navigates obstacles must perceive its environment, preserve balance, and adjust actions when obstacles block its path. Achieving these capabilities requires creating assets and environments, modeling physical interactions in simulation, and training and validating policies within those simulations.
1. Planning: from prompt to project
The workflow starts with a prompt that describes the obstacle course and Microduck’s objectives. On RTX Spark, Hermes and NVIDIA Nemotron 3.5 Bolt translate that prompt into a step‑by‑step project plan that specifies the task, available resources, expected outputs, and the stages agents should follow. The plan includes reviewing Microduck’s repository and documentation to summarize capabilities, constraints, and open questions, and defining course layout, obstacle dimensions, clearances, and intended route.
A local image‑generation model produces a photorealistic concept image of the course. Hermes and Nemotron then analyze that image and create an inventory of visible, reusable assets: each asset gets an identifier, a description, and a cropped reference image. The inventory records where details are obscured or cropped so later stages can separate visible evidence from modeling assumptions.
Hermes bundles the project plan, Microduck research and links, the concept image, the asset inventory, cropped references, and execution instructions into a handoff package, and delegates the next steps to Astra.
2. Creating simulation assets (Astra)
Astra uses the project plan and references to produce the 3D assets needed for the course. Since cropped references can be low resolution, Astra applies cloud image editing to enhance and isolate objects before modeling. It compares the enhanced references with the originals to preserve silhouettes, proportions, colors, and distinctive features; any dimensions or hidden surfaces that cannot be inferred from images are recorded as modeling assumptions.
Astra directs FreeCAD to build base geometry and an editable CAD structure, then uses Blender to add geometric detail and materials, all via programmatic execution. Generated assets are saved as editable models and exported as OpenUSD assets, with previews and output checks for review. Using SimReady Central Library (more than 2,000 OpenUSD assets) can reduce the amount of custom asset work needed.
3. Assembling the simulation environment (Isaac Sim + Newton)
Astra composes the obstacle course in NVIDIA Isaac Sim. Omniverse libraries supply RTX rendering and sensor‑simulation capabilities, while Newton models Microduck’s physical behavior, including movement and balance. The robot’s articulated structure and movement are defined in the simulator through links, joints, and actuators, which determine how policy actions translate into simulated motion.
Simulated sensors provide observations about the obstacles ahead. As Microduck attempts to walk, the simulation captures whether it maintains posture, makes progress, or stumbles and falls. The result is a complete learning environment: an agent that can act, an environment it can interact with, and feedback from those interactions for reinforcement learning.
4. Training perception and locomotion policies (Isaac Lab)
With the robot and environment prepared, NVIDIA Isaac Lab provides an open‑source framework to train and evaluate policies at scale. Isaac Lab runs many parallel Microduck simulations to generate experience used by an RL algorithm to improve a shared policy. For each task, developers define observations, available actions, and reward signals, and implement the task using Isaac Lab APIs.
Newton simulates how Microduck moves and interacts with ground and obstacles in response to policy actions. Training will typically encounter unstable steps, collisions, and falls; the algorithm uses those outcomes together with the reward function to push the policy toward more successful behavior. The simulation loop is: simulated observations → policy actions → simulated motion → updated observations.
Parallel simulation accelerates experience collection. In the described experiment, ramps and uneven surfaces were used as obstacles: Microduck had to traverse a curved ramp while maintaining balance. The training run adapted a pretrained locomotion policy to ramps and obstacles using head‑camera depth and robot‑state observations, executing in 4,096 parallel environments on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU.
5. Validating the trained policy on RTX Spark
After training, the policy is validated locally in Isaac Sim on an RTX Spark. During evaluation the policy is fixed: Microduck receives simulated observations, predicts actions, moves through the environment, and the team checks whether it detects obstacles, adjusts gait, and completes the course. Validation metrics include course completion, falls, collisions, and situations where progress halts. Paired with rollout videos, these metrics guide further iterations of environment or training configurations.
6. Preparing for physical deployment (sim‑to‑real)
When a policy performs reliably in simulation, the next step is to run it on the robot’s onboard computer. The article documents the simulation‑side work leading to that point but does not detail hardware deployment.
Isaac Lab supports Better Actuator Models (BAM) to model servo response characteristics such as command delays, mechanical backlash, and voltage drop under load. Matching these models to measured Microduck behavior can reduce the sim‑to‑real gap. Domain randomization varies physical parameters and observation noise across realistic ranges so a policy learns robustness to differences between simulation and reality. Hardware tests then reveal where additional simulation or training adjustments are required.
Purpose and applicability
This workflow ties AI‑assisted planning, asset creation, physics simulation, cloud‑scale RL training, and local validation into a repeatable pipeline. The steps and tools are adaptable to other robots and tasks and provide a foundation for later physical deployment.
Platforms and resources mentioned
- SimReady Central Library (OpenUSD assets)
- Omniverse libraries (rendering and sensor simulation)
- RTX Spark (local development and evaluation)
- Isaac Sim (robot and environment simulation)
- Newton (physics simulation)
- Isaac Lab (reinforcement and imitation learning training)
- Isaac Lab Learning Path and Microduck example environments
Acknowledgements
The workflow was developed in collaboration across NVIDIA teams, including Aaron Young, Antoine Richard, Chase Telegin, Gavriel State, Hammad Mazhar, Kelly Guo, Milad Rakhsha, Oyindamola Omotuyi, Pierre Fleau, Rishabh Chadha, Rogelio Olguin, Ruben D’Sa, and Yenal Kal.
