Navigation turns perception and motion into purposeful autonomy: a navigation system must continuously localize the robot, interpret changing surroundings, choose a route and avoid obstacles to reach a goal. Transferring navigation to a new robot or scene typically requires new data, simulation assets, robot interfaces, training, diagnostics and evaluation — work that is costly and hard to reproduce if repeated for every robot–scene pair.
COMPASS (Cross-Embodiment Mobility Policy via Residual RL and Skill Synthesis) is a framework that enables scalable cross-embodiment mobility by reusing expert behavior from a single embodiment. It adapts a pretrained NVIDIA X-Mobility base policy by training a residual specialist: a reinforcement-learning policy that corrects the base action for a selected robot and environment instead of relearning navigation from scratch. Multiple specialists can later be distilled into a shared cross-embodiment policy.
This article describes an agent-driven COMPASS development workflow using the Boston Dynamics Spot quadruped as the reference robot. It covers three scene sources — the built-in COMPASS warehouse (combined_multi_rack), a generated SAGE-10K scene, and an optional Omniverse NuRec reconstruction — and walks through setup, smoke testing, residual training, checkpoint evaluation, and runtime integration including optional cuVSLAM odometry.
Why use an agent-driven workflow?
An agent-driven workflow reduces developer overhead: the developer specifies the robot, scene source and navigation goal; a coding agent uses repository skills to validate dependencies, prepare assets, run smoke tests, launch training, diagnose failures and compare checkpoints. Human approval gates control scene acceptance, the one-environment smoke test and checkpoint promotion. The trained policy and robot controller run at inference time without the coding agent.
The COMPASS repository packages the development steps as skills. During development the tutorial uses Codex (or Claude Code /compass), but inference runs do not require the agent.
Reference workflow and minimum requirements
Reference path:
- Robot: Boston Dynamics Spot
- Built-in scene: combined_multi_rack (COMPASS warehouse)
- Generated scene: SAGE-10K
- Optional reconstructed scene: NVIDIA Omniverse NuRec
Recommended hardware and software (tested configuration):
- Ubuntu 22.04 or 24.04, at least 32 GB RAM
- RTX-capable NVIDIA GPU with at least 16 GB VRAM (Isaac Sim 6.0 minimum reference GPU: GeForce RTX 4080)
- Linux driver 580.95.05 (tested for Isaac Sim 6.0)
- Docker Engine 24+ with NVIDIA Container Toolkit
- Hugging Face account and read token with access to gated nvidia/COMPASS and nvidia/X-Mobility repositories
- Tested stack: NVIDIA Isaac Lab 3.0 with NVIDIA Isaac Sim 6.0
Run the Isaac Sim Compatibility Checker before installation.
Installing COMPASS and downloading assets
Clone the COMPASS repository and follow the COMPASS Handbook quick start with the repository-pinned container. Accept access to the gated nvidia/COMPASS and nvidia/X-Mobility Hugging Face repos and create a Hugging Face read token; expose the token only in the current shell and do not paste it into an agent prompt or source control. Example steps:
export HF_TOKEN=hf_xxx ./docker/run.sh assets ./docker/run.sh build source ./docker/activate
The assets step downloads registered simulation assets to ./assets/usd/ and the pretrained X-Mobility checkpoint to ./assets/x_mobility.ckpt. Authentication errors (401/403) typically indicate insufficient repository access.
Phases and reviewable evidence
Each phase produces evidence before the next phase begins:
- Validate: software and asset inventory, environment report, smoke-test log
- Prepare scene: registered scene configuration, occupancy map, visual inspection evidence
- Train: pinned command and configuration, logs, telemetry, periodic checkpoints
- Evaluate: matched protocol, standard COMPASS metrics, videos, promotion recommendation
- Package: approved checkpoint, configuration, evaluation record, artifact manifest
Approval criteria are project-specific but should answer: are required inputs present, did expected outputs appear, are there unresolved errors, and is evidence sufficient to continue?
Step 1 — Set up the COMPASS agentic workflow
Prepare the repository and expose COMPASS skills for the coding agent (Codex or Claude Code). For Codex, place skills under .agents/skills and invoke them with $compass; in Claude Code use /compass. The coding agent can clone, build, download non-secret assets and validate the stack. The developer must accept gated repository terms and enter the Hugging Face token outside the chat.
A baseline prompt for validation and a one-environment smoke test: $compass Validate the COMPASS environment for Spot. Confirm the pinned repository revision, container, GPU, Isaac Lab and Isaac Sim versions, simulation assets, and pretrained X-Mobility checkpoint. Run a one-environment smoke test, save the validation report, and stop for approval.
If runs fail, $compass-doctor performs a read-only health check and reports likely causes.
Step 2 — Choose and prepare a navigation scene
Three scene sources are supported: built-in COMPASS warehouse, a SAGE-10K generated scene, and an Omniverse NuRec reconstruction. Each path requires registration, occupancy map generation and human approval gates before training.
Path 1 — Built-in warehouse:
- Use combined_multi_rack for a fast, reproducible baseline since robot, scene and occupancy map are already registered.
Path 2 — SAGE-10K:
- SAGE-10K contains 10,000 generated indoor scenes across 50 room types. Select suitable candidate scenes (no need to download the whole dataset). The path includes two human approval gates: inspect the converted USD in Isaac Sim for geometry, materials, scale and collision meshes before registration; after registration and occupancy-map generation, approve the one-environment preview before full training.
Path 3 — Omniverse NuRec:
- Use NuRec to convert stereo RGB captures into an Isaac Sim-ready reconstruction for fine-tuning and evaluating COMPASS on a reconstruction of the deployment environment. The documented NuRec path registers the rendered scene, verifies occupancy maps and origin conventions, inspects robot clearance and runs a one-environment smoke test before training.
The hands-on tutorial continues with the SAGE-10K path so one generated scene is followed from preparation through evaluation.
Step 3 — Validate robot–scene integration
Before scaling training, run a one-environment preview: confirm Isaac Sim starts, the scene loads, Spot spawns in a valid location, camera observations are available and the robot responds to policy commands without clipping, falling or unresolved simulation errors. The coding agent should summarize preview logs and visual evidence, identify blockers and stop for human approval.
Step 4 — Train the residual specialist
After the one-environment smoke test is approved, launch the standard residual RL workflow to adapt X-Mobility to the selected robot and scene. Run training in a persistent session or scheduler, write logs and checkpoints to the configured output directory, and record the command, repository revision, scene key, configuration, checkpoint interval and stopping criteria.
Training guidance:
- Use a single environment for the smoke test; set --num_envs according to GPU memory for larger runs.
- Monitor reward components, goal progress, contacts and falls, episode terminations, throughput and GPU memory.
- Save periodic checkpoints and evaluate them under matched conditions; the final iteration is not guaranteed to be best.
- COMPASS supports distributed multi-GPU training for larger experiments; training time varies with hardware and scene complexity.
Diagnose failures using the COMPASS diagnostic workflow: map authentication errors to Hugging Face access, scene-loading or collision errors to scene preparation, camera or action-interface errors to smoke-test stage, and memory errors to environment-count or multi-GPU configuration. Preserve configuration, command, revisions, occupancy maps, logs, checkpoints and artifact manifests; require human approval before changing dependencies, assets, rewards or training settings.
Step 5 — Evaluate before promoting a checkpoint
Compare the pretrained X-Mobility base policy and residual candidate checkpoints under matched conditions (same seeds, goals, initial states, rollout length and active terminations). Standard COMPASS evaluation reports goal-reached rate, fall-down rate and travel time. Label derived analysis clearly and save matched videos and the exact evaluation command. Promote a checkpoint only after matched evidence satisfies navigation and safety gates and a human approves packaging.
Example evaluation prompt (adapt to your robot and scene): $compass Compare the pretrained X-Mobility base policy with the available Spot residual checkpoints in the selected scene under matched seeds, goals, initial states, rollout length, and active terminations. Report the standard COMPASS evaluation metrics, save matched videos and the exact evaluation command, clearly label any derived evidence, and stop for human approval before promoting or packaging a checkpoint.
Step 6 — Connect the policy to robot runtime
The trained policy is exported and the reference ROS 2 integration (compass_inference) converts front-camera images, navigation target or route and robot speed from odometry into policy inputs; it publishes forward-linear and angular-velocity commands on /cmd_vel. Validate coordinate frames, update rates, normalization, command limits, stopping behavior and the physical robot controller for deployment.
cuVSLAM odometry:
- Use NVIDIA cuVSLAM when the deployed robot lacks compatible, validated odometry in GPS-denied or GPS-intermittent environments. cuVSLAM provides camera-based odometry that can be remapped to /chassis/odom and requires an odom-to-base_link transform. cuVSLAM is not part of COMPASS training and should run as a separate, version-matched ROS 2 component. The tutorial notes optional $cuvslam-onboard and $cuvslam-troubleshoot skills for configuring and diagnosing the odometry component.
Onboarding a new embodiment:
- For robots not yet registered, $compass-newembodiment guides robot configuration, environment registration, action mapping and a one-environment visual smoke test. Onboarding is an engineering task separate from training a specialist, but it follows the same validation and approval pattern.
Export, ROS 2 integration and deployment
The tutorial stops at checkpoint evaluation. Export to ONNX, JIT or TensorRT, ROS 2 integration and physical deployment require separate validation for the target robot and runtime. Scene quality, training duration and checkpoint performance depend on embodiment, environment, reward design and available compute; the workflow does not define a universal success threshold.
Getting started
- Set up the reference environment: clone COMPASS, follow the Handbook quick start, accept gated model terms and download COMPASS simulation assets and the X-Mobility checkpoint.
- Run the agentic workflow: invoke $compass in Codex with a supported robot and built-in scene, retain approval after the smoke test, then train and evaluate the specialist under matched conditions.
- Extend and package deliberately: use $compass-newembodiment for unregistered robots and save configuration, checkpoint, logs and matched evaluation results and videos needed for subsequent engineering decisions.
Resources
- SAGE-10K dataset for generated indoor scenes
- Omniverse NuRec developer resources, COMPASS NuRec workflow and Isaac Lab NuRec guide for captured-scene reconstruction and training
- NVIDIA Isaac Sim and NVIDIA Isaac Lab documentation for the simulation and robot-learning stack
- Isaac ROS Visual SLAM documentation for optional cuVSLAM-based deployment odometry
This article presented the agent-driven COMPASS workflow and its practical steps for adapting a pretrained X-Mobility policy to new robots and scenes, emphasizing reproducibility, human approval gates and preservation of evaluation evidence.
