NVIDIA has published ovrtx as a prerelease SDK on GitHub: a lightweight C and Python library that lets developers run RTX‑based sensor simulation (camera, lidar, radar, semantic segmentation and other sensor outputs) from OpenUSD scenes inside their existing applications. ovrtx is part of the NVIDIA Omniverse libraries, which are now included in the NVIDIA Agent Toolkit. The library is intended to let teams produce physically grounded sensor outputs and return them into the application loop they control, without forcing a wholesale redesign of their software stack.
Practical uses
ovrtx targets workflows where 3D visualization, physical AI, robotics, autonomous vehicles, product design or industrial digital twins require realistic sensor data and path‑traced rendering. It is designed to integrate into existing OpenUSD pipelines, Blender workflows, CAD systems and domain‑specific apps so applications and AI agents can call a contained service to inspect scenes, generate sensor outputs, test changes and consume reliable evidence.
How ovrtx fits in the Omniverse libraries
ovrtx provides the RTX sensor simulation layer; ovstage supplies a shared runtime stage for OpenUSD scene data so other libraries (for example ovphysx, ovstream, ovstorage) can read and write the same scene state. This avoids separate scene copies and enables cross‑library exchange of transforms, simulation state and prim identities.
Integrating ovrtx into an application
The integration philosophy is simple: the host application keeps ownership of the main loop, UI and data model; ovrtx produces requested render and sensor outputs on demand. A typical integration follows these steps:
- Create and configure an ovrtx renderer instance.
- Load or reference OpenUSD content from the app’s pipeline.
- Define RenderProducts and RenderVars for the needed outputs (camera, lidar, radar, semantic maps, etc.).
- Step rendering or sensor simulation from the application loop, passing delta_time.
- Map results back to CPU or GPU memory for display, save, analysis or downstream consumption.
ovrtx exposes both C and Python APIs. The host calls renderer.step(...) or step_async(...) inside its loop and then maps each RenderVar as a tensor on CPU or GPU (DLPack/NumPy/PyTorch are supported patterns).
Core components
- Omniverse RTX Renderer: leverages NVIDIA RTX hardware (RT Cores, Tensor Cores) for real‑time, physically‑based ray tracing and sensor modeling.
- ovstage: shared USD runtime that provides consistent scene read/write/query APIs across CPU and GPU memory.
- OpenUSD: the foundational data layer for 3D scene representation—geometry, photometric lights, OpenPBR materials, cameras and sensors.
- RenderProducts / RenderVars: USD‑native constructs to declare the exact outputs (types, resolutions, formats) the renderer should produce.
When to combine ovrtx with other libraries
- ovrtx alone: when you need RTX sensor simulation, viewport output or visualization from OpenUSD.
- ovrtx + ovphysx: when physics simulation (gravity, collisions) should feed rendered or sensor outputs.
- ovrtx + ovstream: when you need to stream rendered frames to a browser or remote viewer.
- ovrtx + ovstorage: when managing OpenUSD scene data across storage and sync workflows.
- ovstage + ovrtx/ovphysx: to share runtime scene state across rendering, physics and application code.
Examples and reference materials
NVIDIA publishes the NVIDIA‑Omniverse/ovrtx GitHub repository with direct SDK examples, setup instructions and API references. Omniverse‑Labs hosts experimental samples, integration patterns and proof‑of‑concept apps illustrating how the libraries work together. Representative examples:
- Minimal "hello world": load a USD scene, render a single image, and write a PNG or open it in a viewer; useful as a smoke test before adding more app logic.
- Interactive USD viewer: combine ovrtx with a viewer layer (Omniverse Realtime Viewer skill) and ovstream for browser streaming; the vulkan‑interop example shows low‑level Vulkan/CUDA viewport integration.
- ovrtx + ovphysx sample: a two‑process design where an ovphysx worker computes physics and streams pose updates back to the ovrtx viewer for visualization.
- SimReady Blender workflow: shown at SIGGRAPH 2026, a blueprint for integrating Omniverse libraries (including ovrtx) into Blender to add agent‑ready simulation and validation while keeping creators in control.
Cloud‑native CAD example: PTC Onshape
The PTC Onshape Render Studio team explored ovrtx in cloud‑native CAD and PDM workflows to evaluate how RTX rendering can integrate into existing service and app layers without changing the product data model or user experience. The PTC work focuses on three requirements: application‑controlled rendering, cloud‑native integration and OpenUSD‑based scene exchange. Brendan Hack, Senior Director of Software Development at PTC Onshape, commented: “ovrtx gives us a lower‑level API path for bringing NVIDIA RTX into systems cloud‑native design tools while keeping the application architecture and product experience in our control.”
Getting started
- Explore the NVIDIA‑Omniverse/ovrtx GitHub repo for SDK binaries, examples and API documentation.
- Review Omniverse‑Labs for early experimental samples and agent‑assisted integration walkthroughs.
- Use NVIDIA Agent Skills to automate repeatable setup, simulation and validation tasks.
- For local deployment, NemoClaw packages (including the Nemotron 3 Ultra model, a secure agent runtime and Omniverse libraries) can be installed on NVIDIA DGX Station in a short setup process to create a local AI + simulation workstation.
Summary
ovrtx offers a practical, embeddable path to add RTX sensor simulation and physically based render outputs into existing OpenUSD workflows. As a prerelease on GitHub and as part of the Omniverse libraries within the NVIDIA Agent Toolkit, it can be used alone for rendering and sensors or combined with ovphysx, ovstream and ovstorage to create integrated, application‑controlled simulation pipelines.
