ROS 2 Lyrical introduced the rosidl::Buffer abstraction for variable-length primitive array fields, and NVIDIA contributed a CUDA buffer backend that allows GPU-resident message payloads to be exchanged without serialization or host copies when runtime conditions permit. This preserves standard ROS 2 message types and node boundaries while reducing CPU-based copies and serialization overhead.
Problem and the change
GPU acceleration alone does not guarantee faster end-to-end ROS 2 graphs: when messages traverse node boundaries via CPU memory and serialization, the GPU performance gains are eroded. The combination of rosidl::Buffer and NVIDIA’s CUDA buffer backend enables publisher and subscriber pairs that meet runtime requirements to share payloads without host copies. If requirements aren’t met, ROS 2 falls back automatically to the CPU-backed path, keeping compatibility with existing nodes.
Supported RMW implementations include rmw_fastrtps_cpp and rmw_zenoh_cpp, and the optimized path requires the same host, CUDA device, Linux user, and a supported RMW implementation.
Example: migrating the DA3 TensorRT ROS 2 node
The tutorial uses the Depth Anything 3 (DA3) TensorRT node as an example. DA3 predicts metric depth from arbitrary visual inputs and already runs inference on the GPU: the node converts an incoming ROS image to an OpenCV view, runs TensorRT-based monocular depth inference, converts the resulting cv::Mat back to a ROS Image, and publishes a floating-point depth image.
Rather than changing message types or the model, the migration targets the Image.data field’s backing storage so that it can hold CUDA-managed memory. That avoids host allocations and payload-sized device-host/device-device copies at the ROS boundary while preserving the original ROS contract.
The AI agent and the migrate-node-to-rosidl-buffer skill
An AI coding agent is well suited to trace payloads through callbacks and helper libraries, find host-device boundaries, preserve the node API, and coordinate source, dependency, launch, and test changes. The migrate-node-to-rosidl-buffer skill turns that analysis into a repeatable workflow. Its steps include:
- Recording the starting revision and target ROS environment
- Confirming message field compatibility and adding cuda_buffer and cuda_buffer_backend dependencies
- Tracing each message field from receipt to publication, including CUDA calls, strides, streams, optional outputs, and ownership
- Running a read-only copy-boundary audit and inspecting results in context
- Making a per-field migration plan identifying removable copies and required materializations
- Implementing the smallest interface-preserving patch
- Verifying semantics, backend negotiation, separate-process transport, buffer lifetime, and actual memory-copy behavior
Typical code-level changes in the DA3 node
Main changes produced by the skill include:
- Adding cuda_buffer and cuda_buffer_backend to package dependencies while continuing to use existing message types like sensor_msgs/msg/Image
- Updating subscription options to accept CUDA-backed buffers: options.acceptable_buffer_backends = "cuda"
- Allocating CUDA-backed storage for Image.data with cuda_buffer_backend::allocate_buffer and using cuda_buffer_backend::from_input_buffer()/from_output_buffer() to obtain stream-aware handles for the TensorRT wrapper
After inference is enqueued on the CUDA stream, the write handle is released so an associated CUDA event can be recorded before publishing. The node still publishes with the same publish() call, but the underlying data field can now be backed by CUDA memory; middleware and the backend handle memory sharing and downstream compatibility automatically.
Keeping optional CPU work
Optional CPU-only consumers (for example point-cloud construction or debug visualization) are left intact. If enabled, they may trigger device-to-host copies and synchronization, but they do not change the representation delivered on the main depth topic. The migration preserves those optional boundaries rather than complicating the optimized path.
Build, run and verify
rosidl::Buffer was introduced in ROS 2 Lyrical, so a migrated node targets Lyrical and newer releases with supported RMW implementations. The CUDA buffer backend is provided as a ROS 2 plugin—building and sourcing the cuda_buffer_backend packages in the same workspace makes the backend available at runtime. Typical build steps shown in the tutorial are:
- git clone https://github.com/ros2/rosidl_buffer_backends.git
- colcon build --symlink-install --packages-up-to cuda_buffer_backend
- source install/setup.bash
- colcon build --symlink-install --packages-up-to depth_anything_v3
- source install/setup.bash
- export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
Use NVIDIA Nsight Systems to inspect GPU activity and memory transfers. On an eligible CUDA path, the migrated node should not show payload-sized host-to-device or device-to-host transfers at its ROS boundary. Record latency measurements before and after migration to quantify gains.
You can also verify backend negotiation on the subscriber side: msg->data.get_backend_type() should report "cuda" when both endpoints meet backend requirements. The cuda_buffer_backend APIs handle CPU fallback automatically (promoting CPU buffers to CUDA when needed), so subscriber callbacks typically do not need distinct CPU and CUDA code paths.
Testing and reproducibility
The skill helps generate small source and sink nodes for testing. One test pipeline publishes CPU-backed messages to exercise the CPU fallback; another publishes CUDA-backed messages to verify the zero-copy CUDA path. The migrated TensorRT node should accept both kinds of sources without code changes.
Deployment on NVIDIA Jetson AGX Thor
This agent-driven workflow can be applied to other CUDA-accelerated ROS 2 nodes that use variable-length primitive fields. NVIDIA Isaac ROS 5.0 integrates this workflow into a robotics software stack, and NVIDIA Jetson AGX Thor provides an edge compute platform for running the resulting graphs for perception, inference, and autonomy workloads.
How to get started
- Download NVIDIA Isaac ROS 5.0
- Review the ROS 2 Lyrical rosidl::Buffer and CUDA buffer backend documentation
- Install the migrate-node-to-rosidl-buffer agent skill used in this workflow
- Run the agent-guided migration on an existing CUDA-accelerated ROS 2 node
- Deploy and profile the migrated graph on NVIDIA Jetson AGX Thor
The rosidl::Buffer abstraction, together with the NVIDIA CUDA buffer backend and an AI-guided migration skill, enables existing CUDA-enabled nodes to exchange GPU-resident data with minimal code changes, reducing unnecessary serialization and CPU copies while preserving standard ROS 2 interfaces.
