NVIDIA-accelerated Image Pipeline.
Isaac ROS Image Pipeline is a metapackage of functionality for image processing. Camera output often needs pre-processing to meet the input requirements of multiple different perception functions. This can include cropping, resizing, mirroring, correcting for lens distortion, and color space conversion. For stereo cameras, additional processing is required to produce disparity between left + right images and a point cloud for depth perception.
This package is accelerated using the GPU and specialized hardware
engines for image computation, replacing the CPU-based
image_pipeline metapackage.
Considerable effort has been made to ensure that replacing
image_pipeline with isaac_ros_image_pipeline on a Jetson or GPU
is as painless a transition as possible.
Note
Some image pre-processing functions use specialized hardware engines, which offload the GPU to make more compute available for other tasks.
Rectify corrects for lens distortion from the received camera sensor
message. The rectified image is resized to the input resolution for
disparity, using a crop before resizing to maintain image aspect ratio.
The image is color space converted to YUV from RGB using the luma
channel (the Y in YUV) to compute disparity using
SGM. This
common graph of nodes can be performed without the CPU processing a
single pixel using isaac_ros_image_pipeline; in comparison, using
image_pipeline, the CPU would process each pixel ~3 times.
The Isaac ROS Image Pipeline metapackage offloads the CPU from common image processing tasks so it can perform robotics functions best suited for the CPU.
This package uses rosidl::Buffer, a feature built into ROS 2 Lyrical, to
avoid unnecessary copies of large payloads between CPU and accelerator
memory. The CUDA buffer backend builds on this native ROS 2 feature to provide
CUDA memory storage and transport. Most applications can use standard ROS
messages and conversion packages without depending directly on a buffer
backend. See rosidl::Buffer and Buffer Backends for details.
| Sample Graph |
Input Size |
AGX Thor T5000 |
AGX Thor T4000 |
AGX Orin |
Orin Nano Super 8GB |
DGX Spark |
x86_64 w/ RTX 5090 |
x86_64 w/ RTX 5070 |
|---|---|---|---|---|---|---|---|---|
| Rectify Node |
1080p |
1150 fps 0.27 ms @ 30Hz |
683 fps 0.27 ms @ 30Hz |
1220 fps 0.28 ms @ 30Hz |
470 fps 0.58 ms @ 30Hz |
2030 fps 0.24 ms @ 30Hz |
7500 fps 0.12 ms @ 30Hz |
5680 fps 0.13 ms @ 30Hz |
| Stereo Disparity Node |
1080p |
171 fps 5.9 ms @ 30Hz |
141 fps 8.4 ms @ 30Hz |
124 fps 8.6 ms @ 30Hz |
66.1 fps 17 ms @ 30Hz |
156 fps 4.9 ms @ 30Hz |
512 fps 2.1 ms @ 30Hz |
415 fps 2.5 ms @ 30Hz |
| Stereo Disparity Graph |
1080p |
159 fps 6.2 ms @ 30Hz |
129 fps 9.2 ms @ 30Hz |
117 fps 9.3 ms @ 30Hz |
61.5 fps 19 ms @ 30Hz |
143 fps 5.3 ms @ 30Hz |
450 fps 2.2 ms @ 30Hz |
380 fps 2.6 ms @ 30Hz |
Please visit the Isaac ROS Documentation to learn how to use this repository.
isaac_ros_depth_image_procisaac_ros_image_pipelineisaac_ros_image_procisaac_ros_stereo_image_proc
Update 2026-09-21: Migrated the image pipeline nodes from NITROS to rosidl::Buffer with the CUDA buffer backend