Emergent Vision Technologies 25GigE Camera Helps Power NVIDIA AI Medical Devices
At the 2022 GTC AI developer conference in September, NVIDIA announced that its Clara Holoscan SDK AI computing platform for medical devices achieved ultra-high-speed frame rates using Emergent Vision Technologies’ Bolt 25GigE cameras. According to NVIDIA, the technologies will provide new levels of real-time awareness and control for surgical guidance.
Low Latency, Low Jitter at High Speeds
Operating rooms require reliable high-speed surgical video streams with low latency. Ultra-high-speed cameras offer all of the above, helping power AI applications for surgical navigation, image-guided intervention such as endoscope, and medical robots. Using Emergent’s Bolt HB-9000-G 25GigE camera, NVIDIA captured 4K video at 240 fps for its Clara Holoscan SDK v0.3, enabling developers to combine data from several sensors and build AI applications that can provide surgical guidance.
Through its high-speed 25GigE SFP28 interface, the HB-9000-G reaches frame rates of up to 290 fps at full 4200 x 2160 resolution. The HB-9000-G features Gpixel’s 9.07MP GMAX2509 CMOS image sensor while also offering the low-latency and low-jitter capabilities that are crucial to NVIDIA’s Clara Holoscan SDK.
Unmatched Edge AI Performance
High-speed GigE Vision cameras from Emergent Vision Technologies can be integrated with GPU support. GPUDirect technology is available to eliminate host memory and CPU interaction. NVIDIA’s GPUDirect allows network adapters and storage drivers to directly read and write to/from GPU memory, which eliminates unnecessary memory copies, decreasing CPU overheads, and reducing latency. Emergent Vision Technologies’ 10Gige, 25GigE, and 100GigE network interface cards (NICs) support GPUDirect technology.
In this application, NVIDIA leveraged the ConnectX SmartNIC and NVIDIA Rivermax SDK with GPUDirect RDMA that bypasses the CPU to enable high-speed camera input. Doing so allowed for high-speed Ethernet output of data from the cameras into the AI system, which resulted in unmatched performance for edge AI, according to NVIDIA. The company demonstrated this capability by integrating Emergent cameras with Clara Holoscan, and the results spoke for themselves.
Traditional GStreamer and OpenGL-based endoscopy pipelines have an end-to-end latency of 220 ms on a 1080 p 60 Hz stream, but the Clara Holoscan high-speed pipelines delivered an end-to-end latency of only 10 ms on a 4K 240 Hz stream. Furthermore, the NVIDIA team achieved a latency as low as 8 ms by leveraging the NVIDIA IGX Orin edge AI computing platform. When streaming data at 4K 60 Hz at less than 50 ms on the IGX Orin, it’s possible to run 15 concurrent AI video streams and 30 concurrent models.