Zero Copy GigE Vision and GPUDirect and GPU Plug-ins with eCapture Pro

Our eSDK is the low level software programmer option for complete custom application flexibility. It has runnable source code examples which exercise all the critical functionality like efficiently grabbing frames, PTP synchronization, GPUDirect, multicast, IO triggering, and more.

Our eCapture Pro software is more application based with a focus on system deployment, rapid prototyping, and includes options for customized GPU and FPGA card processing. We handle all the heavy lifting associated with bringing images to your processing resource and do so with top performance. You focus on your application code.

Let’s watch eCapture Pro in action in the following demos. Seeing is believing.

The first demo has a single camera using GPUDirect to an NVIDIA RTXA6000 GPU. We have a polarization plug-in loaded to the GPU. Using the plug-in parameters, we first select normal viewing of this IMX253 polarization sensor based 10GigE camera. We then select the polarization direction plug-in option. The color hue is a measure of the angle of the polarized light. We then select the polarization degree plug-in option which allows us to see in brighter areas when the reflected light is more polarized.

Looking at another subject we first select normal viewing. Note the strong reflection off the windshield and the hood. Using the polarization removed plug-in option we can now improve the visibility of both windshield and hood. Using the polarization degree plug-in option we can see brighter areas on the windshield and hood which is indicating strong polarized light in these areas which are thus good candidate areas for polarization removal. Let’s look at the polarization direction for this subject.

Lastly, we look at a lens cap with a barely distinguishable surface defect. Using the polarization degree plug-in option we get a more clear view of this defect.

Let’s look at how one can create their own custom plug-in for use within eCapture Pro taking full advantage of the zero-copy, zero loss, GPUDirect functionality while writing only plug-in code. We first write our code which is cuda based for the NVIDIA GPU. With eCapture Pro, we provide a number of working examples like the polarized plug-in you have just witnessed. We then compile our code which creates the plug-in DLL which we later load within eCapture. We now load the plug-in DLL. We then instantiate this plug-in on the selected GPU and set the plug-in option for initial viewing. After connecting the desired camera to this GPU we run the plug-in to view the resultant output.

The next example shows a very general purpose OTS inference detection plug-in. We load the plug-in. We instantiate the plug-in on the desired GPU. We connect the camera to this GPU. And finally, we run the plug-in and watch the plug-in detect and classify the cups in the scene noting that this tracks the objects as they move.

The next example shows a very general purpose pattern matching plug-in. The pattern template is created in advance. As before, we load the plug-in. We instantiate the plug-in on the desired GPU. We specify the path to the pattern template. We connect the camera to this GPU. And finally, we run the plug-in to watch how the pattern is tracked on the moving circuit board.

Let’s now show how easy it is in eCapture Pro to rearrange the distribution of processing of 21 10GigE cameras with 3 RTXA4000 GPUs using a very simple brightness plugin. We start by directing all 21 camera outputs to GPU 1. Then we run the plugin. Note here we are using PTP for 1us sync accuracy. Next we disable GPUDirect with a simple click to illustrate how we can easily support GPUs or environments which do not support GPUDirect. We perform high performance copies to move images from system memory to the GPU.

Let’s repeat the previous test but this time we direct 7 camera outputs to each GPU for a balanced processing distribution.

Again, we show how to configure this without GPUDirect. We note that without GPUDirect that regardless of high performance memory transfers that we are utilizing 2x the system memory bandwidth so definitely GPUDirect is preferred.

Now let’s take a look at a high camera count system to see how this plug-in feature can be utilized in such systems with incredible ease of use.

We start with a 24 camera based system running on a single mid-range server and we first expand this to 48 cameras. We add a couple of extra network ports to the system with cameras connected via the same 48 port switch. We then add another GPU to handle the extra workload.

In the data flow view, we have loaded and instantiated a H.265 compression plug-in. Note how we have multiple GPUs in this single server and we need to direct the other 24 cameras to the second GPU. While all camera images are being compressed and stored on a single local m.2 drive we also configured one camera to stream to an RTMP client like YouTube.

Lastly, we can play back the h265 files by converting them to mp4 for example.

In a similar way, we have many systems out there with large camera counts using eCapture and the applicable plug-ins for volumetric capture applications.

This particular system is a 48 x 25GigE 25MP 60fps design storing all images to SSDs. We show here how all cameras are connected to the NICs through a 48 port switch.

In the data flow view we show how we route the cameras to specific SSDs. We can edit the parameters of one camera and then broadcast the settings to all other cameras in the system. We then start the short test recording using PTP synchronization mode with 1us accuracy.

While we are recording we view the System tab to illustrate no issues during the recording such as dropped frames, loss of PTP sync or missed saves due to drive full events – to name a few. With the recording complete, we can review the project summary.

1. Is the GPU plugin source code free for the demos shown?

Absolutely, it is included and free as is eCapture Pro for up to 2 cameras. You could take one of these examples and quickly prototype your own.

2. Which GPUs do you support with GPUDirect and zero copy?

For example, RTXA6000, A5000, A4000, RTX6000 ADA to name a few.

3. Can plugins work with a GPU that does not support GPUDirect?

Absolutely, we have an option that does a high perfomance copy to the GPU from system memory once frame is received. This amounts to 2x memory bandwidth and usable on most systems depending on overall camera system throughput.

4. Can AI/Inference/Deep learning calculations keep up with high performance multi-camera single server applications?

Absolutely, we are working on optimal trained models to illustrate this but we note we have customers already running many over 10 10GigE cameras on a single server running such inference models using GPUDirect.

5. Is your technology more focused on sports and virtual reality applications?

Absolutely not. The technology we develop is just as applicable to machine vision and other applications as this presentation has shown. Over 50% of our market is in machine vision applications.

Here is a recap of what Emergent is all about…

• 10+ Awards for innovation and pioneering the high speed GigE Vision imaging movement
• 10+ years shipping 10GigE cameras with more than 140 models
• 5+ years shipping 25GigE cameras with more than 55 models
• 2+ years shipping 100GigE cameras with more than 16 models
• Camera technology performance leader
• Focused on high-speed Ethernet/GigE Vision
• Focused on enabling the processing of high-speed image data
• Area scan and Line scan models
• UV, NIR, Polarized, Color, Mono models for multispectral applications
• Emergent eSDK for full application flexibility
• Emergent eCapture Pro for a highly comprehensive software solution
• Most comprehensive range of product and support for high-speed imaging applications
• Any speed, any resolution, any cable length
• Available NOW!

We are a multi-award winning company with a focus on high speed GigE Vision product.

We have many years shipping product ranging in speeds from 10GigE up to 100GigE.

We have a strong focus on providing end-to-end technologies and support for our customers applications.

We can fullfil most application needs.

Lastly, products presented are available now.

Here is a quick snapshot of the adoption of GigE Vision products ranging in speeds from 10GigE up to 100GigE. Emergent has shown how top performance can be achieved and opened up many markets including machine vision to the use of such technologies. Some companies are just now leveraging our efforts toward releasing 25G and higher speed products but still a ways to go to release ratified and performance products.

Figure: Emergent Vision Technologies is the first provider of cameras based on 10GigE, 25GigE, 50GigE, and 100GigE interfaces.