Case Study: 100GigE Cameras Help Transport Sports Fans Into Immersive Virtual Reality Experiences
Technological advancements have been changing the way fans consume sports for more than 100 years. In 1921 the first baseball game was broadcast over the radio. Eighteen years later, the first game was aired in grainy black and white on a nascent technology called television. Eighty-five years later, fans can watch baseball and any other live sport in stunning high definition on their televisions, tablets, and smartphones.
Recent technological advancements have even made it possible for fans to insert themselves into game action in virtual reality (VR), but challenges exist when it comes to creating captivating, high-quality VR experiences. These include capturing images at high resolutions and high speeds without dropping frames, and moving, analyzing, processing, and storing vast amounts of data. But by leveraging innovative high-speed cameras and novel AI techniques, sports VR company Sport Vision Technology (SVT) has created an immersive VR world in which fans can experience the game from multiple perspectives — even from the ball’s perspective!
Figure 1: Sport Vision Technology’s capture system can be deployed for a range of different sporting events.
See Sports From Your Preferred Perspective
Staffed by a group of AI enthusiasts, SVT aims to deliver a new level of sports and entertainment spectatorship through immersive VR experiences. Fans can experience events up close and personal, with unprecedented levels of participation according to the company. In addition to providing the fan experience, SVT footage can be used to analyze player movements for mechanical improvements and training purposes. SVT’s capture technology also extends beyond the sports arena says Sebastian Konkol, CTO of SVT.
“In firefighting, for example, if the client wants to analyze where its employees are and how they move and behave in a given scenario or how they use equipment, our system can be leveraged here as well,” he said.
Ultimately, however, SVT’s capture technology lets fans see their favorite games from their preferred perspective, which could be a paradigm shift when it comes to watching sports on television. Of course, a lot of work must happen before a fan can put on their Meta Quest 3 or Apple Vision Pro VR headset to watch their favorite soccer team play.
Leveraging AI, High-Speed Advancements
Figure 2: 100GigE cameras from Emergent Vision Technologies allowed SVT to conquer previous speed and resolution related challenges.
SVT software leverages AI techniques — including convolutional neural networks — to find people, identify postures, and cut figures out of the image for digital re-creation. This process starts with setting up a data feed from the cameras — which are synchronized via IEEE 1588 PTP and integrated with the SVT software using Emergent eCapture Pro software — and performing a 2D posture analysis. From there, software triangulates the 3D representation of the 2D posture. Smoothing and stabilizing algorithms are then applied to avoid a shaky video stream, according to Gennadii Semenyshyn, product director at SVT.
“From here, the software applies inverse kinematics algorithms within Unreal Engine to estimate full body pose, and users can then visualize and save the data for later use,” he said.
Figure 3: A look at the SVT software as the system captures a teqball game.
The system leverages an NVIDIA A100 GPU inside a server, the CPU of which is tasked with data acquisition from the camera which — along with other background and system processes — takes only about 5% of the total processing workload, while image processing on the GPU utilizes its maximum potential, according to Semenyshyn. SVT leveraged Emergent’s eSDK software to utilize GPUDirect technology, which enables the transfer of images from the camera directly to the GPU, which bypasses system memory and the CPU, delivering zero-copy and zero-loss imaging capabilities.
This short animation illustrates the zero-transfer process using GPUDirect, which completely bypasses the memory and utilizes only the PCIe end-points of the CPU for 0% memory bandwidth and 0% CPU utilization.
GigE Vision Cameras Deliver
Previous attempts using other industrial cameras produced a result that took fans out of the experience for several reasons, says Konkol.
“In previous designs with other cameras, customers were sending equipment back because the cameras were producing hot pixels and overall were just not sufficient for the application,” he said. “Resolution also became an issue because at some point we were using around 80 low-res cameras. With Emergent cameras, however, it drastically reduces the number of cameras needed and the overall size of the system.”
He continued, “In addition, a lot of other systems have boasted centimeter resolution, but based on physics, this is not possible. Emergent cameras deliver unparalleled resolution of about 1 centimeter per pixel resolution at 100 meters, and with other systems this could require 100-some odd cameras.”
“One of the driving forces behind Emergent is helping our customers solve their difficult high-speed, high-resolution imaging challenges,” said John Ilett, president and CTO at Emergent Vision Technologies. “With SVT, it’s been extremely rewarding to help them bring their vision to life and I look forward to seeing their technology grow and transform the way people enjoy sports.”
Figure 4: A teqball game is recreated digitally using SVT’s system.
SVT’s system is also mobile, so the team wanted resilient equipment that does not require a lot of other components, such as copper wires, repeaters, and frame grabbers, which also add cost. As a result, SVT steered clear of CoaXPress cameras in favor of GigE Vision, which allows the company to put a few kilometers of fiber around a soccer pitch.
“Overall, using the high-resolution, high-speed Emergent cameras proved to be a much better experience for us,” Konkol said. “In fact, [the comparison] is nowhere close, to be honest.”
FOR FURTHER INFORMATION:
Emergent Vision Technologies’ High-Speed Cameras:
https://emergentvisiontec.com/area-scan-cameras/
