NOVEL MACHINE VISION TECHNOLOGY FOR AUTOMATING MODERN WAREHOUSE ENVIRONMENTS

Automation adoption within warehousing and logistics continues to grow as companies seek a means to enhance efficiency, increase throughput, and drive revenue to stay competitive. In China,the largest industrial robot market in the world – for example, warehouse automation will grow at a compound annual growth rate of 13.5% over the next five years, according to market intelligence company Interact Analysis.

Automation systems in warehouses come in several shapes and sizes, leveraging technologies such as machine vision, autonomous mobile robots, automated guided vehicles, traditional industrial robots, collaborative robots, motion control, and more. Even within the machine vision realm specifically, different types of technologies are deployed to help businesses optimize various processes. The following looks at the various ways that machine vision technologies – including high-resolution and high-speed cameras, barcode scanners, and software – add value in today’s logistics and warehousing environments.

Modern warehouse and logistics environments deploy a wide array of automation technologies, including industrial robots, autonomous mobile robots, machine vision cameras, barcode readers, smart cameras, 3D cameras, and different types of software. From the moment goods enter a logistics facility to the point where they’re packed into a truck for delivery, machine vision plays a significant role. 

Several application types exist, from palletizing and depalletizing and dimensioning to high-speed sorting and scan tunnels. Of course, the needs of each application vary greatly, and not every warehouse application is a match for the high-speed, high-resolution cameras offered by Emergent Vision Technologies. Still, as applications evolve and speed up, certain tasks on the warehouse floor will benefit from such cameras. These include:

Several types of automated inspection tasks exist within a logistics facility, from inspecting goods for damage upon arrival to ensuring that packaging is complete and optimized before shipping. Area scan cameras paired with machine vision software can help reliably automate these processes and more, including documenting. This may include capturing images of pallets, boxes, trailers, and other equipment to assess overall quality and conditions. 

Machine vision technologies also help with warehouse storage, performing tasks such as stock verification, damage detection, and robot guidance such as automated storage and retrieval systems.  Meanwhile, in fulfillment, machine vision helps with several tasks, including verification of the condition and dimensions of packages, verifying picking and dispensing, and documenting outbound shipments.

Logistics and warehousing is a low-margin industry, so speed and accuracy are important when it comes to choosing a machine vision camera. Global shutter area scan cameras are commonly deployed in logistics applications because products are in motion at high rates of speed, so stopping this process means lost productivity. High-speed cameras have allowed machine vision systems in warehouse environments to adapt to these needs and keep processes in motion.    

For warehouse automation applications where reliable image transmission with no dropped packets or frames are required, Emergent Vision Technologies cameras support an optimized GigE Vision Stream Protocol (GVSP) approach. This means that all Emergent cameras deliver zero-copy-image transfer and zero-data-loss imaging capabilities, which ensures efficient image transfer with minimal CPU overhead, which is crucial for multi-camera systems and real-time processing.

Emergent’s 5GigE, 10GigE, 25GigE cameras, and 100GigE cameras featuring the latest Sony Pregius S and Gpixel CMOS sensors provide a full range of options to quality control and many other warehouse automation applications.  Cameras in our 5GigE Eros and 10GigE HR series leverage the following Sony Pregius S CMOS image sensors: 

• 5.1MP IMX547: HE-5000-SBL 5GigE camera (45.5fps), HR-5000-SBL 10GigE camera (99fps)
• 8.1MP IMX546: HE-8000-SBL 5GigE camera (36.5fps), HR-8000-SBL 10GigE camera (73fps)
• 12.4MP IMX545: HE-12000-SBL 5GigE camera (34fps), HR-12000-SBL 10GigE camera (68fps)
• 16.13MP IMX542: HE-16000-SBL 5GigE camera (26fps), HR-16000-SBL 10GigE camera (52fps)
• 20.28MP IMX541: HE-20000-SBL 5GigE camera (21.5fps), HR-20000-SBL 10GigE camera (43fps)
• 24.47MP IMX540: HE-25000-SBL 5GigE camera (17.5fps), HR-25000-SBL 10GigE camera (35fps)

The 25GigE Bolt series also leverages the 5.1MP IMX537 in its HB-5000-SB (269fps), 8.1MP IMX536 in its HB-8000-SB (201fps), 12.3MP IMX535 in its HB-12000-SB (192fps), 20.28MP IMX531 in its HB-20000-SB, and the 24.47MP IMX530 in its HB-25000-SB (98fps).

Fig 1: Vision-guided robots can leverage vacuum grippers to pick and place boxes from conveyor lines.

When deployed alone, robots can reliably automate repeatable, pre-programmed tasks across many different industries. When the robots are paired with machine vision, however, the systems become much more capable and flexible, making a new set of applications possible. In warehouse and logistics environments, this often means guiding a robot to locate, recognize, pick, and place objects for tasks including inventory management, order picking and packing, sorting, palletizing and depalletizing, and quality inspection.

While many barcode reading applications are accomplished with smart cameras and barcode readers, scenarios exist where it makes sense to pair a machine vision camera with software that can decode 1D and/or 2D barcodes. For example, let’s say a vision-guided robot is already deployed for a palletizing/depalletizing application that now requires code reading and verification of package labels. Using images acquired from the existing camera offers a streamlined approach to solving both problems. 

Machine vision cameras also typically offer higher speeds and resolutions than dedicated barcode reading devices, which helps with image acquisition if the pallets are moving or are being imaged from a distance. In addition, certain applications may require high-resolution cameras, including scenarios where smaller barcodes must be read from a distance or if a facility wants to capture images for photo documentation or acquire a view of an entire pallet as it comes to a dock door. 

For such instances, Emergent offers a wide range of high-resolution models across its different camera families, ranging up to the Zenith HZ-100-G 100GigE camera (103.7MP Gpixel GMAX32103) and Bolt HB-127-S 25GigE camera (127.7MP Sony IMX661) cameras.

Fig 2: Scan tunnels leverage machine vision technologies for barcode reading and quality inspections at high speeds.

Line scan cameras have also long served the logistics industry. Vitronic, for example, installed its parcel identification technology at UPS’s largest parcel distribution center in the world in the 1990s. In this setup, workers manually coded parcels that had not been read by Vitronic’s system of fixed line scan cameras that read barcodes and labels on parcels moving on a conveyor belt. Today, the company still uses line scan cameras in its Vipac identification system and achieves close to 100% identification rate utilizing machine vision software in place of human workers for barcodes that are difficult to read. 

To meet the high-speed requirements of today’s warehousing environments, Emergent’s 10GigE, 25GigE, and 100GigE line scan camera families can be reliably deployed to deliver zero-copy, zero-data-loss imaging capabilities. Models range from the 4K LR-4KG35 10GigE line scan camera capable of 172KHz to the LZ-16KG5 100GigE line scan camera, which can reach 400KHz.

Certain applications exist where specialized imaging technologies may be required, such as detecting anti-counterfeiting watermarks or security codes on labels. Shortwave infrared cameras can be used to image through these bottles to ensure accurate fill levels. Cameras such as the HE-300-S-I, HE-1300-S-I, HE-3200-S-I, and HE-5300-S-I – which are part of the Eros 5GigE camera series – leverage the advanced Sony SenSWIR sensors, which are capable of capturing images in the 400-to 1700-nm range. These cameras are based on Sony’s 0.33MP IMX991, 1.31MP IMX990, 3.14MP IMX993, and 5.24MP IMX992 sensors. 

Elsewhere in the warehouse, applications involving the inspection of shiny or reflective materials that produce glare present problems for traditional imaging techniques. Reading barcodes on a shiny can, for example, can be nearly impossible because of the reflection. Polarization cameras like the HR-12000-S-P and HR-5000-S-P offer improved brightness and color and can also capture details, which cannot be detected by a normal image sensor in applications such as plastic packaging inspection. These cameras are based on Sony’s 5MP Sony IMX250MZR (mono) and IMX250MYR (color) and 12MP Sony IMX253MZR (mono) and IMX253MYR (color), respectively. These sensors feature microscopic wire-grid polarizers over every lens and have 0°, 45°, 90°, and 135° polarization angles in four-pixel groups; the groups are interpolated, reducing the sensor’s overall functional resolution by 4x. In other words, each four-pixel block equates to one pixel of output.

Emergent leverages an optimized GigE Vision approach and ubiquitous Ethernet infrastructure for reliable and robust data acquisition and transfer, with best-in-class performance instead of relying on proprietary or point-to-point interfaces and image acquisition boards. Emergent supports direct transfer technologies such as NVIDIA’s GPUDirect, which enables the transfer of images directly to GPU memory. The technology mitigates the impact of large data transfers on the system CPU and memory, and instead utilizes more powerful GPU capability for data processing, while maintaining compatibility with the GigE Vision standard and interoperability with compliant software and peripherals.

Within the realm of GigE Vision, one of the problems that has led to the use of Transmission Control Protocol (TCP) or remote direct memory access (RDMA) and RDMA over Converged Ethernet (RoCE) is the need to dissect Ethernet packets at the receiver to provide the image data to the application in contiguous form, which requires splitting off Ethernet packet headers. Doing so is possible using software, but comes at a performance cost with triple the memory bandwidth and a higher CPU utilization, which is something that RDMA users tout when discussing pros and cons of traditional GigE Vision and RDMA. 

Emergent Vision Technologies leverage a zero-copy image transfer approach, which has become a must-have requirement for top performance in high-speed imaging. This approach minimizes CPU and memory bandwidth by using built-in splitting features available in current network interface cards. This animation shows the zero-copy memory bandwidth usages of a system using the optimized GigE Vision Stream Protocol (GVSP) for zero-copy image transfer. The first part of the animation shows that the system is not optimized and the buffer in the NIC overflows, while the second part shows data flowing freely and reliably via zero-copy and system optimization.

For additional camera options, check out our interactive system designer tool.