PACKAGING INSPECTION: USING MACHINE VISION TO ENSURE PRODUCT QUALITY AND SAFETY

Packing inspection has long been a critical task that is successfully addressed by automated inspection systems using machine vision technologies. A broad range of industries benefit from packaging inspection, including pharmaceutical, medical device, tobacco, food and beverage, and general consumer products. Machine vision verification of package quality issues — such as label presence and position, deformed or mis-formed containers, fill levels, and more — ensures product aesthetic quality and consumer satisfaction.

More important, inspections that, for example, verify that a date/lot/product code is correctly and legibly printed, or ensure that container labelling represents the actual product in the package, or check for correct and intact package seals and safety components all can be essential in delivering a safe product to the consumer. Validation of correct, intact, and quality packaging can also reduce production costs and help avoid expensive product recalls. While many types of machine vision applications exist in packaging environments today, the following looks at some of the common ways that these systems add value.

Machine vision technologies accomplish several different types of tasks when it comes to label inspection and verification. These technologies can detect the presence or absence of a label, ensure proper product labeling, and verify that a label is complete and that the product is ready for shipping. In addition, these technologies can make sure labels are in the correct location, detect any defects such as smudges, and verify the presence of warning labels or logos. By using this technology, companies ensure product traceability and product quality while increasing productivity and minimizing risk.

While barcode scanners, smart cameras, and software tools may accomplish several of these tasks in many different scenarios, area scan cameras and line scan cameras may also be deployed. For instance, if a company has a vision-guided robot that palletizes packages and boxes for shipping, but also wants to add label inspection and verification tools, software can use images acquired from the existing camera to do so. Machine vision cameras also typically offer higher speeds and resolutions than barcode reading devices, which helps with image acquisition if the pallets are moving or are being imaged from a distance.

For packaging inspection applications where reliable image transmission with no dropped packets or frames are required, Emergent Vision Technologies cameras support an optimized GigE Vision Stream Protocol (GVSP) approach. This means that all Emergent cameras deliver zero-copy-image transfer and zero-data-loss imaging capabilities, which ensures efficient image transfer with minimal CPU overhead, which is crucial for multi-camera systems and real-time processing.

Emergent’s 5GigE, 10GigE, 25GigE cameras, and 100GigE cameras featuring the latest Sony Pregius S and Gpixel CMOS sensors provide a full range of options – ranging from 0.5MP to 100MP+ — for this and many other packaging inspection applications, whether they require high speeds, high resolution, or both. Cameras in our 5GigE Eros and 10GigE HR series leverage the following Sony Pregius S CMOS image sensors: 

• 5.1MP IMX547: HE-5000-SBL 5GigE camera (45.5fps), HR-5000-SBL 10GigE camera (99fps)
• 8.1MP IMX546: HE-8000-SBL 5GigE camera (36.5fps), HR-8000-SBL 10GigE camera (73fps)
• 12.4MP IMX545: HE-12000-SBL 5GigE camera (34fps), HR-12000-SBL 10GigE camera (68fps)
• 16.13MP IMX542: HE-16000-SBL 5GigE camera (26fps), HR-16000-SBL 10GigE camera (52fps)
• 20.28MP IMX541: HE-20000-SBL 5GigE camera (21.5fps), HR-20000-SBL 10GigE camera (43fps)
• 24.47MP IMX540: HE-25000-SBL 5GigE camera (17.5fps), HR-25000-SBL 10GigE camera (35fps)

The 25GigE Bolt series also leverages the 5.1MP IMX537 in its HB-5000-SB (269fps), 8.1MP IMX536 in its HB-8000-SB (201fps), 12.3MP IMX535 in its HB-12000-SB (192fps), 20.28MP IMX531 in its HB-20000-SB, and the 24.47MP IMX530 in its HB-25000-SB (98fps). Options range all the way up to the Zenith HZ-100-G 100GigE camera (103.7MP Gpixel GMAX32103) and Bolt HB-127-S 25GigE camera (127.7MP Sony IMX661) cameras. In addition, while most applications may not require such speeds, the Zenith 100GigE camera series leverages Gpixel’s GSPRINT series in the HZ-2000-G (2.5MP GSPRINT4502, 3462fps), HZ-10000-G (10MP GSPRINT4510, 1000fps), and the HZ-21000-G (21MP GSPRINT4521, 542fps).

Fig 1: Scan tunnels leverage machine vision technologies for barcode reading an quality inspections at high speeds.

For certain applications, line scan cameras can also be used for label inspection and verification tasks. In the 1990s, Vitronic installed its parcel identification technology at UPS’s largest parcel distribution center in the world. Workers manually coded parcels that had not been read by Vitronic’s system of fixed line scan cameras, which read barcodes and labels on parcels moving on a conveyor belt. Today, the company still deploys line scan cameras in its Vipac identification system and achieves close to a 100% identification rate utilizing machine vision software in place of human workers for barcodes that are difficult to read.

For the high-speed packaging environments of today, Emergent’s 10GigE, 25GigE, and 100GigE line scan camera families can be reliably deployed to deliver zero-copy, zero-data-loss imaging capabilities. Models range from the 4K LR-4KG35 10GigE line scan camera capable of 172KHz to the LZ-16KG5 100GigE line scan camera, which can reach 400KHz.

Businesses of all types rely on machine vision and automation technology to inspect primary and secondary packaging to ensure product quality, prevent escapes, eliminate product recalls, and ultimately maintain customer satisfaction. Product packaging types range greatly from industry to industry, but content inspection and verification for quality control purposes is a common application. Examples here include: 

• Blister pack inspection: Blister packs are pre-formed plastic packages that have a cavity made from plastic or aluminum and a lid typically made of plastic, paper, aluminum, or a soft foil-type lamination; the cavity contains the product and the lidding seals the product in the package. Typically used in food and beverage or for consumer-packaged goods, blister packs can contain anything from toys to medicine. Machine vision technology including area scan cameras and software can verify the content of these packs while also ensuring that the packaging is intact and properly sealed, as a failure to do so could have significant consequences. In some instances, inspecting plastic blister packs may be difficult due to the reflections coming off the plastic, especially when the packaging is uneven. In these cases, using polarized cameras from Emergent can help create the additional contrast needed to check for package completeness or inspect for damaged tablets, for example. If the plastic is opaque, SWIR cameras from Emergent can be deployed to inspect inside sealed products. More on polarization and SWIR below. 
• Kitting: In manufacturing environments, kitting refers to the packaging of products such as food, medical supplies, pet supplies, fashion, or other consumer goods into a larger package that is shipped off to the customer. Machine vision technologies including area scan cameras, smart cameras, and software tools can inspect an item such as a medical supply kit to ensure that every item is in the package, in the correct location, and in good condition for shipping off to the customer.

Larger packaging operations often deploy machine vision systems to ensure that packages are sent to the right lines for palletization prior to shipping. Companies do not want damaged packages being sent down the line to end up on store shelves, so vision system technologies such as area scan cameras, smart cameras, and software help inspect incoming boxes to ensure the box does not need to be repacked due to cosmetic defects or damage such as dents, scratches, or cracks. These technologies can also be used to read barcodes or QR codes at high speeds to ensure that, for example, the correct type of soup is being sent to the correct packaging station. At the end of the line, machine vision technology can also record the number of pallets, expiration dates, and product type, which helps inform shipping label information and input crucial data into the company’s enterprise resource planning (ERP) system. 

Throughout the packaging process, companies must also ensure that packages are safe and free of any foreign materials or contaminants that could lead to reputation damage. Area scan cameras, smart cameras, and machine vision software can help packaging operations avoid this scenario by performing automated inspections during the packaging process. Emergent’s camera families deliver the zero-data-loss, zero-copy image transfer capabilities needed for the packaging inspection systems of today, ensuring efficient image transfer with minimal CPU overhead, crucial for multi-camera setups, and real-time processing.

In some instances, this could mean the use of SWIR cameras to inspect the inside of, say, a container of cookies wrapped in opaque foil. Cameras such as the HE-300-S-I, HE-1300-S-I, HE-3200-S-I, and HE-5300-S-I – which are part of the Eros 5GigE camera series – leverage Sony’s SenSWIR sensors, which are advanced sensors capable of capturing images in the 400-to 1700-nm range. These cameras are based on Sony’s 0.33MP IMX991, 1.31MP IMX990, 3.14MP IMX993, and 5.24MP IMX992 sensors.

Several other applications within packaging environments may require the use of SWIR cameras, including cap closure inspection to ensure that the cap is aligned with the tamper ring. Defective or unacceptable bottles, jars, or other containers — which may have missing caps or tamper rings (or both), improper tampering of the caps, or otherwise defective caps — can lead to recalls, waste, and rework. Machine vision cameras and software can be used not only to verify the presence and position of caps and tamper bands, but the color and proper closure of the cap as well. In addition, visible wavelength cameras and software can perform fill level inspection on clear and tinted bottles.

Fig 2: Fill level inspection applications involve the use of machine vision technologies to verify fill level, sometimes with the use of SWIR cameras.

In certain applications, such as those where bottles or containers are opaque, Emergent’s SWIR cameras can capture images in the 400 to 1700 nm range to verify fill levels where visible light cameras cannot. In addition, some packages may use thermal sealing techniques such as induction sealing or band sealing. In these instances, products should not be shipped to customers if they are improperly sealed as they pose product safety and quality risks, so companies can leverage infrared cameras to inspect for a proper seal.

Many types of products, including food, cosmetics, and consumer goods arrive wrapped in plastic. Even prior to inspecting a wrapped package before palletizing and shipping, plastic packaging itself has likely been inspected by a machine vision system. Tears, rips, and breaks during printing can lead to financial and productivity losses, making plastic printing quality inspection an important step. Web inspection systems can leverage high-speed line scan cameras from Emergent Vision Technologies to inspect plastic at high speeds to identify possible defects before it is sent off for shipping.

Fig 3: Machine vision technologies can inspect contents of packages through shrink wrap while also inspecting the shrink wrap itself.

Other types of inspections involving plastics also benefit from machine vision technologies, including the inspection and verification of contents within a package after the plastic wrapping has been applied and inspecting the seal integrity of the plastic wrap itself. Inspecting the content of a package through transparent plastic packaging can be done with visible wavelength area scan cameras and software, while SWIR cameras can be used if the packaging is opaque. 

Another application involves inspecting the plastic shrink wrap itself after a product has been packaged to ensure there are no tears or foreign objects present. Inspecting clear plastic wrap presents challenges, however, due to its shiny and reflective nature. Polarization cameras such as the HR-12000-S-P and HR-5000-S-P can help detect surface defects like dirt, bumps, scratches, and bends. These cameras are based on Sony’s 5MP Sony IMX250MZR (mono) and IMX250MYR (color) and 12MP Sony IMX253MZR (mono) and IMX253MYR (color). The sensors feature microscopic wire-grid polarizers over every lens and have 0°, 45°, 90°, and 135° polarization angles in four-pixel groups; the groups are interpolated, reducing the sensor’s overall functional resolution by 4x. In other words, each four-pixel block equates to one pixel of output.

Emergent has developed a plug-in for its eCapture Pro software that uses GPUDirect to get the images directly onto a GPU with zero copy and zero loss technology. The plug-in includes options to review the benefits of the characteristic outputs of a standard polarized processing pipeline such as degree of polarization or direction or angle of polarization. End users can even remove the polarized light or select to output one of the four orientation options — 0°, 45°, 90°, or 135°.

In all its high-speed, high-resolution GigE Vision cameras, Emergent ensures best-in-class performance by using an optimized GigE Vision approach and ubiquitous Ethernet infrastructure for reliable and robust data acquisition and transfer, instead of relying on proprietary or point-to-point interfaces and image acquisition boards. In addition, Emergent supports direct transfer technologies such as NVIDIA’s GPUDirect, which enables the transfer of images directly to GPU memory, mitigating the impact of large data transfers on the system CPU and memory. Such a setup leverages the more powerful GPU capability for data processing, while maintaining compatibility with the GigE Vision standard and interoperability with compliant software and peripherals.

One of the problems that has led to the use of Transmission Control Protocol (TCP) or remote direct memory access (RDMA) and RDMA over Converged Ethernet (RoCE) within the GigE Vision standard is the need to dissect Ethernet packets at the receiver to provide the image data to the application in contiguous form, which requires splitting off Ethernet packet headers. Doing so is possible using software, but requires triple the memory bandwidth and a higher CPU utilization, which is something that RDMA users tout when discussing pros and cons of traditional GigE Vision and RDMA. 

Emergent Vision Technologies leverage a zero-copy image transfer approach, which has become a must-have requirement for top performance in high-speed imaging. This approach minimizes CPU and memory bandwidth by using built-in splitting features available in current network interface cards. This animation shows the zero-copy memory bandwidth usages of a system using the optimized GigE Vision Stream Protocol (GVSP) for zero-copy image transfer. The first part of the animation shows that the system is not optimized and the buffer in the NIC overflows, while the second part shows data flowing freely and reliably via zero-copy and system optimization.

For additional camera options, check out our interactive system designer tool.