HOW MACHINE VISION SYSTEMS HELP INDUSTRIAL ROBOTICS TRANSFORM MANUFACTURING

Due to a tight labor market, today’s manufacturers have a seemingly insatiable need to increase flexibility, improve productivity, and employ humans only for tasks where they add the most value. From machine tending, assembly, and bin-picking, to pick-and-place, material handling, and packaging, the use of industrial robots and collaborative robots (cobots) is on the rise in factory automation.

The primary reasons for deploying robot applications include cost, volume, quality, and precision, as well as safety and repetitive motion. Is the current cost per unit too high? Are resources to increase production volume lacking? Are there quality problems with deliverables? Do part size or precision requirements exceed what humans can handle? Are the materials handled hazardous to humans, or are repetitive motions causing injury? If so, an industrial robot or cobot application could be the answer.

Many manufacturers are already using vision systems in robotics to automate their assembly and inspection processes. In order to specify a vision system for robot guidance, the application must be thoroughly evaluated and understood, and the performance requirements must be defined. Accuracy, repeatability, precision, robustness, and throughput are a few considerations that determine the performance of robotic-vision systems. It is not only important to choose the right robot, but also to select the correct vision-system hardware for the task at hand. 

While it is possible to reduce integration time with a VGR package from a robot supplier for simple robot-vision system applications, the majority of VGR applications present significant challenges. Machine vision companies are usually adept at image formation, image processing, and image analysis, giving them a significant advantage over robot vendors in delivering high-performance vision-based robotics solutions. Furthermore, specifying a camera for a robot-vision system from a reputable supplier with machine vision experience offers the flexibility to use the robot of your choice and then choose from a wider range of camera offerings to optimize your robot-vision system applications.

Emergent Vision Technologies can integrate and ship turnkey performance systems involving multiple machine vision cameras, multiple servers, GPUs, FPGA cards, SSDs, Switches, and other components, designed to specific application requirements. This approach is suitable for rapid prototyping and general deployment of performance algorithms. Such a system allows for performance testing and even prototyping of custom GPU algorithms like AI, inference, pattern matching, 3D reconstruction, Gaussian Splatting, and compression. It is also possible to work with the saved images using software such as NERF (neural radiance fields), RealityCapture, Theia Markerless, and Agisoft.

Many new VGR applications are easily cost-justified today due to the lower cost of vision systems. VGR can also be deployed faster and easier than ever thanks to simplified calibration, easier integration, and new connectivity standards. State-of-the-art machine vision systems are easy to integrate with other factory systems and can communicate with all other devices on the network. This makes them a good choice for manufacturers who are looking to create a more connected and automated factory.

The cameras on robots serve as their eyes, taking high-resolution photos or video feeds of the robotic work-cell area. Robot vision cameras can range from standard RGB cameras to specialized variants such as NIR or depth cameras. Matching the right camera to the specific vision-guided robot application is critical for a successful VGR implementation. 

Emergent Vision Technologies’ cameras support an optimized GigE Vision Stream Protocol (GVSP) approach for VGR applications that require reliable image transmission. For real-time processing and multi-camera VGR systems, Emergent cameras have zero-copy-image transfer and zero-data-loss imaging capabilities, which ensure efficient image transfer with minimal CPU overhead.

VGR and many other quality control and inspection applications benefit from Emergent’s 5GigE, 10GigE, 25GigE, and 100GigE cameras with Sony Pregius S and Gpixel CMOS sensors. The following Sony Pregius S CMOS image sensors are used in our 5GigE Eros and 10GigE HR cameras: 

• 5.1MP IMX547: HE-5000-SBL 5GigE camera (45.5fps), HR-5000-SBL 10GigE camera (99fps)
• 8.1MP IMX546: HE-8000-SBL 5GigE camera (36.5fps), HR-8000-SBL 10GigE camera (73fps)
• 12.4MP IMX545: HE-12000-SBL 5GigE camera (34fps), HR-12000-SBL 10GigE camera (68fps)
• 16.13MP IMX542: HE-16000-SBL 5GigE camera (26fps), HR-16000-SBL 10GigE camera (52fps)
• 20.28MP IMX541: HE-20000-SBL 5GigE camera (21.5fps), HR-20000-SBL 10GigE camera (43fps)
• 24.47MP IMX540: HE-25000-SBL 5GigE camera (17.5fps), HR-25000-SBL 10GigE camera (35fps)

The 25GigE Bolt series also leverages the 5.1MP IMX537 in its HB-5000-SB (269fps), 8.1MP IMX536 in its HB-8000-SB (201fps), 12.3MP IMX535 in its HB-12000-SB (192fps), 20.28MP IMX531 in its HB-20000-SB, and the 24.47MP IMX530 in its HB-25000-SB (98fps). 

Moreover, camera resolution in robotic vision systems likely varies depending on the application requirements. Many robotic vision applications are successful with robot vision cameras that have a few megapixels of resolution. For high resolution inspection or for machine vision to guide robots in large FoV (Field of View) applications, on the other hand, it is possible for some robot vision cameras to have 64 or more megapixel resolution.

For VGR applications that require high-resolution cameras, such as when small features must be identified from a distance or in very large field of view applications, Emergent offers a wide range of high-resolution models across its different camera families, ranging up to the Zenith HZ-100-G 100GigE camera (103.7MP Gpixel GMAX32103) and Bolt HB-127-S 25GigE camera (127.7MP Sony IMX661) cameras.

Certain VGR applications exist where specialized imaging technologies may be required, such as detecting anti-counterfeiting watermarks or security codes on labels. Shortwave infrared cameras can be used to image through these bottles to ensure accurate fill levels. Cameras such as the HE-300-S-I, HE-1300-S-I, HE-3200-S-I, and HE-5300-S-I – which are part of the Eros 5GigE camera series – leverage the advanced Sony SenSWIR sensors, which are capable of capturing images in the 400-to 1700-nm range. These cameras are based on Sony’s 0.33MP IMX991, 1.31MP IMX990, 3.14MP IMX993, and 5.24MP IMX992 sensors.

A number of robot vision cameras can be used for robotic guidance and inspection, including remote direct memory access (RDMA) cameras. Using this technology, data can be moved between devices in a network without CPU involvement on a packet-by-packet basis. 

Emergent utilizes the ubiquitous Ethernet infrastructure for reliable and robust data acquisition and transfer, instead of proprietary or point-to-point interfaces. It supports NVIDIA’s GPUDirect technology, which enables images to be directly transferred to GPU memory, with its optimized GigE Vision implementation.

Using Emergent eCapture Pro software, cameras featuring GPUDirect technology can also directly transfer images to GPU memory. By deploying this technology, zero CPU utilization and zero memory bandwidth imaging are achieved without data loss. By using more powerful GPU capabilities, this technology reduces the impact of large data transfers on the CPU and memory of the system, while maintaining compatibility with the GigE Vision standard and interoperability with compliant peripherals and software.

Industrial robots and cobots have become an essential part of the manufacturing process, offering a wide range of benefits, including increased productivity, efficiency, quality, and safety. Manufacturers use several types of robots. Each type of robot has its own unique features and is suited to different types of tasks in the manufacturing process.

Articulated Robots: Articulated robots, the most widely used, are classified by the number of points of rotation, or axes, they have. The most common is the 6-axis articulated robot; most robots and cobots fall into this category. Also known as robotic arms, articulated robots are extremely versatile due to their rotary joint configuration and can easily adapt to process changes or variations through programming or integration with robot vision systems. Articulated robots can cover large workspaces due to their multiple axes of rotation, which allow them to reach all angles, work with a variety of workpieces, and handle complex tasks such as assembly, welding, material handling, and machine tending.

Selective Compliance Articulated Robot Arm (SCARA) Robots: SCARA robots excel at vertical assembly tasks that require movement between parallel planes such as transferring parts from a tray to a conveyor. Among the fastest industrial robots, SCARAs are ideal for automating high-speed processes or significantly reducing cycle times. In assembly applications, SCARA robots combine flexibility and rigidity and offer a competitive price-performance ratio due to smaller size and fewer mechanical axes. Their relatively compact size and short reach makes them ideal for small or confined work areas. SCARA manufacturing robot arms are incredibly precise, making them suitable for assembly or disassembly applications, pick and place, material handling, and other operations that require precision, speed, and smooth motion.

Delta Robots: Delta robots use three base-mounted motors to actuate control arms that position the wrist. Due to their parallel-link construction and the weight distribution of their motors, delta robots are faster than most other types of robot manipulators. In fact, they are 30% faster than SCARA robots, and can process up to 3 parts per second or 180 parts per minute. Common applications include pick and place, adhesive dispensing, packing and sorting, soldering, assembly operations, 3D printing, and haptic controllers.

Fig 1: Machine vision technologies can help guide delta robots to make high speed picks.

Cartesian Robots: Cartesian robots commonly move linearly along the X, Y, and Z axes. A rotational axis on the end of the Z can bring the number of axes up to four. They are easy to customize, maintain, and control using a simple PLC for point-to-point movement as opposed to articulated arm robots, which require more complex robotic control systems and programming. With a high load-carrying capability and high degree of mechanical rigidity and accuracy, cartesian robots find application in pick-and-place operations, welding, inspection, CNC applications, 3D printing, plasma/laser cutting, wood routing, and PCB assembly.

Polar Robots: Polar robots, also known as spherical robots, have joints that are connected by a twisting mechanism, which allows for long reach capability in the horizontal position, and good load lifting capabilities. They are suitable for applications where a small amount of vertical movement will suffice. Known for high speed and precision, handling heavy loads, and compact size, polar robots are suitable for use in limited workspaces. Applications include machine tool loading, material movement, stacking, heat treatment operations, forging, welding automation, assembly operations, and diecast process tending, as well as in painting systems and paint line tending applications.

Cylindrical robots: Cylindrical robots operate within a compact, cylindrical-shaped work envelope. Mechanics are cost-effective and simple with one rotary axis for rotation at the base and two linear axes for height and arm extension. Cylindrical robots have a higher load carrying capacity and provide high rigidity to the manipulator. They are generally suitable for pick-and-place applications in packaging, machine tending, material handling, palletization, dispensing, and simple welding and soldering tasks.

Before choosing the type of robot that is best suited for a particular application, it’s important to define the specific requirements of the task, such as the required speed, accuracy, and payload, as well as workspace requirements and budget.

Traditional automation, which includes fixed automation systems and conventional blind robots and cobots, are best suited for applications where very high volumes of a single part or a few similar parts are produced. However, if the parts cannot be fed or fixtured within proper tolerances for the robot to accurately grasp, then machine vision will be required for robot guidance to ensure precise pickup or placement.

Fig 2: Adding machine vision to an industrial robot creates a flexible system with more capabilities.

A vision-guided robot is a much more flexible and versatile automation solution. Robot vision system applications enable flexible manufacturing by empowering robots to adapt to product changes, and automating a wide range of assembly applications, including those with small batches of parts or mixed batches of multiple parts. With vision-guided robots, parts don’t need to be held by expensive precision fixtures, additional labor isn’t required to load and orient parts, and upstream actuators aren’t required to sort and feed parts.

In addition to automating a wide range of assembly applications, a VGR system may also be used for inspection of components or finished assemblies; VGR is also useful in automated palletizing, conveyor tracking, and random part picking. Furthermore, an industrial robot with integrated vision system guidance will likely also be able to measure parts and products, as well as read barcodes on them.

As manufacturers struggle to keep up with customer demands for custom products delivered at lightning speed, they must adopt automated production and quality assurance systems. However, traditional machine vision for robotics that manufacturers have relied on in the past may not be as effective today, due to increased variation and frequency of changes in product, process, and package types.

In such cases, AI, machine learning, and deep learning have stepped in to assist machine-vision systems locate parts with a very high level of accuracy and more efficiently perform complex automated inspections on a wide variety of products and even large final assemblies. As a result, manufacturers today are installing automated inspection systems that check virtually any product with lower false-negative rejection rates than could be achieved with either AI or rule-based vision systems alone by combining the capabilities of AI machine-vision solutions with the strengths of traditional machine-vision algorithms, including 3D vision.

In addition, deep learning reduces micro stops due to corner-and-edge cases and enables robots to perform both assembly and inspection tasks simultaneously. Combining these tasks reduces productivity losses caused by part variations and late detection of defects.

Fig 3: High-speed machine vision cameras from Emergent Vision Technologies can help robots perform a wide variety of tasks.

For additional camera options, check out our interactive system designer tool.