Industrial Automation Magazine India

Smart Sensors and Machine Vision

Published on : Wednesday 10-05-2023

How machine vision systems make use of smart sensors and digital cameras to provide innovative applications in factory automation.

Smart sensors not only generate and receive data, they also enable effective processing of the information which is then helping or facilitating many activities on the shop floor from planning to troubleshooting; and predictive maintenance. For example, recently Adhoc Networks of Germany, a company engaged in providing smart waste solutions for intelligent level indicators and waste container emptying, launched the world’s smallest, smart, waste sensor module, OSCAR, measuring only 71x43x28mm. The optics can measure up to 4m within a container with a cone angle of 27° to determine the fill level. The company’s waste sensor modules are the key to its automated, fill-level detection and collection scheduling technology for local authorities and private companies that can reduce unnecessary collection trips to containers that are only partly filled. “Knowing the exact level in every container ensures that there are no wasted collections of part filled containers and, even more importantly, no overflowing containers,” explained Ole Ostermann, adhoc networks’ CEO. “Our approach is the perfect solution for monitoring the new generation of waste containers that are larger and discretely located below ground level. These are not currently easy to check the fill level apart from lifting a lid so they are usually emptied more often than necessary just to be sure, but that is a waste of resources.”
 
Smart sensors are widely used in industry in the IIoT era for a wide range of applications ranging from the usual measuring or pressure, temperature and other parameters for constant monitoring of processes, but they also have commercial and domestic applications. For example, IKEA’s new VINDSTYRKA air quality sensor allows measuring and monitoring of indoor air pollutant levels, complementing the existing IKEA range of smart solutions that enable better air quality in the home. Similarly Bosch Sensortec, a technology leader in sensing solutions, recently launched the new BHI360, a programmable IMU-based sensor system that combines a gyroscope with an accelerometer that enables full customisation. A variant of this, the BHI380 equips the sensor with additional algorithms. BHI380 is based on the same architecture but also includes self-learning AI software suitable for a wide variety of fitness tracking, thus making training and tracking a breeze and enabling personalised workouts. Typical applications include pedestrian navigation, 3D audio, personalised fitness tracking, and human-machine interaction.
 
Besides such standalone functions, smart sensors are used extensively in basic machine vision applications like object detection, barcode reading, shape and size detection, etc. Machine vision systems make use of smart sensors and digital cameras to provide innovative applications in factory automation. Advanced machine vision systems are complex as well as flexible and can be used in a variety of applications from object recognition to material inspection, flaw detection to pattern recognition and in combination with robots, used in production lines for picking and sorting, etc. 

Industrial vision or machine vision basically refers to image-based systems used in industrial and manufacturing applications like identification, inspection, sorting and gauging. These are also used for guidance in robotic applications. Vision based technology has been in use for over four decades now, though some devices were in use even earlier. Keyence – today a leading global supplier of sensors, measuring systems, laser markers, microscopes, and machine vision systems – started the development of image processors in the early 1980s for a general-purpose image processing sensor. Similarly, Automatix (now part of OMRON) had demonstrated its Autovision II – a basic machine vision system in 1983. 
 
Broadly, there are three types of machine vision systems that have evolved over the years – 1D or line scan systems used mostly in continuous process like sheets and rolled products (paper, metal, etc); 2D machine vision with a digital camera to capture an image of an object, commonly used in barcode scanners, label orientation, etc; and 3D machine vision systems with multiple cameras and laser displacement sensors that are typically used in surface inspection and volume measurement. So what began as a very basic identification system initially has now been refined for use in very sophisticated devices that make use of emerging technologies like artificial intelligence and machine learning.
 
While there are many different descriptions for machine vision, they all essentially say the same thing by using different terminology. According to the Automated Imaging Association (AIA), the world's largest machine vision trade group, machine vision encompasses all industrial and non-industrial applications in which a combination of hardware and software provide operational guidance to devices in the execution of their functions based on the capture and processing of images. Intel, the company that put silicon in Silicon Valley, describes machine vision as one of the founding technologies of industrial automation. Machine vision has helped improve product quality, speed production, and optimise manufacturing and logistics for decades. Now this proven technology is merging with artificial intelligence and leading the transition to Industry 4.0.
 
According to P&S Intelligence, the global machine vision market size was valued at USD 14.4 billion in 2022 and is expected to increase to USD 27.86 billion in 2023 at a CAGR of 8.60% during the forecast period. Growing demand for quality inspection and automation in different industry verticals are the main growth drivers backed by surging R&D spends by leading players in the field. Increasing penetration of robots in the smart manufacturing ecosystem is another factor. With rapid advances in technology, especially in deep learning algorithms, more and more applications are now possible with machine vision, apart from mere detection (absence/presence) of objects or sorting and pick and place. While bar code reading was one of the early applications, optical character recognition and verification are now optimised to near zero defects. Apart from this, the other common applications are: automated vision testing and measurement; colour verification; defect detection; part verification; pattern matching; traceability; and the more recent field of vision guided robots and AGVs/AMRs.
 
While there is no definite record of the number of companies engaged in the machine vision field, major players in the field include Cognex Corporation, Basler AG, Omron Corporation, Keyence, National Instruments, Sony Corporation, Teledyne Technologies, Texas Instruments, Intel Corporation, ISRA Vision, Sick AG, FLIR Systems, Optotune AG, USS Vision, ViDi Systems SA, Bosch Rexroth and Euclid Labs, among others. 

Cognex Corporation recently released the In-Sight® 3800 Vision System. Designed for high-speed production lines, In-Sight 3800 offers an extensive vision toolset, powerful imaging capabilities, and flexible software to deliver a fully integrated solution for a wide range of inspection applications. "The In-Sight 3800 offers twice the processing speeds of previous systems, performing tasks like a quality inspection in as little as one-third of a blink of the eye," said Lavanya Manohar, Vice President of Vision Products. "This added power allows users to maximize throughput and accommodate faster lines while delivering the high accuracy that they have come to expect from the In-Sight product line."
 
This new system is embedded with a comprehensive set of vision tools that includes Artificial Intelligence (AI)-based edge learning technology and traditional rule-based algorithms. Easy-to-use edge learning tools solve tasks with high variability and are set up in minutes with just a handful of training images. The industry-proven rule-based tools are well-suited to solve deterministic tasks with specific parameters.
 
In another application example that combines sensor solutions with machine vision, German robotics startup Magazino’s SOTO, a mobile robot for industrial production makes use of several technologies from SICK, an intelligent sensor solutions specialist. These include microScan3 safety laser scanners for localisation and navigation purposes; safety light curtains that prevent humans from interfering with the moving vehicle; DFS60 incremental encoders; and the Visionary-T Mini 3D camera controls the travel path from above to help with navigation and detect obstacles that are not at the height of the safety laser scanners. The fully autonomous SOTO automates material supply in the manufacturing industry. The robot perceives its environment and makes decisions itself, thanks to its intelligent software. Computer vision and numerous sensors enable the robot to understand the environment and navigate freely – even in a dynamic environment where people, industrial trucks and other autonomous vehicles are constantly moving.
 
B&R, a company that has taken integrated machine vision to a whole new level, is also enhancing its smart camera portfolio with powerful deep learning functionality. Machine vision algorithms based on deep learning are opening exciting new ways to improve quality, boost productivity and prevent waste while making manufacturing more flexible. The first product to emerge from the project is a deep-learning-based optical character recognition (OCR) function. This deep OCR achieves remarkably fast read rates, even on fonts that are otherwise difficult to recognise. High-performance deep learning algorithms require a powerful processor. Yet, implementation as an edge device also made power consumption a critical issue. Recently, the company has entered into collaboration with machine vision software specialist MVTec and AI processor specialist Hailo, which makes the B&R camera a powerful and efficient edge device. As the company’s press release states, its collaboration with MVTec gives machine builders access to the best selection of the best performing vision functions around, being an industry leader in both deep learning and classic rule-based algorithms – two complemental approaches in which each plays an essential role in machine automation.
 
ISRA Vision is focused on the development of advanced embedded technologies and highly complex algorithms and bringing them to market as easy-to-use touch-and-automate and plug-and-automate systems. The company during the K 2022 trade fair in Düsseldorf in 2022 launched “Cloud Xperience” – a completely new online software platform for its SMASH inspection systems that combines the benefits of cloud computing with intelligent analytics tools for the first time. With its innovative approach to clustering and classifying defects based on artificial intelligence (AI), Cloud Xperience can help to ramp up production faster, ensure knowledge is accessible at all times, and control processes automatically.
 
In fact AI is one of the most promising future trends in machine vision. Deep learning technology is helping customers to enhance the quality of a product. Even a micron size defect can be identified rendering inspection accuracy close to 100%. This technology is getting rapidly adopted by many industries such as automotive, pharma, digital to name just a few. 
 
In conclusion, it may be noted that machine vision technologies are evolving rapidly to keep pace with the requirements of the industry, at times even surpassing them. The systems are becoming faster to match the ever-increasing production rates, and becoming affordable too. They are also getting more intuitive and easy to use, with plug & play options. With the help of emerging technologies, machine vision is also gaining intelligence to deal with flexible production schedules and other process variations.