Industrial Automation Magazine India

Machine Learning Meets Industrial Automation

Published on : Saturday 09-11-2019

TwinCAT 3 Machine Learning: How seamless integration with control technology offers the greatest promise.

Advances in automation technology have spawned several interesting and promising innovations in recent years – most prominently, the integration of various specialised technologies like image processing and condition monitoring. These examples illustrate the progress of optimisation in automation. However, sophisticated technological advancements continue to open up new potential that is waiting to be tapped. Machine learning (ML), integrated seamlessly into control systems with TwinCAT 3 Machine Learning software, ranks among the technologies with the greatest promise.

Experts agree that incorporating machine learning into automation (see info box 1) is crucial to sustain and enhance Germany’s competitiveness, going forward. This, however, will take time to accomplish in real-world applications. One reason is that obtaining the requisite data can be a challenge – mainly due to the costs involved both in collecting data in production environments and in manually preparing it for training purposes (the definition of training data known as labeling). The data can also be noisy – in other words, insufficiently precise. Another reason is that wrong predictions might occur that cause errors in system operation. This could result in significant negative impacts, such as damage to a machine, complete failure or – worse still – harm to humans working in production. Consequently, a number of requirements have recently emerged that are of critical importance when integrating machine learning into automation technology (see info box 2).

Nonetheless, one immediate advantage is already clear: Integrating ML would greatly benefit robotics, mathematical optimisation, anomaly detection, model predictive control and other applications, as several recent case studies have shown. Two software products from Beckhoff, TwinCAT 3 Neural Networks Inference Engine and TwinCAT 3 Machine Learning Inference Engine, now offer automation specialists and machine builders an industry-ready solution to integrate inference – the execution of a trained ML model – into their systems.

The workflow begins with data collection

Having an effective data infrastructure in place is essential to ensuring that training processes and inference from learned models are based on suitable data sets. Many manufacturing systems today already capture a large amount of data that can be utilised in machine learning. Integrating these systems seamlessly with TwinCAT 3 Machine Learning makes it possible to apply a range of frameworks, including TensorFlow, PyTorch and MATLAB^®, directly. This will lead to increasing convergence between the realms of the data scientist and the machine builder, creating immense synergy effects in the process.

The workflow begins with the collection of data from a fully or partially automated process. The actual data recorded and evaluated depends, of course, on the application in question. As a rule, it makes sense to begin with a specific and clearly delineated use case, and then define the data that the use case requires. Hence, domain knowledge – in this case, machine builders’ specific expertise – is crucially important.

As experts within their domain, machine builders understand the empirical and physical relationships between defined target values and process data; they can give meaning to abstract variables. Therefore, it is important to work with these experts to determine how the desired or derivable target values can be obtained in temporal correlation with other information. That might be machine data gathered from additional sensors or from ERP or other systems, or data entered manually by workers either as text or through a machine’s HMI. The availability and the quality of target values define the difference between supervised and unsupervised learning as in the following examples:

• An end-of-line test based on a classifier: A component is inspected by sensors. The measurements captured for a representative set of components must then be labelled; the measurements might, for instance, be reviewed manually in parallel so that they can be defined as training data.
• A virtual sensor as a substitute, or redundant monitor, for a physical sensor: The physical sensor can be used in the training process to generate the target value. If this sensor and the machine data from which the sensor signal is to be estimated are both processed and stored by the machine controller, the input and target values are in temporal sync with one another.
• Process optimisation: The target value for supervised learning can be obtained by simulating the process on a digital twin. The simulation needs to be sufficiently precise in terms of the observed target value. Based on the simulation, an optimisation algorithm can run through, and specifically optimise, several variants of the process. The result can then serve as a label, and the entire data set can be used for supervised learning.
• Anomaly detection: The advantage when developing a system to detect non-typical behaviour is that only “normal” machine behaviour is used as a training data set. There is no need for data on machine malfunctions in this case. However, “normal” machine behaviour has to be assessed in detail with domain experts and recorded as representative in the training data set.
• One approach can be to train a one-class support vector machine (SVM) in order to process potentially high-dimensional training data using an algorithm. This is termed semi-supervised learning.
• Unsupervised learning with missing labels: A condition monitoring system or end-of-line test, for instance, can be created using just an algorithm’s input data, provided the data exhibits sufficient changes in its structure. Principal component analysis, for example, or a k-means algorithm (for clustering analysis) can help identify such changes. The meaning of a particular change in the data at a given time can then be assessed with the help of a domain expert.

Besides the data types and the evaluation method, the tools used are of major importance to the workflow, too. The following TwinCAT 3 Measurement and TwinCAT 3 Connectivity products are good choices:

• TwinCAT 3 Scope, with its ability to store data in a range of formats, including binary, CSV and TDMS, is a simple, flexible tool for capturing and storing data during the engineering phase.
• If an SQL or noSQL database is available – locally, on the network or in the cloud – TwinCAT 3 Database Server is an ideal tool that allows users to aggregate even large quantities of data at machine runtime.
• If a machine is connected to a public or private cloud over a message broker, the TwinCAT 3 IoT family of products can be used to save data through the broker to a data store or warehouse.
• The TwinCAT 3 Analytics Logger offers a convenient way to write large amounts of data from the machine controller, during live operation and cycle-synchronously, either to a local file or via a message broker to a data lake or database.

Training a machine learning model

If training data is available, it has to be loaded into an external ML framework and processed. There are plenty of suitable ML frameworks for users to choose from. Many are based on Python and are open-source; others, such as MATLAB^® and SAS, are commercial products. They provide a large number of standard functions along with toolboxes and libraries to access the stored data.

The work in the ML framework consists of preparing data, selecting a suitable ML algorithm, starting the training process and evaluating the learning results. This data-based engineering stage produces a trained ML model that represents the relationship between input and target data. Like the data interface, the interface chosen to deploy the trained ML model in TwinCAT 3 is open and supports the industry-standard Open Neural Network Exchange (ONNX) format.

ONNX is designed to enable interoperability between different frameworks. Each ML framework uses its own format to represent trained models whereas ONNX provides an interoperability layer for sharing models across different frameworks. It can be used to provision models for framework-independent transfer learning, and to execute inference on diverse platforms, regardless of the selected training framework.

Having open interfaces to import data and export models also encourages interdepartmental cooperation and collaboration with external service providers. The workflow described here sets out defined interfaces between different domains in engineering and information technology. Provided that data is made available, which can be used to train ML models, all that is required to run an application on a machine is a model file, which can even be encrypted if necessary.

Deploying a model in TwinCAT 3 Runtime

The final stage in the workflow is to deploy the ML model in TwinCAT 3 in two steps:

• The exported ML model is converted to a protected binary format (.bml).
• The .bml file is loaded onto the target system (an Embedded or control cabinet PC).

Conversion to a .bml file (with optional encryption) helps protect the creator’s intellectual property (IP) because, in this format, it is impossible to determine how the trained model was built. It also enables external service providers to send deliverables to customers without revealing proprietary information.

A new TwinCAT 3 component, a standard TcCOM module that represents a generic inference module, is used to load and execute the trained ML model. It also provides the following new functions:

• TwinCAT 3 Machine Learning Inference Engine for classic ML algorithms, such as support vector machines and principal component analysis
• TwinCAT 3 Neural Network Inference Engine for neural networks, such as multilayer perceptrons and convolutional neural networks

CPU-based inference in real-time

When the .bml file is loaded, the runtime module configures itself to execute the ML model’s inference computations efficiently on the CPU of the Industrial PC. This module is a real-time-capable inference engine, seamlessly integrated into TwinCAT 3 that supports all common programming methods in the software platform:

• Calling the module from the PLC
• Calling the module from C/C++
• Direct calls from a cyclical task

The inference engine’s core capabilities include direct, synchronous access to all control data and fieldbus devices, scalable performance based on CPU speed levels, and non-blocking parallelisation, which enables the module to be called from within different task contexts.

Many machine learning methods have runtime properties that make them suitable for cycle-based systems operating at a fixed cycle time, because the compute time per run is deterministic and does not depend on the input data. This applies particularly to neural networks, where each inference run consists of the same computation steps that do not have to branch to other program segments. Methods like these are also suitable for use in high-frequency control loops in precision motion control systems, which, with their support, can learn and implement optimisations that are based on classic motion control algorithms but physically complex to model.

When using the TwinCAT 3 Neural Network Inference Engine, which is purely software-based, the inference computations execute solely on a single CPU core. This approach avoids communication delays that would inevitably occur if, for example, accelerator hardware such as graphics cards were used for vector calculations. Instead, the TwinCAT 3 Neural Network Inference Engine makes extensive use of the CPU-specific single instruction, multiple data (SIMD) instruction sets, such as SSE, AVX and AVX-512. These allow parallelised, optimised vector computations on a single core at maximum performance. In addition, tailored caching and efficient memory access strategies enable potential performance gains up to a factor of 30x compared to reference implementations without SIMD. Moreover, computations can also be parallelised across multiple CPU cores, allowing performance with large neural networks to scale almost linearly in step with the number of computing cores available.

Live demonstrations at Hannover Messe

At Hannover Messe 2019, Beckhoff presented a demonstrator featuring TwinCAT 3 Neural Networks Inference Engine integrated with TwinCAT 3 Motion Control that showed how neural networks can be used in real-time control applications. The demo consisted of two XTS systems placed on either side of a conveyor belt. Ten movers on each XTS were synched successively to markings on the conveyor and driven parallel to the markings over a distance of a meter. The movers then decoupled and travelled back to the start, getting ready for the next synchronised run.

One XTS was conventionally controlled with a time-optimised, seven-phase profile of the kind commonly used in motion control. During the disengaged phase of travel and immediately prior to re-synching, in particular, the system exhibited high wear and power consumption due to heavy acceleration and deceleration. The second XTS, controlled using a neural network trained specifically for this setup, proved to be substantially more efficient. It was controlled with specific function blocks that allowed the trained network to be accessed easily from the PLC or a C++ program. The neural network was trained using an optimisation algorithm based on a simple power consumption and wear model.

TwinCAT 3 Neural Network Inference Engine enabled the use of the neural network to handle NC and other motion control tasks in real time. Due to specific requirements associated with motion control, the motion profile generated had to follow the predecessor profile exactly through to the point of acceleration, and not exceed the maximum permitted speed or acceleration. TwinCAT 3 handled this without the need for user intervention. The XTS controlled by the neural network consumed less power and showed less wear than the conventionally controlled system, because the neural network’s “softer” profile spread the acceleration and deceleration required for synchronisation across a larger track section of the XTS.

Also at Hannover Messe 2019, Beckhoff demonstrated a virtual sensor system, consisting of a fan with a pressure sensor as well as vibration and speed monitoring. This showed that the air pressure produced by the fan could be predicted precisely using TwinCAT 3 Scope, based on vibration and rotation speed readings. Anomaly detection was also demonstrated on a setup consisting of two fans of the same type, each fitted with pressure, vibration and speed sensors. One of the two fans was fully functional, the other had intentional damage done to one of its blades. Using data recorded with TwinCAT 3 Database Server and the TwinCAT 3 Machine Learning Inference Engine, it was possible to predict the condition of each fan – directly, in real-time, in the PLC or from a C++ program, without the need for complex modeling.

Beckhoff Automation Pvt Ltd. Email: [email protected]

Further information: www.beckhoff.co.in/machine-learning

How machine learning works

Machine learning begins with a non-parametric model that is trained using existing data. The training data is fed into an untrained or partially trained model, then computations are run to determine the deviation from a desired result. The error is minimised by changing the model’s parameters according to set mathematical rules. The training process in machine learning amounts to an error minimisation process in which the model parameters are set in such a way that the model can replicate the training data as accurately as possible. The underlying model will then, however, also be capable of predicting input data that were not included in the training data. In other words, the model can generalise. The skill with machine learning lies in selecting a sufficiently complex but not excessively large model which, with only a small amount of training data, can achieve good generalisation across the entire range of relevant input data.

Challenges of integrating machine learning

The following are the key technical challenges involved in integrating machine learning into automation:

• Data scientists must have access to an open interface (for data capture, training and deployment, for instance) to ensure interoperability will be possible across different frameworks.
• Machine learning must be simple enough to use without expert knowledge; in other words, solutions must be capable of integrating with the existing software infrastructure.
• Many machine learning methods are inherently imprecise and should be treated accordingly. Solutions that are not suitable must be amended or discarded. This assures users that the trained algorithms are reliable.
• The training methods applied need to be fairly robust – that is, capable of functioning even with small quantities of noisy data.
• Transparency and demonstrability are of crucial importance for many companies. The more complex their requirements, the more important it is to properly understand the algorithms used. This is an area in which research activities still have a long way to go.

All of the above requirements need to be met for a successful integration of machine learning with real-time-based control.