Master’s thesis looks at the future of machine learning

Machine learning opens up perspectives for process control in many sectors. Even in environments with technology that is difficult to master. For example, last year a young Belgian developed a model that detects anomalies when using injection kicker magnets in the world’s largest particle accelerator. Linked to the interactive cluster algorithm ‘COBRAS’, experts can easily link specific domain knowledge to the model. The results of experiments were already very promising.

Since 1954, CERN (Conseil Européen pour la Recherche Nucléaire) has been conducting physical research into elementary particles. This is done in the largest particle accelerator in the world, the Large Hadron Collider (LHC). It is located in an annular tunnel with a circumference of 27 km that was built at a depth of 50 to 175 metres on the Franco-Swiss border near Geneva. “In the particle accelerator complex, protons are accelerated in the opposite direction until they almost reach the speed of light,” says Thiebout Dewitte. “A sequence of particle accelerators is used for this purpose. In the last particle accelerator the particles eventually collide. It’s this interaction that’s being studied by physicists.”

Experimenting with machine learning

The LHC is a high-tech system that is unparalleled anywhere else in the world. That is why the construction and application is a continuous process of trial and error. “Occasionally something goes wrong when conducting experiments,” says Thiebout Dewitte. “Sometimes the cause lies with the hardware. Unfortunately, it is difficult to put your finger exactly on it. The installation is very large and contains numerous components that interact with each other in a complex way. Of course, sensors and safety systems are used to avoid problems. However, such large amounts of data are collected that the technical team does not always see the forest through the trees. A manual data analysis is therefore not scalable for the even larger particle accelerators that will be built in the future. Faster detection of anomalies – and finding the cause – would also save a lot of time to use the particle accelerator more efficiently. This is why CERN is experimenting with techniques such as machine learning to develop models that indicate when something goes wrong and, in time, perhaps why it is happening.

Research for master’s thesis

Given the complexity of the particle accelerator, knowledge institutes are designated research partners to figure out optimisations. This gave Thiebout Dewitte the opportunity to partially translate CERN’s ideas into practice. “The development of models to evaluate and interpret the data of the injection kickermagnets was a research theme that KU Leuven proposed for my thesis as a civil engineer in computer sciences”, he says. “I was immediately triggered, because it seemed a fantastic idea to combine machine learning with physics. Even more, I was able to work in an environment that is at the very highest level in terms of technology”.

Based on normal behavior

The injection kicker magnets are used in the LHC to send the particles from one accelerator to the other. Thiebout Dewitte: “Injecting particles into the LHC is a crucial step, so it can be expected that important patterns will appear in this step. Together with my thesis supervisor, we decided to base the automatic anomaly detection on machine learning methods that model the normal behaviour of magnets. Let me explain this with an example. If magnets normally act at a temperature of 20°C for 5 milliseconds, there is a problem if they do so at a temperature of 10°C as well. Everything that is ‘normal’ behaviour, we’ve poured into a model. Each time new data is generated, it can be sent through this model and the operator will know very quickly whether an anomaly has occurred”.

Based on normal behavior

The injection kicker magnets are used in the LHC to send the particles from one accelerator to the other. Thiebout Dewitte: “Injecting particles into the LHC is a crucial step, so it can be expected that important patterns will appear in this step. Together with my thesis supervisor, we decided to base the automatic anomaly detection on machine learning methods that model the normal behaviour of magnets. Let me explain this with an example. If magnets normally act at a temperature of 20°C for 5 milliseconds, there is a problem if they do so at a temperature of 10°C as well. Everything that is ‘normal’ behaviour, we’ve poured into a model. Each time new data is generated, it can be sent through this model and the operator will know very quickly whether an anomaly has occurred”.

Vague timestamps

Another great novelty is the use of a handmade logbook to evaluate the quality of the learned model. Thiebout Dewitte explains: “Each time a maintenance action is carried out, this is accurately recorded in the logbook. If something goes wrong with the equipment during an experiment, this must be detected and logged manually. This can take quite a long time. What’s more, sometimes a problem is not noticed at all because of its complex nature.” In the application of Thiebout Dewitte such vague ‘timestamps’ are taken into account by carrying out predictions and evaluation on short time segments that indicate a single use of the magnets. In industrial complex applications, logs with vague timestamps are often the reality. A specific segmentation method offers a way to deal with this.

Even smarter thanks to COBRAS

In addition to the self-learning model based on historical data, an extra dimension of domain knowledge can be added to the model. This is done through a link with ‘COBRAS’[1], an interactive cluster algorithm developed by Toon Van Craenendonck, PhD student at KU Leuven. By answering some interactive questions about historical data, COBRAS will cluster the output of the model and use its accumulated knowledge to modify the prediction (anomaly or not). “For example, it is possible that a strong peak in the flow manifests itself several times,” says Thiebout Dewitte. “If this happens in the event of a short circuit, this usually refers to a problem. It is possible that the model learns this. However, when such a peak occurs during maintenance, it is desirable that no alarm is given. Suppressing such alarms can be done by adding this specific knowledge by answering questions. And that is only a first example, because the scope of application is very large. Reasons like the above can be made in both directions and it will also be possible to introduce ad hoc knowledge. ”

High quality predictions

Although the model can also be perfectly adapted to industrial applications, Thiebout Dewitte has no immediate plans to commercialise his ideas. Nevertheless, he strongly believes that future machine learning applications will evolve in the direction of his solution. “In many factories, there is a multitude of different data. This is not pre-designed to be applied in machine learning algorithms,” he explains. “The application in the LHC shows that a model can detect anomalies based on such diverse data. In addition, we demonstrated that the quality of prediction can be greatly improved when combined with additional expert knowledge. “