Monte Carlo-driven reinforcement learning for smart manufacturing systems

Abstract

Developing a smart control system capable of accounting for the dynamics of industrial equipment state changes and promptly detecting features of developing failures is currently a pressing issue. The goal of this study is to develop an intelligent technology for industrial equipment state diagnostics based on a modified MonteCarloDQN algorithm, with parameter tuning using the Monte Carlo Tree Search (MCTS) method and integration with the GEMMA (Guide d’étude des modes de marche et d’arrêt) model. The result of the algorithm's operation is sent to the GEMMA model “zone D”, which is responsible for fault detection and the control system’s response after a failure. This approach enables GEMMA to initiate timely transitions to a safe state while preserving the existing industrial control logic. The simulation and experiments are presented on the data obtained from the Modicon M340 programmable logic controller and the Altivar 630 frequency converter. The performance of the MonteCarloDQN algorithm is higher on 5 independent runs by the metrics Accuracy = 0.9609 ± 0.0129, Balanced Accuracy = 0.9368 ± 0.0242, Precision = 0.9617 ± 0.0117, Recall = 0.9609 ± 0.0129, F1-score = 0.9596 ± 0.0145 and G-mean = 0.9571 ± 0.0153 compared to the basic algorithms DQN, SVM, KNN and Informer Transformer.