Reinforcement Learning for Predictive Maintenance of Industrial Plants

Open access


The reinforcement learning is a well-known approach for solving optimization problems having limited information about plant dynamics. Its key element, named “critic” is aimed at prediction of future “punish/reward” signals received as a result of undertaken control actions. The main idea in the present work is to use such a “critic” element for prediction of approaching alarm situations based on limited measurement information from the industrial plant. In order to train the critic network in real time it is proposed to use a special kind of a fast trainable recurrent neural network, called Echo State Network (ESN). The approach proposed is demonstrated on an example for predictive maintenance of a mill fan in Maritsa East 2 Thermal Power Plant


  • 1. Barto, A. G., R. S. Sutton, C. W. Anderson. Neuronlike Adaptive Elements that Can Solve Difficult Learning Control Problems. - IEEE Trans. on Systems, Man and Cybernetics, 13 (5), 1983, 834-846.

  • 2. Bellman, R. E. Dynamic Programming. Princeton, NJ. Princeton Univ. Press., 1957.

  • 3. Bertsekas, D. P., J. N. Tsitsiklis. Neuro-Dymanic Programming. Athena Scientific, Belmont, MA, 1996.

  • 4. Doukovska, L., P. Koprinkova-Hristova, S. Beloreshki. Analysis of Mill Fan System for Predictive Maintenance. Int. Conf. Automatics and Informatics’11, 3-7 Oct. 2011, Sofia, Bulgaria, B-331-B-334.

  • 5. Jaeger, H. Tutorial on Training Recurrent Neural Networks, Covering BPPT, RTRL, EKF and the “Echo State Network” Approach. GMD Report 159, German National Research Center for Information Technology, 2002.

  • 6. Jaeger, H. Adaptive Nonlinear System Identification with Echo State Networks. Advances in Neural Information Processing Systems 15 (NIPS 2002), Cambridge, MA, MIT Press, 2003, 593-600.

  • 7. Koprinkova-Hristova, P., G. Palm. Adaptive Critic Design with ESN Critic for Bioprocess Optimization. Lecture Notes in Computer Science, 6353, 2010, 438-447.

  • 8. Koprinkova-Hristova, P., M. Oubbati, G. Palm. Adaptive Critic Design with Echo State Net-work. Proc. of 2010 IEEE Int. Conf. on Systems, Man and Cybernetics, Istanbul, Turkey, 10-13 Oct. 2010, 1010-1015.

  • 9. Koprinkova-Hristova, P., L. Doukovska, S. Beloreshki. Fuzzy Classifier of Mill Fan System Working Regimes. XIX Int. Symposium Process Control of Power Plant Systems, Bankya, Bulgaria, 2011, 25-28.

  • 10. Koprinkova-Hristova, P., L. Doukovska, P. Kostov. Working Regimes Classification for Predictive Maintenance of Mill Fan Systems. 2013 IEEE International Symposium on Innovations in Intelligent Systems and Applications, IEEE INISTA 2013, 19-21 June 2013, Albena, Bulgaria, DOI: 10.1109/INISTA.2013.6577632.

  • 11. Koprinkova-Hristova, P., N. Tontchev. Echo State Networks for Multi-dimensional Data Clustering. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7552 LNCS (PART 1), 2012, 571-578.

  • 12. Lenardis, G. G. A Retrospective on Adaptive Dynamic Programming for Control. Proc. of Int. Joint Conf. on Neural Networks, Atlanta, GA, USA, 14-19 June 2009, 1750-1757.

  • 13. Lukosevicius, M., H. Jaeger. Reservoir Computing Approaches to Recurrent Neural Network Training. - Computer Science Review, 2009, 3, 2009, 127-149.

  • 14. Prokhorov, D. V. Adaptive Critic Designs and Their Applications. Ph.D. Dissertation. Department of Electrical Engineering, Texas Tech. Univ., 1997.

  • 15. Schrauwen, B., M. Wandermann, D. Verstraeten, J. J. Steil. Improving Reservoirs Using Intrinsic Plasticity. - Neurocomputing, 71, 2008, 1159-1171.

  • 16. Si, J., Y.-T. Wang. On-line Learning Control by Association and Reinforcement. - IEEE Trans. on Neural Networks, 12 (2), 2001, 264-276.

  • 17. Sutton, R. S. Learning to Predict by Methods of Temporal Differences. - Machine Learning, 3, 1988, 9-44.

Information Technologies and Control

The Journal of Institute of Information and Communication Technologies of Bulgarian Academy of Sciences

Journal Information


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 50 50 40
PDF Downloads 12 12 10