Dynamic maintenance model for a load sharing k-out-of-n: G system using deep reinforcement learning

Abstract

In this paper, a deep reinforcement learning approach is used to provide a dynamic maintenance model for a load sharing <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>k</mml:mi> </mml:mrow> </mml:math> -out-of- <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>n</mml:mi> </mml:mrow> </mml:math> : <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>G</mml:mi> </mml:mrow> </mml:math> system with identical components, where all remaining components share the load equally. It is assumed that the degradation path of each component in the system follows an Inverse Gaussian process and the system degradation is the only cause of the system failure. The maintenance problem is expressed as a Markov decision process, and the optimal maintenance action is found by solving the problem using a advantage Actor-Critic algorithm, where the algorithm and its modifications on its input are extensively discussed in this paper. Finally, the performance of this modified deep reinforcement learning approach in finding the optimal policy for a load sharing <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>k</mml:mi> </mml:mrow> </mml:math> -out-of- <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>n</mml:mi> </mml:mrow> </mml:math> : <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>G</mml:mi> </mml:mrow> </mml:math> system is demonstrated through a numerical example and compared with the performance of standard Actor-Critic algorithm. Furthermore, this paper uses a decision tree to depict the optimal policy, providing a more interpretable tool than the neural network.