Stable reinforcement learning with recurrent neural networks

Key words:  Stability analysis  Integral quadratic constraint  Recurrent neural network  Reinforcement learning  Linear matrix inequality

摘要:

Abstract:

In this paper, we present a technique for ensuring the stability of a large class of adaptively controlled systems. We combine IQC models of both the controlled system and the controller with a method of filtering control parameter updates to ensure stable behavior of the controlled system under adaptation of the controller. We present a specific application to a system that uses recurrent neural networks adapted via reinforcement learning techniques. The work presented extends earlier works on stable reinforcement learning with neural networks. Specifically, we apply an improved IQC analysis for RNNs with time-varying weights and evaluate the approach on more complex control system.