Fast Reinforcement Learning For Optimal Control of Nonlinear Systems Using Transfer Learning
Yujia Wang, Ming Qing Xiao, Zhe Wu · 2024
Traditional reinforcement learning (RL) methods for optimal control of nonlinear processes often face challenges such as substantial demands on computational resources and training time, and the difficulty of ensuring the safety of the closed-loop system during training. To overcome these challenges, this work proposes a safe transfer reinforcement learning (TRL) framework. The algorithm leverages knowledge obtained from pre-trained source tasks to expedite learning in a new yet related target task, thereby significantly reducing both learning time and computational overhead for optimizing a control policy. Additionally, the proposed TRL method collects data and optimizes the control policy within a control invariant set (CIS) to ensure the safety of the system throughout the learning process. Furthermore, we develop a theoretical analysis for the TRL algorithm that establishes an error bound between the approximate control policy and the optimal ones, accounting for the discrepancy between the target and source tasks. Finally, we validate our approach using an example of optimal control of a chemical reactor, showcasing its effectiveness in solving the optimal control problem with improved computational efficiency and safety guarantees.