Data-Driven Solutions to Mixed $H_{2}/H_{\infty}$ Control: A Hamilton-Inequality-Driven Reinforcement Learning Approach
Yongliang Yang, Majid Mazouchi, Hamidreza Modares · 2020 IEEE Conference on Control Technology and Applications (CCTA) · 2020
Today's industrial systems have complex and possibly unknown dynamics, and are under the effect of unknown disturbances. This paper presents a model-free reinforcement learning (RL) algorithm for solving the mixed H2/H∞control design for industrial systems to respond favorably to both disturbance attenuation and performance requirement specifications, despite uncertainties in dynamics. The mixed H2/H∞performance optimization is first formulated as a non-zero sum game problem, which results in solving coupled Hamilton-Jacobi (HJ) equations. To solve these coupled HJ equations, a relaxed optimization framework based on a Hamiltonian-driven framework is presented that performs optimization subject to two Hamiltonian-inequalities corresponding to H2and H∞performances. This allows Sum-of-Square (SOS) programs to be used to find efficient solutions to the problem. An SOS-based iterative algorithm is developed to solve the formulated optimization problem with the constraints represented by the Hamiltonian inequalities. The relation between the original and relaxed H2/H∞performance optimization is discussed in terms of performance comparison. To obviate the requirement of complete knowledge of the system dynamics, a data-driven reinforcement learning approach is proposed to solve the SOS optimization problem in real-time using only the information of the system trajectories measured during a time interval. Finally, a simulation example is provided to show the effectiveness of the proposed algorithm.