Off-Policy Reinforcement Learning for $H_\infty$ Control of Linear Discrete-Time Systems With Network-Induced Dropouts
Yi Jiang, Tao Yang, Weinan Gao, Jin Chu Wu, Tianyou Chai, Frank L. Lewis · IEEE Transactions on Automatic Control · 2025
This paper studies an adaptive discrete-time linear$H_\infty$control problem with networked induced dropouts. First, such problem is formulated as a zero-sum game problem, and it is shown that the formulated problem can be solved via an optimal feedback control policy and a worst disturbance policy. These policies result from one positive definite solution to a modified game algebraic Riccati equation (MGARE). Then, the solvability of the MGARE, the stochastic asymptotical stability and the disturbance attenuation level of the closed-loop system are rigorously analyzed. To obtain such solution to the MGARE, two model-based reinforcement learning (RL) algorithms, namely, policy iteration (PI) and value iteration (VI) algorithms, are proposed and their corresponding convergence analysis are given as well. Based on model-based RL algorithms, two data-driven RL algorithms, namely, data-driven PI and VI algorithms, are designed by directly using the data transmitted via communication networks in a model-free sense, in which the optimal control policy and the worst disturbance policy are thus obtained iteratively. Furthermore, a data-driven computation algorithm for drawing the feasible area of such MGARE approximately is designed. Finally, simulation examples are given to show the effectiveness of the proposed approaches.