Inverse Reinforcement Learning Control for Linear Multiplayer Games
Bosen Lian, Vrushabh S. Donge, Frank L. Lewis, Tianyou Chai, Ali Asghar Davoudi · 2022 IEEE 61st Conference on Decision and Control (CDC) · 2022
This paper proposes model-based and model-free inverse reinforcement learning (RL) control algorithms for multiplayer game systems described by linear continuous-time differential equations. Both algorithms find the learner the same optimal control policies and trajectories as the expert, by inferring the unknown expert players’ cost functions from the expert’s trajectories. This paper first discusses a model-based inverse RL policy iteration that consists of 1) policy evaluation for cost matrices using a Lyapunov equation, 2) state-reward weight improvement using inverse optimal control (IOC), and 3) policy improvement using optimal control. Based on the model-based algorithm, an online data-driven inverse RL algorithm is proposed without knowing system dynamics or expert control gains. Rigorous convergence and stability analysis of these algorithms are provided. Finally, a simulation example verifies our approach.