Imitation Learning From Inconcurrent Multi-Agent Interactions
Xin Zhang, Weixiao Huang, Yanhua Li, Renjie Liao, Ziming Zhang · 2021 60th IEEE Conference on Decision and Control (CDC) · 2021
Multi-agent imitation learning (MA-IL) aims to inversely learn policies for all agents using demonstrations collected from an expert group. However, this problem has only been studied in the setting of Markov games (MGs) allowing participants for concurrent actions, and do not work for general MGs, with agents inconcurrently making decisions in different turns. In this work, we propose iMA-IL, a novel multi-agent imitation learning framework for general (inconcurrent) Markov games. The learned policies are proven to guarantee subgame perfect equilibrium (SPE), a stronger equilibrium than Nash equilibrium (NE). The experiment results demonstrate that compared to state-of-the-art baselines, our iMA-IL model can better infer the policy of each expert agent using their demonstration data collected from inconcurrent decision-making scenarios.