Mastering First-person Shooter Game with Imitation Learning
Yue Ying · 2022
Imitation learning has been heavily studied as a technique for solving sparse-rewarding sequential decision problems such as video games and robotics. Previous research has mostly focused on training agents with large-scale behavioural cloning. However, behavioural cloning can perform poorly if the size of the dataset is not big enough, while large-scale labelled human demonstrations are usually unavailable. We propose a new imitation learning framework called Imitational Actor-Critic (IAC), which unifies behavioural cloning and actor-critic based reinforcement learning. Specifically, we show that with a small amount of human demonstrations we can train an agent that masters the first-person shooter (FPS) game. Our model obtains performance gains over behavioural cloning counterparts on hard-exploration tasks that are impossible to learn from scratch via reinforcement learning.