Monoceros: A New Approach for Training an Agent to Play FPS Games
Ruiyang Yang, Hongyin Tang, Beihong Jin · 2020
In the deep reinforcement learning, the sparse reward problem directly impacts the quality of agent training. Existing methods have not been satisfactory, especially for the scenarios with high-dimensional state information. In this paper, we propose a new approach Monoceros to training a game agent. Monoceros can work for the scenarios with high-dimensional state information and alleviate the sparse reward problem during the agent training. Specifically, we present a composite reward function which combines both the knowledge implied in expert trajectories and manually-set reward functions. Moreover, we design a specific policy network to adapt to the high-dimensional information scenarios, and adopt the behavior clone as a pre-training strategy to accelerate the training process. Technically, Monoceros can be applied to train the agents to play First Person Shooter (FPS) games. We conduct extensive experiments on three scenarios in the VIZDoom platform. Experimental results show that in all the scenarios, the agent trained by Monoceros outperforms the agents trained by Arnold and GAIL, which are representative methods in the deep reinforcement learning and the imitation learning, respectively.