HIFAS: A Hybrid Interactive FPS Agent System for Large Game Maps
Chen Zhang, Huan Hu, Yuan Hua Zhou, Xu Jie Wang, Elvis S. Liu · IEEE Transactions on Games · 2025
First-person shooter (FPS) games have consistently been one of the most popular genres in gaming, encouraging competition among players and featuring complex and diverse maps. With the advancement of graphical technology, the complexity of FPS game maps has increased, and their scale has expanded accordingly. The application of deep reinforcement learning (DRL) in modern FPS games has attracted much research interest. Previous work has largely focused on developing DRL agents that can surpass human players, often selecting more laboratory-like environments. These environments tend to be smaller, simpler, and emphasize short-term competitive scenarios while neglecting the complex maps found in modern FPS games. This paper presents the design and deployment of a DRL agent system in a modern FPS game. We have implemented an interactive DRL agent in the multiplayer online 3D FPS gameArena Breakout, published by Tencent Games in 2022.Arena Breakoutfeatures large complex maps and intricate gameplay mechanics, necessitating the deployed DRL system to manage complex action and state spaces. To address the global navigation challenges posed by large maps and the complexities of the state-action space, we designed the Hybrid Interactive FPS Agent System (HIFAS). This system incorporates a multi-modal state perception module to enhance the perception capabilities of the DRL agent. By introducing rule-enhanced reinforcement learning, we tackle the global navigation issues arising from large maps and the realistic shooting problems within the complex action space. We deployed this system in the actual game and integrated it into the long-term operation of the game to validate the system's effectiveness.