Deep reinforcement learning with procedural content generation in RTS games
Yifu Li, Santiago Ontañón · 2024
This thesis lies at the intersection of Digital Media (DM) and Artificial Intelligence (AI). In general, we study how techniques developed for game environments can be used to better evaluate AI algorithms. Specifically, we focus on Real-time strategy (RTS) games, which present a challenging environment for artificial intelligence, due to their inherent complexity and dynamic nature. Deep Reinforcement Learning (DRL) has emerged as a promising approach to tackle these challenges, but achieving generalization across diverse scenarios remains an open problem. Procedural Content Generation (PCG) is a family of techniques commonly used in digital media and video games to automatically generate content. This study investigates the role of Procedural Content Generation (PCG) in enhancing the generalization capabilities of DRL agents in RTS games. The research question driving this thesis is: Does PCG increase the generalization of Deep Reinforcement Learning in RTS games? To address this question, we designed and implemented a DRL agent for a chosen RTS game: [mu]RTS, adapting state-of-the-art algorithms to the game's specific requirements. Subsequently, we integrated a PCG system to generate diverse game scenarios, especially maps. By exposing the DRL agent to a variety of procedurally generated scenarios during training, we aimed to foster generalization and adaptability. We evaluated the performance of those DRL agents with PCG through various generalization measures and compared it against a baseline DRL agent without PCG. Our results demonstrate that incorporating PCG significantly improved the generalization capabilities of the DRL agent, enabling it to adapt more effectively to unseen scenarios and to transfer knowledge across different game situations. These findings contribute to the understanding of the interplay between DRL and PCG in RTS games and provide valuable insights for future research in the development of more robust and adaptable AI systems for complex environments.