An Analysis of Multi-Agent Reinforcement Learning for Emergent Target Attack Strategies
Eugen Victor Cristian Rusu · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2024
Choosing parameters in the process of modeling scenarios in multiagent simulation design is an inherit challenge.Scenario design decisions include defining the rules and rewards governing the simulated environment and specifying the capabilities and perception properties of the agents.We present a detailed analysis of over 50 Multi-Agent Reinforcement Learning training experiments using a novel stylized target-attack type game, wherein agents must learn to cooperate to accomplish a shared objective.We consider parameters such as the number of cooperating agents, agent perception limitations, the complexity and size of the simulation, and various reward schemes.Our results highlight the effectiveness of limited perception over full perception in both training performance and efficiency of resulting policies.Additionally, we discuss the impact of sparse team-based rewards vs shaped individual rewards, revealing emergent dynamics under different rewards schemes and simulation parameters and the nuanced strategies developed by cooperating agents.Our analytic approach provides insights into the relationship between simulation parameters and the emergent outcomes, thereby providing valuable information for the refinement and optimization of simulation parameters.