Diffusion-Based Multi-Agent Reinforcement Learning with Communication
Xinyue Qi, Jianhang Tang, Jiangming Jin, Yang Zhang · 2024
Multi-agent systems (MAS) have been widely used as a modeling tool to analyze the behaviors of members in groups, such as swarms, autonomous vehicle fleets, and smart Internet of Things devices. With their distributed architecture and model-free nature, reinforcement learning (RL) is often deployed to address issues of decision-making and action-taking in multi-agent systems to optimize overall system performance. However, as multi-agent systems involve agent coordination and interactions, this leads to extensive communication and computation overhead, resulting in decreased performance and slower convergence of multi-agent reinforcement learning. This work integrates a diffusion model into RL training to accelerate decision-making, taking into account communication among system agents. The effectiveness of this method is validated in a grid-based multi-agent predator-prey system scenario, where agents interact and confront each other via mutual communication networks. Experimental results show significant improvements in terms of learning efficiency and task performance.