ATG: Adaptive Task-Graph for Multi-Agent Reinforcement Learning in Complex Collaborative Task
Jingwen Yi, Yu Zhang, Xiaolong Su, Qilin Yan, Le Hao, Ziyan Fang · 2025
As the complexity of urban traffic networks increases, the scale of state space and action space in multi-agent systems continues to grow, and the constraints in the cooperation among heterogeneous multi-agent are also increasing, which may lead to dimensionality explosion and slow computation. To further improve the convergence speed of multi-agent reinforcement learning(MARL) in complex collaborative tasks with more constraints, we propose an adaptive task-graph(ATG) model to solve the problems of multi-agent collaboration in complex tasks. Firstly, we analyze the connection between the distribution of agents and how they execute complex tasks, and generate agent collaboration and decision networks by graph attention neural networks. Then, we reshape the action space of agents to enable them to make more effective decisions under complex constraints. In addition, we divide the collaborative task into multiple stages, ensuring that rewards are also distributed during the task completion process, to address the issue of sparse rewards in MARL. We demonstrate that the ATG model outperforms existing methods in complex cooperative tasks and the algorithm convergence speed is significantly improved. It can also enhance the interpretation and the transferability of learned policies to new tasks with different agent compositions.