Diverse Effective Relationship Exploration for Cooperative Multi-Agent Reinforcement Learning
Hao Jiang, Yuntao Liu, Shengze Li, Jieyuan Zhang, Xinhai Xu, Donghong Liu · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · 2022
In some complex multi-agent environments, the types of relationships between agents are diverse and their intensity changes during the policy learning process. Theoretically, some of these relationships can facilitate cooperative policy learning. However, acquiring these relationships is an intractable problem. To tackle the problem, we propose a diverse effective relationship exploration based multi-agent reinforcement learning (DERE) method. Specifically, a potential fields model is firstly designed to represent relationships between agents. Then to encourage the exploration of effective relationships, we define an information-theoretic objective function. Finally, an intrinsic reward function is designed to optimize the information-theoretic objective, meanwhile, guide agents to learn more effective collaborative policies. Experimental results show that our method outperforms state-of-the-art methods on both super hard StarCraft II micromanagement tasks (SMAC) and Google Research Football (GRF).