Learning to Communicate Using Action Probabilities for Multi-Agent Cooperation
Yidong Bai, Toshiharu Sugawara · 2023
While communication is an essential tool for cooperation in multi-agent systems (MAS), existing approaches generally assume high-dimensional messages genereated by deep networks, which are uninterpretable for humans and require expensive transmission and complex encoding/decoding networks in agents. Uninterpretability may in turn raise reliability and security issues for the systems. To achieve low-dimensional and interpretable communication, we demonstrate that each agent's action probabilities can be used as messages, inspired by the fact that humans often share likely actions during collaboration. Our proposed method, communication based on action probabilities (CAP), is a simple yet effective communication architecture for coordination and collaboration in MAS, which facilitates our understanding of the agents' learned coordinated and cooperative behaviors. Moreover, because messages can be generated by the actor network, which is usually implemented in deep reinforcement learning agents, we can eliminate the requirement of extra message-generator networks. Our experiments show that CAP can achieve comparable performance to those of the state-of-the-art methods, with quicker convergence, simpler network structures and better interpretability.