Position-Aware Communication via Self-Attention for Multi-Agent Reinforcement Learning
Tsan-Hua Shih, Hsien-I Lin · 2020
Multi-agent reinforcement learning is important for real-world applications but is still a challenging problem. A feasible way is to share information among all agents via a communication channel. In recent years, attentional communication emerged in order to differentiate valuable information especially with a large number of agents. However, existing attentional communication, which relies on long short-term memory (LSTM) units with attention mechanism, makes parallelization difficult. Another problem is that the output of LSTM is dependent on the sequence of agents, but the relationship between agents, in general, is not sequential. In this paper, we proposed using multi-head self-attention layer, which is proposed from a well-known net “Transformer”, as a communication channel for parallelization. Attention mechanism is independent of the sequence of agents. In addition, we also incorporate position information via positional encoding. In our experiments, the proposed method achieves improvements in terms of reward compared with existing approaches.