Feature-Based Local Ensemble Framework for Multi-Agent Reinforcement Learning
Xinyu Zhao, Jianxiang Liu, Faguo Wu, Xiao Zhang · 2024
The use of centralized value networks is an important training method in multi-agent reinforcement learning (MARL). Existing methods usually utilize value decomposition to enable agents to achieve localized learning. However, these methods do not take into account sample efficiency, and the introduction of additional networks will also bring more computational costs. Therefore, it is necessary to introduce the ensemble idea of making full use of the replay buffer. The general ensemble learning approach is to introduce sub-models into the value network, and its application in multi-agent systems is not yet mature. In this paper, we firstly implement a multi-agent ensemble technique with locally shared parameters in the network by introducing a feature-based grouping mechanism. Secondly, we propose random and feature-based grouping principles and design a dimension reduction scheme for action-state space. Finally, our approach was evaluated on the benchmark platform, Multi-Agent Particle Environment (MPE), and compared with baseline algorithms. The results demonstrated that our method exhibited notable advantages in terms of achieving higher scores and converging rapidly.