A Value Factorization Method for MARL Based on Correlation between Individuals
Xiong Liqin, Lei Cao, Xiliang Chen, Jun Lai, Xijian Luo · Mathematical Problems in Engineering · 2022
Value factorization is a popular method for cooperative multi-agent deep reinforcement learning, which effectively solves explosion of state-action spatial dimension and partial observability problems. However, most existing algorithms only consider the impact of individuals rather than correlation between individuals, which leads to poor coordination between agents in complex environments. In order to resolve this problem, this paper proposes a multi-agent deep reinforcement learning value factorization method based on correlation between individuals, CI-VF, which promotes coordination between agents effectively. Firstly, the individual value function vectors are obtained according to the output of individual networks in each round. Secondly, a Spearman correlation coefficient matrix can be calculated by the vectors to measure the correlation degree of agents, and the joint correlation coefficient can be obtained to optimize joint value function. Next, we use optimized joint value function to train individual networks. Experimental results show that our method outperforms QMIX and other baselines in various scenarios under the StarCraft Multi-Agent Challenge environment.