The differences between VDN and COMA algorithms in PyMARL and implications for their win rate in different SMAC scenarios
Jiajun Song · 2022
In the current SMAC paper, almost no one has analyzed the reason of the difference of success rate% between PyMARL algorithms used in SMAC. The purpose of this article is to study the code differences and the idea differences between VDN algorithm and COMA algorithm in PyMARL framework used in SMAC environment, and the influence of these two algorithms on the result of SMAC experiment. The conclusion of this paper is that VDN algorithm is superior to COMA algorithm in SMAC. Because in almost all scenarios of SMAC, the algorithm based on value function (VDN) can solve simple cases very quickly and effectively, while the algorithm based on strategy gradient (COMA) converges to local minimum easily, and it is extremely difficult for COMA to come up with a perfect strategy.