Improving Cooperation via Joint Intrinsic Motivation Exploration in Multi-Agent Reinforcement Learning
Hongfei Du, Zhiyou Yang, Yunhan Wu, Ning Xie · 2024
Multi-agent reinforcement learning (MARL) encoun-ters the enduring challenge of sparse rewards, which becomes particularly apparent in scenarios requiring coordinated actions among agents. To handle this issue, we consider adding an intrinsic reward to the environmental reward for enhancing the policy exploration capabilities in multi-agent cooperation settings. Our approach focuses on incentivizing strategic behaviors characterized by collective novelty among agents. Specifically, we introduce a self-supervised learning model to measure the novelty of diverse coordination patterns within a team of agents. To this end, we present a multi-agent intrinsic motivation framework called Joint Intrinsic Motivation Exploration (JIME) that adheres to the centralized learning with decentralized execution paradigm. Empirical evaluations demonstrate the crucial role of JIME in addressing tasks that require intricate coordination for optimal strategy execution. Our findings underscore the significance of incorporating intrinsic motivation mechanisms in MARL systems to facilitate effective collaboration among agents.