Signaling-Driven Incentive Communication for Enhanced Multiagent Reinforcement Learning in Dynamic Environments
Kexing Peng, Pengyi Li, Jianye Hao · IEEE Transactions on Cybernetics · 2025
Centralized training and decentralized execution (CTDE) frameworks in cooperative multiagent reinforcement learning (MARL) address nonstationarity and scalability in dynamic environments. However, coordination among agents remains challenging due to limited observability, often leading to inefficient exploration of policy spaces and increased communication overhead. Existing communication mechanisms partially alleviate these issues but typically add complexity without adapting to changing conditions. We propose the signaling-driven incentive communication (SDIC) framework, a novel approach that integrates Markov signaling games (MSGs) into CTDE to enable more efficient and targeted interagent communication. By integrating value-based methods with sparse communication, SDIC reduces unnecessary exchanges while generating tailored signals that enhance policy alignment and improve coordination. Furthermore, SDIC incorporates partner modeling, allowing agents to anticipate the behavior of others and thus strike an effective balance between communication efficiency and computational complexity. Our experimental results, including extensive evaluations in StarCraft II and SUMO traffic simulations, demonstrate SDIC's superior coordination, task success, and communication efficiency with manageable computational complexity. Ablation studies validate the critical roles of SDIC's components in reducing overhead and ensuring effective policy alignment.