Multi-agent reinforcement learning with layered autonomy and collaboration for enhanced collaborative confrontation
Xiaoyu Xing, Haoxiang XIA · Chinese Journal of Aeronautics · 2025
Addressing optimal confrontation methods in multi-agent attack-defense scenarios is a complex challenge. Multi-Agent Reinforcement Learning (MARL) provides an effective framework for tackling sequential decision-making problems, significantly enhancing swarm intelligence in maneuvering. However, applying MARL to unmanned swarms presents two primary challenges. First, defensive agents must balance autonomy with collaboration under limited perception while coordinating against adversaries. Second, current algorithms aim to maximize global or individual rewards, making them sensitive to fluctuations in enemy strategies and environmental changes, especially when rewards are sparse. To tackle these issues, we propose an algorithm of Multi-Agent Reinforcement Learning with Layered Autonomy and Collaboration (MARL-LAC) for collaborative confrontations. This algorithm integrates dual twin Critics to mitigate the high variance associated with policy gradients. Furthermore, MARL-LAC employs layered autonomy and collaboration to address multi-objective problems, specifically learning a global reward function for the swarm alongside local reward functions for individual defensive agents. Experimental results demonstrate that MARL-LAC enhances decision-making and collaborative behaviors among agents, outperforming the existing algorithms and emphasizing the importance of layered autonomy and collaboration in multi-agent systems. The observed adversarial behaviors demonstrate that agents using MARL-LAC effectively maintain cohesive formations that conceal their intentions by confusing the offensive agent while successfully encircling the target.