An Adaptive Deep Reinforcement Learning Framework for AUV Attack-Defense Games
Wenhao Gan, Lei Qiao · IEEE Transactions on Intelligent Transportation Systems · 2025
This paper develops a Multi-Agent Deep Reinforcement Learning (MADRL) framework for underwater attack-defense scenarios with constrained sensing/communication. While maintaining balanced learning capacities, attackers exhibit higher mobility whereas defenders hold numerical superiority. The objective is to develop a versatile defense strategy that indirectly manipulates the attackers’ payoff function, compelling them to choose retreat over engagement. Firstly, a dynamic game model with sensing and communication constraints is developed, leveraging the Apollonius circle to quantify underwater attack-defense interactions. This model lays the foundation for payoff functions design and the optimization of defense strategies that exploit terrain advantages to deter speed-advantaged attackers. Secondly, a adaptive AUV decision-making framework is proposed, where the Actor model combines situational awareness with soft attention for dynamic threat prioritization and multi-scale game expansion. The Critic model employs a twin architecture to stabilize multi-scale training while resolving credit assignment. Finally, a decoupled-timescale training method is developed, enabling defensive AUVs to effectively generalize and counter learning-capable attackers. The strategy is theoretically proven to converge to the Nash equilibrium. Comprehensive studies show that the proposed scheme outperforms mainstream MADRL methods in learning performance, and surpasses other defense strategies in win rate and efficiency. Experiments with attackers employing diverse strategies across varying game scales and environmental settings demonstrate the defense strategy’s strong coordination and generalization, the field trial validates its practical feasibility.