Learning to Steer Swarm-vs.-swarm Engagements
Laura G. Strickland, Charles E. Pippin, Matthew Gombolay · AIAA Scitech 2021 Forum · 2021
View Video Presentation: https://doi.org/10.2514/6.2021-0165.vid UAVs are becoming increasingly commonplace, and with their growing popularity, the question of how to counter a swarm of UAVs operated by bad actors becomes more critical. In this paper, we explore the possibility of using a team of fixed-wing UAVs to counter an adversarial swarm of fixed-wing UAVs. To learn to coordinate counter-swarm tactics, we propose Situation-Dependent Option-action Evaluation (SDOE), a distributed and scalable actor-critic RL architecture. Our approach enables each UAV to evaluate options over a set of scripted tactics as well as the option to maneuver freely, allowing for emergent team behavior. A key to the scalability of our approach is a novel, distributed neural network architecture that enables agents to share situational awareness and select tactics in a pairwise fashion, allowing agents to choose who to coordinate with, when, and how regardless of the size of the swarm. We test agents trained with our approach in simulated engagements of up to 16-vs.-16 UAVs, and find that, even as the size of the engagement increases, the agents trained using SDOE against a greedy, non-coordinating tactic win engagements against a team of greedy agents more reliably than another team of greedy agents.