Reinforcement Learning Environment to Realistic Simulations for Multi-Agent System Validation and Deployment
Seth Johnson, Laura Escamilla, Hannah C. Lehman, John Valasek · 2025
Reinforcement learning has been used to train autonomous single-agent and multi-agent systems to solve complex problems in the aerospace domain. This type of machine learning is a long process, often requiring training to run millions of time steps to achieve practical solutions. Training environments are often simplified to expedite training, such as using reduced vehicle dynamics and environmental abstractions to remove dependency on visual-based graphics. However, these complexities may be favorable when realism and accuracy are prioritized over run-time during full autonomous system validation. This paper introduces and develops an architecture that facilitates the transition from a reduced training environment to more realistic simulations. Modularity is emphasized to reduce implementation time and scalability to support large multi-agent systems. The proposed system leverages Unreal Engine 5 for high-fidelity graphics, optimized performance, and developer-oriented workflow to create a simulation for demonstrating and validating multi-agent systems for real-world deployment. The integration of Software-in-the-Loop (SITL) and Hardware-in-the-Loop (HITL) further validate the simulation framework, allowing for detailed testing and adjustment phases. Results presented in the paper investigate case studies on the validation for deployment of a single-agent autonomous tracking UAS and extend the proposed framework to an auction-based multi-agent coordination scenario.