DEFIANCE: Simulate Reinforcement Learning Deployments in Realistic Network Scenarios
Anna-Birgitta Burmeister, J. P. Dittrich, Anna M. Fennig, Tobias Rothe, Rahul Singh · 2025
Reinforcement Learning (RL) is increasingly used in networking research.Since real-world experiments are rarely feasible, new RL approaches are typically trained and evaluated in network simulations.When implementing a new RL approach in the popular network simulator ns-3, one faces two main challenges: (1) integrating an RL library into ns-3 to leverage existing RL algorithms, and (2) implementing the distributed deployment of the RL approach in the ns-3 simulation.Since there is currently no tool that sufficiently simplifies both tasks, this paper presents the DEFIANCE framework.It defines abstract RL applications in ns-3, into which users insert their specific implementation and which allow an easy simulation of the distributed deployment of different RL components.To integrate RL libraries into ns-3, DEFIANCE extends ns3-ai by supporting Multi-Agent Reinforcement Learning (MARL) and parallel execution of environments.The functionality of the framework is demonstrated by two examples.Overall, the framework paves the way for future RL research in ns-3 by simplifying the implementation of RL approaches and their evaluation under realistic conditions.