Deep Multi-Agent Reinforcement Learning With Minimal Cross-Agent Communication for SFC Partitioning
Angelos Pentelas, Danny De Vleeschauwer, Chia‐Yu Chang, Koen De Schepper, Panagiotis Papadimitriou · IEEE Access · 2023
Network Function Virtualization (NFV) decouples network functions from the underlying specialized devices, enabling network processing with higher flexibility and resource efficiency. This promotes the use ofvirtual network functions(VNFs), which can be grouped to form aservice function chain(SFC). A critical challenge in NFV is SFC partitioning (SFCP), which is mathematically expressed as agraph-to-graphmapping problem. Given its NP-hardness, SFCP is commonly solved by approximation methods. Yet, the relevant literature exhibits a gradual shift towards data-driven SFCP frameworks, such as (deep) reinforcement learning (RL). In this article, we initially identify crucial limitations of existing RL-based SFCP approaches. In particular, we argue that most of them stem from the centralized implementation of RL schemes. Therefore, we devise a cooperative deep multi-agent reinforcement learning (DMARL) scheme for decentralized SFCP, which fosters the efficient communication of neighboring agents. Our simulation results (i) demonstrate that DMARL outperforms a state-of-the-art centralizeddouble deep Q-learningalgorithm, (ii) unfold the fundamental behaviors learned by the team of agents, (iii) highlight the importance of information exchange between agents, and (iv) showcase the implications stemming from various network topologies on the DMARL efficiency.