EdgeDQN: Multiple SFC Placement in Edge Computing Environment
Suman Pandey, Tu Van Nguyen, Jae‐Hyoung Yoo, James Won‐Ki Hong · 2021
Network Function Virtualization (NFV) and service orchestration has simplified the Service Function Chain management (SFC) tasks, while the edge cloud infrastructure has reduced the latency. Due to these existing technological advantages, there is an urgent need for a dynamic and flexible service chain placement model that performs resource allocation of substrate network in a delay-sensitive and resource-efficient manner. We propose an off-policy Deep Reinforcement Learning algorithm EdgeDQN for efficient SFC placement in the edge cloud environment. The problem of edge resource scarcity is handled by designing a network model that allows worst-case resource renting from neighbors and data centers. This network model is integrated with EdgeDQN using several constraints. This paper aims to find the optimal placement by minimizing the underlying resource utilization and SFC end-to-end delay for multiple SFCs at the same time. To achieve that, an intuitive reward model is proposed. We compare the proposed EdgeDQN algorithm with DQN, Q-learning, and EdgeQL algorithms in terms of performance parameters such as cumulative reward, cumulative standard deviation, latency, and learning convergence time for 420 different test cases. Extensive test results on a simulated and physical (OpenStack) testbed demonstrate the effectiveness of the proposed EdgeDQN algorithm.