DDPG-Edge-Cloud: A Deep-Deterministic Policy Gradient based Multi-Resource Allocation in Edge-Cloud System
Arslan Qadeer, Myung Joo Lee · 2022
5G and beyond is the key enabler for extreme mobile-broadband (xMBB), Massive and Ultra-reliable machine-type communication (mMTC, uMTC). To handle such large-scale and real-time traffics, Edge Cloud (EC) plays a critical role to minimize the latency and provide compute power at the edge of the network for Internet of Things (IoTs). However, an EC endures limited compute capacity in contrast with the back-end cloud (BC). Intelligent resource management techniques become imperative in such resource constrained environment. This paper studies the problem of compute and wireless resource allocation in an integrated EC and BC environment. Machine learning-based techniques are emerging to solve such optimization problems. However, it is challenging to adopt traditional discrete action space-based methods since they do not scale well in large-scale environments. To this end, to overcome the bottlenecks of wireless bandwidth and compute capacity in resource constrained EC and BC, we propose continuous action space-based DDPG-Edge-Cloud, a deep deterministic policy gradient-based multi-resource allocation (MRA) framework with a pruning principle. The proposed agent is equipped with a Conv1D residual block, gated recurrent unit (GRU) layer and an attention layer for local and long-term temporal feature extraction. We validate the proposed framework by comparing with two alternative agents. Experimental results demonstrate that our proposed agent converges fast and achieves up to 55% and 86.5% reduction in operational cost and rejection rate, and achieves up to 115% gain in the quality of experience on average.