EC-TRL: Evolutionary-Weighted Clustering and Transformer-Augmented Reinforcement Learning for Dynamic Resource Scheduling in Edge Cloud Environments
Xu Zhou, Jing Yang, Yijun Li, Shaobo Li, Zhidong Su, Jialin Lu · IEEE Internet of Things Journal · 2024
With the rapid development of edge computing, devices now offer powerful computing capabilities and diverse applications. However, the surge in smart devices accessing the Internet overwhelms edge servers, which have limited and unevenly distributed resources. This results in challenges like energy management, load balancing (LB), real-time performance, and system complexity. Existing research fails to comprehensively consider these challenges’ combined impact, making it difficult to maximize performance when facing real complex scenarios. To address the above issues, this article proposes an edge cloud resource scheduling scheme based on evolutionary-weighted clustering and transformer-augmented reinforcement learning (EC-TRL). First, server nodes are deployed at the center of user clusters, based on user device locations, to optimize communication delay and evenly distribute resources. Second, the multiobjective scheduling optimization problem under delay constraints is converted into a Markov decision problem, and a deep reinforcement learning method based on soft actor-critic (SAC) is proposed. Finally, actor transformer (AT) and critic transformer (CT) are proposed to improve the network structure of SAC, capture long-term dependencies and complex patterns in long task scheduling sequences, and improve the model’s adaptability and generalization performance in complex dynamic environments. Through comparison experiments with round robin, random, proximal policy optimization, dueling double deep Q-learning network, SAC-L, and SAC-M, the results show that the proposed method improves the optimization performance of energy consumption, LB, and rejection rate of edge cloud resource scheduling by at least 9.57%, 10.90%, and 5.05%.