Actor-Critic with Transformer for Cloud Computing Resource Three Stage Job Scheduling
Yanbo Xu, Jiakun Zhao · 2022
Cloud computing is widely used in academia and industry. Users can provide jobs to cloud service providers that provide pay-as-you-go methods by cloud for calculation and obtaining results. Cloud service providers make reasonable scheduling for different jobs of users and in the case of successful operation, the cost of energy consumption becomes a key factor. Prior works proposed various algorithms based on reinforcement learning on job scheduling. However, due to the lack of online training mode and refined container decision-making, this paper makes some improvements and extensions. This paper presents the algorithm ACT4JS, an actor-critic with transformer for three stage job scheduling algorithm, that finely manages data centers, server nodes, and application containers. The job scheduling algorithm is designed to select the best long-term decision by learning from cloud computing resource model that accepts single-task multi-instance jobs and directed acyclic graph jobs with dependencies. To learn key features and loss optimization, use the transformer network structure and the proximal policy optimization method. To minimize energy consumption costs for large scale job scheduling and running, designs a reward function to reflect optimized energy consumption targets and independent simulation experiments. To achieve low energy consumption cost and low task reject rate, use training methods such as experience replay and advantage estimate. Compared with the mainstream baseline algorithm, the results show that the ACT4JS algorithm is better than the baseline in the online training model. In addition, the effect of ACT4JS is more effective in the long-term experimental.