Predictive Dynamic Scheduling Approach for Heterogeneous Resource Pool
Yajing Kang, Jizhuang Zhao, Jiasheng Yang, Shuai Cheng, Kun Wang · 2022 IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC ) · 2022
In recent years, heterogeneous resource pools play an increasingly important role in high-performance computing. There are some challenges for the existing scheduling algorithms due to the increasing complexity of jobs submitted by users on the heterogeneous resource pools. This paper proposed a heterogeneous GPU cluster scheduling approach based on predicting job resource usage to obtain the optimal scheduling strategy, which constructs a prediction model based on the operation of the classified jobs on each heterogeneous chip. The core of the approach is to predict the computing usage characteristics of each job over a period and deploy them to appropriate nodes, when new jobs are waiting in the resource pool. The scalability and utilization of the resource pool are significantly improved, especially in the complex and heterogeneous GPU cluster environment. In addition, the approach is capable of decoupling and separating the jobs and scheduling them in parallel. Experimental results show the execution efficiency of the system is dramatically increased.