A Novel Two-Step Job Runtime Estimation Method Based on Input Parameters in HPC System
Qiqi Wang, Jing Li, Shuo Wang, Guibao Wu · 2019
Accurate job runtime estimation is one of key parts of scheduling strategy design in high performance computing system. The job characteristics generally contain the execution time and the outer layout parameters such as the consumed processor numbers, the user-estimated execution time and the job ID. Existing researches concentrate on proposing better machine learning methods to achieve accurate job runtime estimation. In this paper, multiple extra job characteristics are introduced to determine job execution pattern, which in turn will help acquire a refined model. Through combining a novel two-step job runtime estimation with a new fusion approach, we get the final job execution time prediction. Experimental results show that our algorithm can improve the accuracy of job runtime estimation up to 18.8%, and the weighted absolute error is 13.8% lower than the baseline.