Efficiency Thresholds of Group Based Job Scheduling in HPC Systems
D. S. Lyakhovets, Anton V. Baranov · Lobachevskii Journal of Mathematics · 2022
Abstract One of mandatory phases of a supercomputer job lifecycle is job preparation or initialization. During initialization, no productive calculation work is performed. High initialization share leads to greater supercomputer idle and decreases its efficiency. To optimize scheduling of jobs with high initialization share, certain methods and algorithms are applied that combine jobs into groups. This article aims at answering the question: what is the minimum initialization share at which, by grouping, the job scheduling efficiency may be increased? In other words, what is the starting threshold where grouping can result in measurable benefit? To answer this question, we developed a simulation model of supercomputer job management system based on Alea simulator and workload generator. Within this model, the algorithm of jobs grouping named Packet was implemented. Our Packet algorithm was compared with standard scheduling algorithms FCFS and Backfill. We performed a series of experiments involving various characteristics of the workload. For different workloads we defined minimal thresholds of initialization share at which Packet algorithm begins enhancing the job scheduling efficiency.