Parallel Backfill: Improving HPC System Performance by Scheduling Jobs in Parallel
Donald Riffel, Daniel Andresen, Scott Hutchison, William Hsu · 2024
High-performance computing (HPC) clusters are widely used as a platform for scientific and engineering research as well as a broad range of data analysis tasks. Demand for HPC resources continues to grow, necessitating more scalable systems and improved management of cluster resources. Job scheduling algorithms are key components of managing the allocation of cluster resources. A common algorithm that is used in many production systems is backfilling, which provides an efficient and feasible approach to scheduling. Many variations of backfilling have been created and studied which aim to improve its performance, but there are still opportunities in this field. In this paper, we propose a new approach named Parallel backfilling which improves scheduling throughput without increasing execution time in production environments. Our concept is to allow for multiple backfill "workers" to process the waiting job queue in parallel, increasing the rate of scheduled jobs and thus improving system turnaround time for users. We present simulated results based on job traces from the Beocat HPC cluster at Kansas State University that show significant improvement in average job wait times and scheduler throughput. We conclude that Parallel backfill provides better performance than traditional backfill and some of its variants, and compare our results with a selection of scheduling optimizations.