SPIN: BSP Job Scheduling With Placement-Sensitive Execution
Zhenhua Han, Haisheng Tan, Shaofeng H.-C. Jiang, Wanli Cao, Xiaoming Fu, Lan Zhang, Francis C. M. Lau · IEEE/ACM Transactions on Networking · 2021
The Bulk Synchronous Parallel (BSP) paradigm is gaining tremendous importance recently due to the popularity of computations as distributed machine learning and graph computation. In a typical BSP job, multiple workers concurrently conduct iterative computations, where frequent synchronization is required. Therefore, the workers should be scheduled simultaneously and their placement on different computing devices could significantly affect the performance. Simply retrofitting a traditional scheduling discipline will likely not yield the desired performance due to the unique characteristics of BSP jobs. In this work, we deriveSPIN, a novel scheduling designed for BSP jobs with placement-sensitive execution to minimize the makespan of all jobs. We first prove the problem approximation hardness and then present howSPINsolves it with a rounding-based randomized approximation approach. Our analysis indicatesSPINachieves a good performance guarantee efficiently. Moreover,SPINis robust against misestimation of job execution time by theoretically bounding its negative impact. We implementSPINon a production-trace driven testbed with 40 GPUs. Our extensive experiments show thatSPINcan reduce the job makespan and the average job completion time by up to$3\times $and$4.68\times $, respectively.SPINalso demonstrates better robustness to execution time misestimation compared with state-of-the-art heuristic baselines.