Interference-aware Multiplexing for Deep Learning in GPU Clusters: A Middleware Approach
Wenyan Chen, Zizhao Mo, Huanle Xu, Kejiang Ye, Chengzhong Xu · 2023
A common strategy for improving efficiency in training deep learning entails multiplexing tasks on a single GPU. To mitigate the interference caused by multiplexing, existing approaches primarily employ kernel-level solutions to regulate GPU kernel execution, or harness hardware-level techniques to explicitly restrict GPU streaming multiprocessors and memory. Nevertheless, none of them perform satisfactorily in optimizing the completion time of tasks.