A software framework for efficient preemptive scheduling on GPU
Xipeng Shen, Huiyang Zhou, G. Chen · NCSU Libraries Repository (North Carolina State University Libraries) · 2016
Modern GPU is broadly adopted in many multitasking environments, including data centers and smartphones.However, the current support for the scheduling of multiple GPU kernels (from different applications) is limited, forming a major barrier for GPU to meet many practical needs.This work for the time demonstrates that on existing GPUs, efficient preemptive scheduling of GPU kernels is possible even without special hardware support.Specifically, it presents EffiSha, a pure software framework that enables preemptive scheduling of GPU kernels with very low overhead.EffiSha consists of a set of APIs, an innovative preemption-enabling code transformation, a new runtime manager, and a set of preemptive scheduling policies.The APIs redirect GPU requests to the runtime manager, which runs on the CPU side as a daemon and decides which GPU request(s) should be served and when a kernel should get preempted.The preemption-enabling code transformation makes GPU kernels voluntarily evict from GPU without the need to saving and restoring kernel states.We demonstrate the benefits of EffiSha by experimenting a set of preemptive scheduling policies, which show significantly enhanced support for fairness and priority-aware scheduling of GPU kernels.