Quantifying context switch overhead of artificial intelligence workloads on the cloud and edges
Kun Suo, Yong Shi, Chih‐Cheng Hung, Patrick Otoo Bobbie · 2021
Context switching is the fundamental technique for providing flexible and efficient utilization of CPU resources in the multitasking system. However, context switching also introduces non-trivial overhead due to its complicated activities, resulting in not only significant performance degradation of applications, but also dramatic system low efficiency. In this paper, we perform a comprehensive and empirical study on the performance and overhead of context switches in modern artificial intelligence workloads, which identifies unrevealed and important impact on cloud servers and edge/IoT devices. Our observations and root cause analysis cast light on optimizing the system stack of modern operating systems to more efficiently support artificial intelligence workloads on the cloud and edge systems.