VELTAIR: towards high-performance multi-tenant deep learning services via adaptive compilation and scheduling
Zihan Liu, Jingwen Leng, Zhihui Zhang, Quan Chen, Chao Li, Minyi Guo · 2022
Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware.