Cutting-Edge Inference: Dynamic DNN Model Partitioning and Resource Scaling for Mobile AI
Jeong-A Lim, Joohyun Lee, Jeongho Kwak, Yeongjin Kim · IEEE Transactions on Services Computing · 2024
Recently, applications using artificial intelligence (AI) technique in mobile devices such as augmented reality have been extensively pervasive. The hardware specifications of mobile devices, dynamic service demands, stochastic network states, and characteristics of DNN (Deep Neural Network) models affect the quality of experience (QoE) of such applications. In this paper, we proposeCutEdge, that leverages a virtual queue-based Lyapunov optimization framework to jointly optimize DNN model partitioning between a mobile device and a mobile edge computing (MEC) server and processing/networking resources in a mobile device with respect to internal/external system dynamics. Specifically,CutEdgemakes decisions of(i)the partition point of DNN model between the mobile device and MEC server,(ii)GPU clock frequency, and(iii)transmission rates in a mobile device, simultaneously. Then, we theoretically show the optimal trade-off curves among energy consumption, throughput, and end-to-end latency yielded byCutEdgewhere such QoE metrics have not been jointly addressed in the previous studies. Moreover, we show the impact of joint optimization of three control parameters on the performances via real trace-driven simulations. Finally, we show the superiority ofCutEdgeover the existing algorithms by experiment on top of implemented testbed using an embedded AI device and an MEC server.