CrowdHMTware: A Cross-Level Co-Adaptation Middleware for Context-Aware Mobile DL Deployment
Sicong Liu, Bin Guo, Shiyan Luo, Yuzhan Wang, Hao Luo, Cheng Fang, Xu Yuan, Ke Ma, Yao Li, Zhiwen Yu · IEEE Transactions on Mobile Computing · 2025
There are many deep learning (DL) powered mobile and wearable applications today continuously and unobtrusively sensing the ambient surroundings to enhance all aspects of human lives. To enable robust and private mobile sensing, DL models are often deployed locally on resource-constrained mobile devices using techniques such as model compression or offloading. However, existing methods, either front-end algorithm level (i.e. DL model compression/partitioning) or back-end scheduling level (i.e. operator/resource scheduling), cannot be locally online because they require offline retraining to ensure accuracy or rely on manually pre-defined strategies, struggle withdynamic adaptability. The primary challenge lies in feeding back runtime performance from theback-endlevel to thefront-endlevel optimization decision. Moreover, the adaptive mobile DL model porting middleware withcross-level co-adaptationis less explored, particularly in mobile environments withdiversityanddynamics. In response, we introduce CrowdHMTware, a dynamic context-adaptive DL model deployment middleware for heterogeneous mobile devices. It establishes anautomated adaptation loopbetween cross-level functional components, i.e. elastic inference, scalable offloading, and model-adaptive engine, enhancing scalability and adaptability. Experiments with four typical tasks across 15 platforms and a real-world case study demonstrate that${\sf CrowdHMTware}$can effectively scale DL model, offloading, and engine actions across diverse platforms and tasks. It hides run-time system issues from developers, reducing the required developer expertise.