SensiX: A System for Best-effort Inference of Machine Learning Models in Multi-device Environments
Chulhong Min, Akhil Mathur, Alessandro Montanari, Fahim Kawsar · IEEE Transactions on Mobile Computing · 2022
Multiple sensory devices on and around us are on the rise and require us to redesign a system to make an inference of ML models accurate, robust, and efficient at the deployment time. While this multiplicity opens up an exciting opportunity to leverage sensor redundancy, it is still extremely challenging to benefit from such multiplicity and boost the runtime performance of deployed ML models without model retraining and engineering. From our experience, we uncovered two prime caveats, device and data variabilities, that affect the runtime performance of ML models. We develop an ML system that addresses these variabilities without modifying deployed models by building on prior algorithmic work. It decouples model execution from sensor data and employs two essential operations between them: a) device-to-device data translation for principled mapping of training and inference data and b) quality-aware dynamic selection of the execution pipeline as a function of runtime accuracy. We evaluate the system on wearable devices with motion and audio-based models. The results show that ML models achieve a 7-13% increase in runtime accuracy solely by running on our system, and the increase goes up to 30% in dynamic environments, at the expense of 3 mW on the host device.