Tabular Reinforcement Learning Methods for Artificial Intelligence Tasks Offloading in Smart Eye-Wears
Abednego Wamuhindo Kambale, Hamta Sedghani, Federica Filippini, Giacomo Verticale, Danilo Ardagna · ACM Transactions on Autonomous and Adaptive Systems · 2025
Virtual and Extended Reality technologies are increasingly adopted in fields such as healthcare, entertainment, and education. These applications heavily rely on Smart Eye-Wears (SEWs) and AI to provide users with new ways to perceive their environment. However, SEWs face limitations in computational power, memory, and battery life. Offloading computations to external servers is a prominent example of edge computation. However, this also presents considerable challenges due to delays caused by varying network conditions and server workloads. This article proposes self-adaptive techniques based on tabular reinforcement learning (RL) to optimize the offloading of Deep Neural Network tasks between the SEW, the user’s smartphone, and cloud servers. The goal is to maintain a high-quality user experience while minimizing energy consumption and 5G connection costs. We evaluated our framework under varying 5G and WiFi bandwidths and cloud latency. The results show that Q-learning, SARSA, and Expected SARSA achieve near-optimal policies, with Q-learning demonstrating superior performance in reducing execution time violations (approximately at 10%) and improving agent stability. Additionally, our approach offers a more favorable tradeoff between energy efficiency and execution time violations compared to two baseline methods. Real-system experiments reveal that the proposed solution can double SEW battery life with respect to local computation while maintaining a good quality of service, with only 11% execution time violations. These findings highlight the effectiveness of our approach in managing resources and enhancing the overall user experience in SEW AI applications.