SC-DRL: A Status Correction-Empowered Deep Reinforcement Learning Algorithm for Dependency-Aware Application Offloading
Li Wei Shao, Liping Qian, Ming Qing Li, Wei Jiang, Weijia Jia · IEEE Transactions on Services Computing · 2025
Mobile edge computing (MEC) is emerging as a critical paradigm to meet the growing computational demands of wireless devices. However, edge servers, wireless devices, and service types in MEC networks are usually time-varying due to configurations, traffic patterns, and operational status, which results in inaccurate state estimations. Therefore, existing Deep Reinforcement Learning (DRL)-based offloading algorithms often fail to effectively handle dependency-aware applications. Furthermore, traditional reward functions adopted in DRL-based algorithms fail to decouple historical dependencies among offloading decisions for subtasks, hindering accurate state updates. To address these challenges, we propose a Status Correction-empowered Deep Reinforcement Learning (SC-DRL) algorithm for making the dependency-aware application offloading decisions in this paper. Specifically, we first adopt the State-Adjusted Bellman Equation to ensure accurate updates of DRL state values. Then, we introduce the dynamic estimate equation to enable DRL agents to estimate system states accurately. Furthermore, we mathematically model device load to extend the dynamic estimate equation to handle real-world complexities. Finally, we propose the Reapplying Reward Technology to reduce reward inaccuracy due to historical dependencies. Both simulations and real-world tests show that the SC-DRL improves the ratio of applications completed within their deadlines by an average of 3.36% and 41.94% compared to the state-of-the-art algorithms, such as Advantage Actor-Critic (A2C), Deep Q-Learning (DQN), and Proximal Policy Optimization (PPO).