Accelerated Deep Reinforcement Learning for Wireless Coded Caching
Zhe Zhang, Meixia Tao · 2019
Coded caching is effective in leveraging the accumulated storage size in wireless networks by distributing different coded segments of a file in different cache nodes. This paper aims to find a wireless coded caching policy to minimize the long-term network cost, which involves both transmission delay and cache replacement cost, using tools from machine learning. The problem is known to be challenging due to the unknown content popularity and the continuous, high-dimensional action space. We first predict the number of file requests using historical information. Based on the predicted results, we then propose a supervised deep deterministic policy gradient (SDDPG) approach. This approach, on one hand, can learn the caching policy in continuous action space by using the actor network to make cache decision and the critic network to evaluate cache decision. On the other hand, it accelerates the learning process by pre-training the actor network with the results of an approximate problem that minimizes the per-slot cost. Real-world trace-based numerical results show that the proposed SDDPG approach is effective.