CPRL: Change Point Detection and Reinforcement Learning to Optimize Cache Placement Strategies
Javane Rostampoor, Raviraj Adve, Ali Afana, Yahia Ahmed · IEEE Transactions on Communications · 2023
Placing selected content at the edge of the network close to the users, known as caching, is an important technique to improve the efficiency of content delivery in wireless networks. In this paper, we consider caching in a cloud radio access network (C-RAN) in which the primary fronthaul link operates in the mmWave range and may switch to microwave frequencies in the case of blockage. We aim to minimize the average long-term network cost by optimizing dynamic fetching and caching decisions. Importantly, we consider the realistic case of user request distributions and blockage rates being a priori unknown and not necessarily stationary. We introduce change point detection (CPD) to detect significant changes in the environment; we couple this step with reinforcement learning (RL): our key contribution, the proposed change point detection assisted reinforcement learning (CPRL) algorithm learns the environment and (re-)optimizes the caching policy to solve the associated Markov decision process (MDP) problem. Essentially, CPD allows our learning algorithm to adapt its caching strategy to the new environment which shows faster convergence. The numerical results show that our proposed approach improves the efficiency of caching in wireless networks, making it more adaptable to changing request patterns over time.