Model-Based and Model-Free Learning-Based Caching for Dynamic Content
Bahman Abolhassani, Atilla Eryılmaz, Y. Thomas Hou · 2024
We propose a "fresh" learning-based caching framework for content distribution networks (CDNs) with distributed front-end local caches and a dynamic back-end database. Users prefer the latest versions of dynamically updated content, while local caches lack knowledge of item popularity and refresh rates. We first explore scenarios with Poisson arrivals at the local cache and characterize the optimal policy’s structure. Building on this, we introduce a model-based learning algorithm for caching dynamic content, demonstrating near-optimal costs and strong performance with limited cache sizes in simulations. For more general environments, we present a model-free Reinforcement Learning (RL) caching policy without prior statistical assumptions. Although model-free RL caching outperforms the model-based approach in high-variance arrival scenarios, it requires significantly longer training times due to its exploration phase. Model-based learning’s quick adaptability to environmental changes and fast convergence rate make it a desirable approach for dynamic network environments, offering efficient fresh caching solutions for CDNs.