T-CacheNet: Transformer-based Deep Reinforcement Learning for Next-Generation Internet Content Caching
H J Kim, Teh-Jen Sun, Eui‐Nam Huh · 2024
The rapid growth of internet users and mobile data traffic has led to increasing demand for efficient content caching strategies. Traditional caching algorithms such as Least Recently Used (LRU) and Least Frequently Used (LFU) struggle to adapt to dynamic environments, resulting in frequent cache misses, inefficient content retrieval, and degraded user experience. A key challenge in content caching is deciding both whether to cache newly requested content and which cached content to replace when capacity is limited. Reinforcement learning (RL)-based approaches offer a dynamic solution by continuously optimizing these decisions based on real-time content demand. In this paper, we propose T-CacheNet, a Transformer-based RL model for content caching, leveraging the self-attention mechanism to effectively manage long-term dependencies in content requests. We introduce two techniques—Miss-based delayed hit reward learning and partial information learning—to address the computational challenges of Transformer models while maintaining high efficiency in making caching and replacement decisions. T-CacheNet significantly improves adaptability in dynamic content delivery networks by increasing cache hit rates and preventing inefficient resource utilization.