Enhanced Prefetching via Dynamic Multistep SARSA-Based Reinforcement Learning
Satyaswaroop Nayak, Anadi Goyal, Sheel Sindhu Manohar, Dip Sankar Banerjee, Palash Das · IEEE Embedded Systems Letters · 2025
Cache prefetchers are essential for system performance, and reinforcement learning (RL) offers a lightweight alternative to heavy machine learning techniques for modern prefetchers. While previous work used 1-step SARSA-based RL for cache prefetching, it can struggle with complex memory access patterns. This study proposes using n-step SARSA instead of 1-step to capture broader contextual learning. We incorporate a dynamic property to reduce the use of outdated Q-values, guiding the RL agent to improve prefetching effectiveness. Our approach achieves approximately 12% better performance than non-RLbased prefetchers in multi-core setups and also outperforms previous RL-based prefetcher.