A Reinforcement Learning Approach to Optimize Cache Prefetcher Aggressiveness at Run-Time
Matthew Joseph Adiletta, Farah Fargo, Mitchell Diamond, Jack Adiletta, Olivier Franza, Simon C. Steely · 2023
Cache prefetching hides memory latencies by speculating about future accesses and increasing data coverage. However, aggressive prefetching may increase cache pollution, bottleneck memory bandwidth, and add latency to critical path demand queues. Managing the aggressiveness of the prefetchers is necessary to mitigate these problems. State of the art hardware prefetcher solutions manage aggressiveness by analyzing telemetry data such as prefetcher accuracy and memory bandwidth consumption. This is an insufficient solution because telemetry data alone does not necessarily correlate with the overall system performance. Furthermore, other solutions presented in literature optimize prefetchers individually, rather than allowing them to work together to improve overall system performance. This paper introduces a novel framework employing reinforcement learning to find the optimal prefetcher aggressiveness policy for multiple prefetchers during run-time, orchestrated by the Aggressiveness Degree Manager (ADM). It is developed in this case for a single-core, single-process environment to optimize Mid-Level Cache (MLC) prefetchers. The ADM agent was evaluated using ten prefetch-sensitive workloads from the SPEC CPU2017 suite and demonstrated a 4.2% increase in instructions per cycle over the state-of-the-art hardware solution and 25.9% increase in instructions per cycle over no prefetching.