Light-weight Cache Replacement for Instruction Heavy Workloads
Saba Mostofi, Setu Gupta, Ahmad Hassani, Krishnam Tibrewala, Elvira Teran, Paul V. Gratz, Daniel A. Jiménez · 2025
The last-level cache (LLC) is the last chance for memory accesses from the processor to avoid the costly latency of accessing the main memory.In recent years, an increasing number of instruction heavy workloads have put pressure on the last-level cache.We find that, for instruction heavy workloads, a simple replacement policy with minimal overhead provides at least the same benefit as a stateof-the-art, high-overhead replacement policy in the presence of aggressive prefetching.Our proposal is based on specifying insertion and promotion vectors (IPVs) as a generalization of re-reference interval prediction (RRIP) in such a way that the space of feasible policies may be searched exhaustively to find the best policy for the training set of workloads.The policies are formulated to deliver the best performance taking into account demand and prefetch accesses.We show that our technique, Prefetch Aware Coarse-grained Insertion and Promotion Vectors (PACIPV), improves performance over a state-of-the-art LLC replacement policy (Mockingjay) for instruction heavy workloads, and remains competitive for data heavy workloads with significantly less hardware overhead.We show that RRIP-based IPVs are very easy to implement but outperform far more complex replacement policies.PACIPV achieves a speedup of 3.3% over the baseline of LRU, outperforming SRRIP by 1.1% and the much more hardware intensive Mockingjay by 0.1%.