EnCache: Improving Cache Energy Efficiency U sing a Software-Controlled Profiling Cache
Sparsh Mittal, Zhao Zhang · 2012
Leakage energy has been estimated to increase by five-times with each CMOS technology generation and this is expected to become a major source of energy dissipation, especially in large last-level caches (LLCs). We present EnCache, a novel cache leakage energy saving scheme that uses a new micro-architecture component called profiling cache. With this hardware support, system software can accurately predict the cache usage and energy efficiency of a running program for 32 possible cache configurations and vary the cache configuration for saving cache leakage power. Unlike existing schemes, En- Cache can directly optimize for the overall memory subsystem power efficiency instead of only cache power efficiency. EnCache employs dynamic performance regulation and hence does not re- quire offline profiling or per-application tuning. The experiments performed over our workload confirm the superiority of EnCache scheme over conventional energy-saving schemes. On a baseline configuration of 2MB cache size, the average improvement of memory subsystem energy and energy-delay-product (EDP) by using EnCache are 31.7% and 28.8%, respectively. in the processor core and DRAM memories. This increase may even offset the power savings in cache. It is very difficult, if not impractical, for non-adaptive hardware-based schemes of cache energy saving to also take into account the components other than the cache. Furthermore, their control mechanism depends on arbitrary parameters (e.g miss-bound, decay in- terval) that must be tuned per application. The presence of large intra-program variations and the differences between the profiled runs and actual programs make the approach of per- application tuning highly ineffective and difficult-to-scale. We present EnCache (Energy saving approach for Caches), a new software-based approach on top of lightweight hardware support. The key component of the hardware support is a sim- ple profiling cache. It is tag-only cache and uses set-sampling to predict cache miss rates of multiple cache configurations of much larger sizes in an online manner. It works non-intrusively and due to its decoupled and parallel operation and small-size, its latency is easily hidden. Profiling cache is not a part of the cache hierarchy and it does not lie at the critical access path of the cache. Further details are explained in Section III-A. The previous approaches such as (4) utilize sampling only to profile different associativities of the current size of the cache, while EnCache provisions a separate cache structure which can profile different associativities at different cache sizes. Thus, EnCache considerably expands upon the potential of sampling. This is a significant difference, which enables the prediction of energy efficiency of multiple cache sizes and thus, guide reconfiguration. Profiling cache has an energy overhead of less than 0.5% of L2 cache energy. Our simulation results show that a profiling cache is highly accurate, with an average error of 0.26MPKI (miss-per-Kilo-instruction) in predicting the cache miss rates for 100 benchmark/configuration combinations. Our profiling cache is designed to also estimate the impact of cache miss-rates on performance, in terms of memory stall cycles. Using these estimates and other performance counters, an OS component periodically predicts the memory-subsystem (which includes LLC and main memory) energy for multiple cache configurations. Then, the cache configuration with the minimum estimated energy is chosen for the next interval and, if necessary, the cache is reconfigured to that configuration. EnCache addresses the aforementioned shortcomings of the hardware-based approaches. It optimizes for memory- subsystem energy rather than merely cache energy. It opti- mizes directly for energy, unlike previous approaches which