Reducing memory latency via non-blocking and prefetching caches
Tien-Fu Chen, Jean-Loup Baer · 1992
Non-blocking caches and prefetehing caches are two techniques for hiding memory latency by exploiting the overlap of processor computations with data accesses.A nonblocking cache allows execution to proceed concurrently with cache misses as long as dependency constraints are observed, thus exploiting post-miss operations, A prefetching cache generates prefetch requests to bring data in the cache before it is actually needed, thus allowing overlap with premiss computations.In this paper, we evaluate the effectiveness of these two hardware-based schemes.We propose a hybrid design based on the combination of these approaches.We also consider compiler-based optimization to enhance the effectiveness of non-blocking caches.Results from instruction level simulations on the SPEC benchmarks show that the hardware prefetching caches generally outperform nonblocking caches.Also, the relative effectiveness of nonblocklng caches is more adversely affected by an increase in memory latency than that of prefetching caches,, However, the performance of non-blocking caches can be improved substantially by compiler optimizations such as instruction scheduling and register renaming.The hybrid design cm be very effective in reducing the memory latency penalty for many applications.