Cache Hierarchy in Modern Processors and Its Impact on Computing
Subrahmanya Bhat, K. R Kamath · SSRN Electronic Journal · 2017
Modern processors have multiple interacting caches on chip. Pipelined CPUs access memory from multiple points in the Pipeline - Instruction Fetch, Virtuval to Physical address translation, and Data fetch. The natural design is to use different physical caches for each of these points. Thus the pipeline naturally ends up with at least three separate caches (instruction, TLB, and data). This implementation needs machines with special architecture like separate instruction and data memories. Most modern CPUs have a single-memory von Neumann Architecture. A victim cache is a cache used to hold blocks evicted from a CPU cache upon replacement. The victim cache lies between the main cache and its refill path, and only holds blocks that were evicted from the main cache. One of the more extreme examples of cache specialization is the trace cache found in the Intel Pentium 4 microprocessors. A trace cache is a mechanism for increasing the instruction fetch bandwidth and decreasing power consumption by storing traces of instructions that have already been fetched and decoded. Multi-level caches generally operate by checking the smallest Level 1 (L1) cache first; if it hits, the processor proceeds at high speed. If the smaller cache misses, the next larger cache (L2) is checked, and so on, before external memory is checked. Some processors, all data in the L1 cache must also be somewhere in the L2 cache. These caches are called strictly inclusive. Other processors (like the AMD Athlon) have exclusive caches — data is guaranteed to be in at most one of the L1 and L2 caches, never in both. The advantage of exclusive caches is that they store more data. This advantage is larger when the exclusive L1 cache is comparable to the L2 cache, and diminishes if the L2 cache is many times larger than the L1 cache. One advantage of strictly inclusive caches is that when external devices or other processors in a multiprocessor system wish to remove a cache line from the processor, they need only have the processor check the L2 cache. In cache hierarchies which do not enforce inclusion, the L1 cache must also be checked.