Latency tolerant architectures
Michael J. Flynn, James Edward Bennett · 1998
Memory latency is an issue of increasing importance for microprocessor design, as processor cycle times have dramatically decreased, while the cycle times of dynamic random access memories (DRAMs) have been relatively stable. The increasing performance of microprocessors and their decreasing cycle times has created a gap between the demand of the microprocessor for memory operands and the ability of DRAM to provide these operands. This thesis studies architectures that are able to tolerate increasing memory latency. The emphasis is on modern microprocessors, which already incorporate many features, such as non-blocking caches and dynamic scheduling, to enhance their ability to tolerate memory latency. The tools used to study latency tolerance in this class of processors (which were developed for this thesis) are described, and the ability of these processors to tolerate memory latency is analyzed. Various techniques, such as stream buffers, that have been proposed for improving memory latency tolerance are also examined. A class of adaptive techniques for tolerating memory latency, prediction caches, are introduced in this thesis. Adaptive techniques allow the caching and prefetching actions of the processor to dynamically adjust to changes in system configuration and program behaviour. The most effective of these techniques combines the features of a victim cache and a stream buffer, and adaptively determines the degree of lookahead. With this technique, the memory latency penalty is reduced by 28% to 30%, over a range of cache sizes and miss penalties, for the benchmarks used in the validation.