A Spatio-Temporal Switchable Data Prefetcher for Convolutional Neural Networks
Jihoon Jang, Hyun Kim, Hyokeun Lee · 2023
In this paper, we propose a spatio-temporal switchable data prefetcher that can adapt to the locality characteristics of CNN models. The proposed prefetcher records the recent delta history by leveraging two tables. The first table predicts spatial address patterns by comparing the delta score with the last delta, while the second table predicts temporal address patterns by recording and reordering the delta sequence from the delta history. Consequently, the proposed prefetcher is capable of appropriately switching between these two prediction methodologies based on spatial and temporal localities. The experimental results on CNN inference workloads show that we achieved high average accuracy of 83.8% and coverage of 81.6%, and hence the proposed prefetcher improves system performance by 33.8% over a baseline with no data prefetcher and 21% over the best-performing prior spatio-temporal prefetcher.