Zero Block Caching for CNN Applications Running on a Vision DSP
Sangheon Lee, Wontae Kim, Hyuk‐Jae Lee, Kyujoong Lee · 2020
Digital Signal Processors (DSPs) nowadays are widely used for edge devices due to their power-efficiency in inference operations of Convolutional neural networks (CNNs). In most DSP systems, the ratio of zero-valued feature map data in a CNN is very high due to fixed-point quantization and activation functions like ReLu. This paper presents a new cache for a DSP system, zero block skip (ZBS) cache, to reduce off-chip memory access by caching the addresses of zero blocks in which all data are zero. In addition, a data remapping method is proposed for more zero block accesses to off-chip memory, which successfully increases the portion of zero blocks among all feature data blocks from 21.66% to 3S.7l% in Tiny YOLO network. However, it is observed that a ZBS cache using typical cache structure suffers from frequent capacity misses in some cases. To relieve the frequent capacity misses, zero block skip vector (ZBSV) for a group of blocks, each bit of which indicates whether a block is zero or not, is proposed. A ZBSV cache keeps the ZBSV entry for the currently accessed group. With DDR3 as an off-chip memory model, experiments show that four 2 KB ZBSV caches operate together effectively, with relatively lower SRAM power consumption. As a result, the proposed ZBSV cache reduces the off-chip memory access by 19.15% and 15.87% for Tiny YOLO and VDSR networks, respectively.