Communication Minimized Model-Architecture Co-design for Efficient Convolution Acceleration
Wendi Sun, Wenhao Sun, Yifan Wang, Yi Kang, Song Chen · 2024
CNN is indispensable for today’s Artificial Intelligence (AI) applications, but brings dominantly large overhead of data communication. Current works mainly focus on prior off-chip or intuitive/heuristic on-chip access optimization, but with the development of Near Memory Processing (NMP), DRAM access cost has greatly dropped and on&off-chip access optimization needs rethinking as a whole. Thus, this paper proposes a holistic on&off-chip communication-minimized model-architecture acceleration scheme for CNN. First, we derive the layer-wise off-chip communication Lower Bound (LB) based on different data reuse strategies. Second, on-chip LB is derived and overall on&off-chip communication analysis model is presented to provide a solid guidance for on-chip storage allocation, dataflow and architecture design. Finally, we design Window-Primitive (WP) dataflow and a Systolic-Cross-Line (SCL) CNN accelerator based on proposed theoretical model. SCL achieves 3.8 × pJ/MAC energy reduction at 1.4 × less on-chip storage area compared with Eyeriss and 1.3~1.8 × reduction at 3~4 × less area compared with CLB. For NMP, we reduce around 2 × access energy compared with previous systolic NMP architecture.