Phantom: A High‐Performance Computational Core for Sparse CNNs
Arslan Munir, Joonho Kong, Mahmood Azhar Qureshi · 2023
Sparse convolutional neural networks (CNNs) have gained significant traction over the past few years as sparse CNNs can drastically decrease the model size and computations, if exploited befittingly, as compared to their dense counterparts. Sparse CNNs often introduce variations in the layer shapes and sizes, which can prevent dense accelerators from performing well on sparse CNN models. Recently proposed sparse accelerators such as SCNN, Eyeriss v2, and SparTen, actively exploit the two-sided or full sparsity, that is, sparsity in both weights and activations, for performance gains. These accelerators, however, either have inefficient micro-architecture (Eyeriss v2, SCNN), which limits their performance, have no support for nonunit stride convolutions (SCNN) and fully connected (FC) layers (SCNN, SparTen), or suffer massively from systematic load imbalance (SCNN, Eyeriss v2). It is also worth noting that some accelerators like SCNN perform worse under low sparsity (i.e., dense models). To circumvent these issues and support both sparse and dense models, this chapter introduces Phantom, a multithreaded, dynamic, and flexible neural computational core. Phantom uses sparse binary mask representation to actively lookahead into sparse computations, and dynamically schedule its computational threads to maximize the thread utilization and throughput. We also generate a two-dimensional (2D) mesh architecture of Phantom neural computational cores, which we refer to as Phantom-2D accelerator, and present a novel dataflow that supports all layers of a CNN, including unit and nonunit stride convolutions, and FC layers. In addition, Phantom-2D uses a two-level load balancing strategy to minimize the computational idling, thereby, further improving the hardware utilization. To show support for different types of layers, we evaluate the performance of the Phantom architecture on VGG16 and MobileNet. Simulation results show that the Phantom-2D accelerator attains a performance gain of 12×, 4.1×, 1.98×, and 2.36×, over dense architectures, SCNN, SparTen, and Eyeriss v2, respectively.