HBP: Hierarchically Balanced Pruning and Accelerator Co-Design for Efficient DNN Inference
Ao Ren, Yuhao Wang, Tao Zhang, Jiaxing Shi, Duo Liu, Xianzhang Chen, Yujuan Tan, Yuan Xie · 2023
Weight pruning is studied to accelerate DNN inference by reducing the parameters and computations. Irregular pruning achieves high sparsity while incurring low computation parallelism and imbalanced workloads. The coarse-grained structured pruning sacrifices sparsity for higher parallelism. To strike a better balance, we propose Hierarchically Balanced Pruning by applying fine-grained but structured adjustments based on irregular pruning. Besides, it partitions the weight matrix into hierarchical blocks and constrains the sparsity of the blocks for balanced workloads. Furthermore, an accelerator is proposed to unleash the power of the pruning method. Experimental results show our method achieves 1.1×-6 higher sparsity than prior studies, and the accelerator achieves 1.2×-13× speedup and 3.3× energy efficiency improvement than its counterparts.