RCW-Pruner: Row-Column Wise Pruning Framework on Systolic Array
Xiaohui Wei, Fengyi Li, Chenyang Wang, Xinyang Zheng, Shiyu Tong, Xin Yuan, Hengshan Yue, Qi Wu · 2024
To accelerate CNN models on edge devices, the coordination of model pruning and the systolic array is an efficient paradigm. However, because of the mismatch of the irregular sparsity of the pruned model and the dedicated regular architecture of the systolic array, the pruned model is hard to cash the theoretical acceleration. To overcome this challenge, this paper proposes the RCW-Pruner which prunes the weights by the granularity of systolic array's row and column. Without any hardware modification, the pruned network can bring significant benefits of performance improvement by skipping the computing of those rows and columns. The experimental results exhibit that the RCW-Pruner achieves an average speedup of 2.62 times, 3.4 times and 2.36 times on 32 × 32, 64 × 64 and 128 × 128 systolic arrav, respectively,