GCPNet: Gradient-aware channel pruning network with bilateral coupled sampling strategy
Ziyang Zhang, Chuqing Cao, Fangjun Zheng, Tao Sun, Li-Jun Zhao · Expert Systems with Applications · 2024
In the realm of deep neural network optimization, network slimming has emerged as a key technique for reducing model size without significantly compromising performance. Traditional pruning algorithms use neural architecture search (NAS) to identify networks with adjustable widths, focusing on extracting representative subnets across varied pruning ratios. However, a significant challenge lies in ensuring that the pruned network maintains high performance (accuracy) while substantially reducing the model size and computational cost. To tackle this challenge, we introduce GCPNet, a gradient-aware channel pruning network designed for efficient neural network slimming. Specifically, GCPNet incorporates a bilateral coupled sampling strategy (BCSS) to sample the smallest, largest, and several middle-sized models and perform forward and backward passes in each training iteration. The gradients of these models are then fused to update the overarching supernet. In addition, we develop a gradient-aware homogenization technique (GHT) to mitigate gradient conflicts between the supernet and the sampled models due to differing gradient directions , this accelerates the convergence of the supernet and ensures comprehensive training for GCPNet. The trained supernet serves as a reliable performance indicator, with the performance of architectures ranked by our supernet exhibiting a high correlation with true performance. Extensive experiments validate that GCPNet outperforms current advanced channel pruning approaches on the ImageNet dataset while employing comparable FLOPs and parameters. For example, under the constraints of 100 Flops and 50 Flops, our pruned MobilenetV2 achieved 68.7% and 63.5% Top-1 accuracy on the ImageNet dataset, outperforming the most advanced BCNet by 0.7% and 0.8% respectively.