A Neural Network Pruning Method by the Sum of Matrix Similarity Measures
Xiaoyu Dong, Mengshu Song, Binqi Li, Yuantao Song · 2022
The development of today's intelligent communication 6G is closely related to deep learning. In semantic communication, images need to be transmitted selectively in combination with image semantic information, but the large-scale parameters of neural networks exacerbate the computational complexity. Parameter pruning is one of the predominant approaches for compressing deep models, which can reduce redundant parameters, reduce communication overhead, and improve the timeliness of information transmission while maintaining the performance of neural networks. Currently, most pruning methods focus on the importance of pruned objects in the overall task. Unlike previous methods, this paper ranks the similarity of the filters in the neural network using the sum of matrix similarity measures based on cosine similarity, removing the most similar and most replaceable filters. We validate the pruning method on CIFAR10 and ImageNet ILSVRC2012 using different network architectures. The results show that the sum of matrix similarity measures has state-of-the-art performance with less model performance loss and better compression.