Compressing Deep Convolutional Networks Using K-means Based on Weights Distribution
Wang Lei, Huawei Chen, Yixuan Wu · 2017
For the application of deep neural networks on devices with limited hardware resources, it is necessary to reduce the computational complexity and storage requirement. Compression is an effective way to achieve this goal. One of the available method is to quantize the weights to enforce weight sharing, which can greatly reduce the parameters of each layer. This paper presents an improved k-means clustering algorithm to compress CNN (convolutional neural networks).By taking weights distribution into consideration when choosing clustering centers to quantize weights, this algorithm automatically chooses and revises centers to compress network. Compared with traditional quantification method, this algorithm can maintain accuracy and increase the compression speed at the same time. Experiments on AlexNet show that using k-means based on weights distribution to quantize the weights can improve compression speed by 5% to 10% and improve accuracy by 6% compared to traditional algorithm. This algorithm provides a better way for the application of convolutional neural networks on mobile devices.