Model Compression for Image Classification Based on Low-Rank Sparse Quantization
Zongcheng Ben, Xin Long, Xiangrong Zeng, Jie Liu · 2019
The increase in computation resources for complicated neural networks in recent years severely hinders their applications in limited-power devices. As a result, compressing and accelerating deep networks have become necessary. Considering the different features of weight quantization and regularization method, we propose a low-rank sparse quantization method to quantize weights and regularize the structures of convolutional networks at the same time. Specially, our method can (1) obtain low-bit quantized networks to reduce memory and computation cost and (2) learn a compact structure from complex neural networks for subsequent channel pruning. We evaluate the proposed method on common datasets. Results show that this method can obtain pretty compact neural network which could achieve large channel sparsity while dramatically compressing networks with slight accuracy loss.