Learning Lightweight Neural Networks via Channel-Split Recurrent Convolution
Guojun Gary Wu, Xin Zhang, Ziming Zhang, Yanhua Li, Xun Zhou, Christopher G. Brinton, Zhenming Liu · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · 2023
Lightweight neural networks refer to deep networks with small numbers of parameters, which can be deployed in resource-limited hardware such as embedded systems. To learn such lightweight networks effectively and efficiently, in this paper we propose a novel convolutional layer, namely Channel-Split Recurrent Convolution (CSR-Conv), where we split the output channels to generate data sequences with length T as the input to the recurrent layers with shared weights. As a consequence, we can construct lightweight convolutional networks by simply replacing (some) linear convolutional layers with CSR-Conv layers. We prove that under mild conditions the model size decreases with the rate of $O\left( {\frac{1}{{{T^2}}}} \right)$. Empirically we demonstrate the state-of-the-art performance using VGG-16, ResNet-50, ResNet-56, ResNet-110, DenseNet-40, MobileNet, and EfficientNet as backbone networks on CIFAR-10 and ImageNet. Codes can be found on https://github.com/tuaxon/CSR_Conv.