Dynamically Mixed Group Convolution to Lighten Convolution Operation
Hang Wei, Zulin Wang, Gengxin Hua · 2021
Convolution operation is the most significant component in convolution neural networks (CNNs). However, the high computation cost limits its application on mobiles and embedded devices. To address this problem, in this paper, we develop a dynamically mixed group convolution operation (DMGConv) to lighten convolution operation. It consists of three mixed primary groups, and each primary group includes G dynamical tiny groups. The mixed primary groups not only compress the computation but also can lead to better efficiency. In addition, instead of manual setting G, we dynamically calculate the greatest common divisor of the input channels and output channels of every primary group as the number of G, which improves the self-adaptability of tiny group convolution. To demonstrate our proposed method's efficiency, we compare its parameters and computation with the popular convolutions. Moreover, we utilize our proposed approach to replace the convolution operation in the state-of-the-art CNNs. We perform them on the benchmark classification dataset CIFAR10. Experiment shows that our proposed approach is more lightweight for large input channel size and output channel size and our method can effectively reduce convolution computation and CNN's model size.