Convolutional Layer Reduction from Deep Convolutional Networks
Nazmul Shahadat · 2023
Recent researchers have used pointwise convolutional neural networks (CNNs) using 1×1 convolutional filter to construct deep convolutional networks (ConvNets). These pointwise CNNs are responsible for reducing (ConvDown) and increasing (ConvUp) the number of channels whose computational costs remain high. This paper introduces a novel, cost-effective architecture that replaces ConvUp pointwise CNN in ConvNets. Here, we concatenate the output channels of ConvDown and spatial 3 × 3 convolution layers to increase the number of channels. As the pointwise ConvUp is trainable, this modification can not improve system performance. To address this limitation, we use the Squeeze-and-Excitation block to train the feature maps after concatenating the channels. Extensive experiment shows that our proposed modifications achieve almost 1% higher performance with about 50%, 52%, 33%, and 37% fewer parameters, and about 49%, 55%, 35%, and 40% fewer flops than the original, and RCN-based SqueezeNext 23 and 44 layer architectures with widening factors 1 and 2 on CIFAR benchmarks, and SVHN image classification datasets. Also, our proposed ResNets improve validation performance with more than 85% and 36% fewer parameters and 91% and 59% fewer flops than the original ResNets and RCNs, respectively. Moreover, our proposed networks take less latency than the corresponding baseline architectures.