COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut
Baoxin Zhao, Haoyi Xiong, Jiang Bian, Zhishan Guo, Chengzhong Xu, Dejing Dou · IEEE Transactions on Multimedia · 2020
In this paper, we extend the classic MaxOut strategy, originally designed for Multiple Layer Preceptors (MLPs), intoCOnvolutionalMaxOut (COMO) — a new strategy making deep convolutional neural networks wider with parameter efficiency. Compared to the existing solutions, such as ResNeXt for ResNet or Inception for VGG-alikes, COMO works well on both linear architectures and the ones with skipped connections and residual blocks. More specifically, COMO adopts a novelsplit-transform-mergeparadigm that extends the layers withspatial resolution reductioninto multiple parallel splits. For the layer with COMO, each split passes the input feature maps through a4D convolution operatorwith independentbatch normalization operatorsfor transformation, then merge into the aggregated output of the original sizes throughmax-pooling. Such a strategy is expected to tackle the potential classification accuracy degradation due to the spatial resolution reduction, by incorporating the multiple splits and max-pooling-based feature selection. Our experiment using a wide range of deep architectures shows that COMO can significantly improve the classification accuracy of ResNet/VGG-alike networks based on a large number of benchmark datasets. COMO further outperforms the existing solutions, e.g., Inceptions, ResNeXts, SE-ResNet, and Xception, that make networks wider, and it dominates in the comparison of accuracy versus parameter sizes.