Evaluating Grouped Spatial Pack Convolutions on Edge CPUs
Perry Gibson, José Cano, Jack Turner, Elliot J. Crowley, Michael O’Boyle, Amos Storkey · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · 2020
Grouped convolutions are a drop-in replacement for standard convolutional layers in neural networks. With an adjustable scaling parameter - the number of groups g - they reduce the number of parameters and multiply-accumulate operations (MACs) at the expense of representational power. However, current implementations of grouped convolutions do not perform optimally. In this paper we discuss Grouped Spatial Packed Convolutions (GSPC), a new approach to grouped convolutions, implemented in TVM’s tensor compute language. We analyse a set of networks leveraging the full range of the g parameter, and evaluate their performance in terms of inference time. We observe that GSPC achives the best performance in all settings, improving the existing implementations of grouped convolutions in TVM, PyTorch and TensorFlow Lite by 3.4x, 8x and 4x on average respectively. Code is available at https://github.com/gecLAB/tvm-GSPC/