Reformulating the direct convolution for high-performance deep learning inference on ARM processors
Sergio Barrachina, Adrián Castelló, Manuel F. Dolz, Tze Meng Low, Hèctor Martínez, Enrique S. Quintana–Ort́ı, Upasana Sridhar, Andrés E. Tomás · Journal of Systems Architecture · 2022
We present two high-performance implementations of the convolution operator via the direct algorithm that outperform the so-called lowering approach based on the im2col transform plus the gemm kernel on an ARMv8-based processor. One of our methods presents the additional advantage of zero-memory overhead while the other employs an additional yet rather moderate workspace, substantially smaller than that required by the im2col+gemm solution. In contrast with a previous implementation of a similar zero-memory overhead direct convolution, this work exhibits the key advantage of preserving the conventional NHWC data layout for the input/output activations of the convolution layers.