Efficient Application of Tensor Core Units for Convolving Images
Stefan Groth, Jürgen Teich, Frank Hannig · 2021
Tensor Core Units (TCUs) are a domain-specific architecture capable of executing small Matrix Multiply-Accumulates (MMAs) in a single clock cycle, showing significant performance improvements over other optimized implementations. When Convolutional Neural Networks (CNNs) are accelerated using TCUs, the layout of the input image is transformed to allow large amounts of filters to be applied to an image using a single large matrix-matrix multiplication. However, there are applications in other domains that only require a small number of filters. To accommodate such applications, we first show the inappropriateness of this standard technique of transforming the data layout. Subsequently, we propose an approach that uses TCUs to convolve one filter with an image. Further, we introduce several optimizations of this method. Finally, we evaluate the performance of our approach and its optimizations by comparing it to code generated using a state-of-the-art image processing language.