Bilinear Upsampling Gradients Using Convolutions In Neural Networks - Generalization To Other Schemes
Nyshadham Phani Kumar, Mittal Archie, Dabhi Levin · 2023
Image upsampling in deep learning is used to increase the resolution of the image or intermediate feature maps in many state-of-the-art neural network models. The methods of interpolation used by upsamplers include nearest neighbour interpolation, bilinear interpolation etc. In this paper we consider the computation of gradient of a bilinear image interpolator in the back propagation stage of gradient descent. Conventionally, image upsampler gradients are known to be computationally expensive and are likely to produce non-deterministic output because of Read-Modify-Write (RMW) operations on SIMD or vector processors. In this paper, we try to solve these problems involved in upsampling gradient computations by proposing an algorithm based on convolutions. We propose the algorithm taking bilinear interpolation as an example and generalize the concept to other interpolation schemes. The proposed algorithm is proved to be accelerated by matrix multiplication engines specialized at performing convolutions. Simulation results of the proposed convolution based algorithm also reflect the efficiency of the proposed algorithm in overcoming the above mentioned challenges faced by the conventional algorithms.