Attention Based Encoding and Decoding for Convolutional Neural Networks
Sudhanshu Upadhyay, Gaurav Kumar Yadav, Rohan Trigune, Nikahat Mulla · 2019 International Conference on Smart Systems and Inventive Technology (ICSSIT) · 2019
Deep learning architectures have been used widely of late for the purpose of image classification tasks. This paper presents an architecture using convolutional neural networks and multi-heads attention. The architecture of our model comprises of two encoders and one decoder. The second encoder and the decoder use multi heads attention mechanism. The proposed model takes individual channels in the image through 3 different encoders and learns from them. The model presented here performs comparatively better even with less number of convolution blocks and its performance significantly improves by adding more layers.