Hierarchical Convolutional Attention Networks for Text Classification
Shang Bing Gao, Arvind Ramanathan, Georgia D. Tourassi · 2018
Recent work in machine translation has demonstrated that self-attention mechanisms can be used in place of recurrent neural networks to increase training speed without sacrificing model accuracy.We propose combining this approach with the benefits of convolutional filters and a hierarchical structure to create a document classification model that is both highly accurate and fast to train -we name our method Hierarchical Convolutional Attention Networks.We demonstrate the effectiveness of this architecture by surpassing the accuracy of the current state-of-the-art on several classification tasks while being twice as fast to train.