An Attention-Based Hybrid Neural Network for Document Modeling
Dengchao He, Hongjun Zhang, Wenning Hao, Rui Zhang, Huan HAO · IEICE Transactions on Information and Systems · 2017
The purpose of document modeling is to learn low-dimensional semantic representations of text accurately for Natural Language Processing tasks. In this paper, proposed is a novel attention-based hybrid neural network model, which would extract semantic features of text hierarchically. Concretely, our model adopts a bidirectional LSTM module with word-level attention to extract semantic information for each sentence in text and subsequently learns high level features via a dynamic convolution neural network module. Experimental results demonstrate that our proposed approach is effective and achieve better performance than conventional methods.