Siamese Discourse Structure Recursive Neural Network for Semantic Representation
Erfaneh Gharavi, Rupesh Silwal, Matthew S. Gerber, Hadi Veisi · 2019
Finding a highly informative, low-dimensional representation for texts, specifically long texts, is one of the main challenges for efficient information storage and retrieval. This representation should capture the semantic and syntactic information of the text while retaining relevance for large-scale similarity search. We propose the utilization of Rhetorical Structure Theory (RST) to consider text structure in the representation. In addition, to embed document relevance in distributed representation, we use a Siamese neural network to jointly learn document representations. Our Siamese network consists of two sub-networks of recursive neural networks built over the RST tree. We examine our approach on two datasets, a subset of Reuters's corpus and BBC news dataset. Our model outperforms latent Dirichlet allocation document modeling on both datasets. Our method also outperforms latent semantic analysis document representation has been beaten by our method by 3% and 6% on the BBC and Reuters datasets, respectively. The proposed method also outperforms TF _ IDF representations by 11 % and 15% and the word embedding averaging representation by 6% and 7% in precision at k retrieved documents on Reuters and BBC datasets, respectively.