Incorporating Topic Priors into Distributed Word Representations
Xin Zhang, Bingquan Liu, Baoxun Wang, Xiaolong Wang, Deyuan Zhang · International Journal of Database Theory and Application · 2016
Representing words as continuous vectors enables the quantification of semantic relationships of words by vector operations, thereby has attracted much attention recently.This paper proposes an approach to combine continuous word representation and topic modeling, by encoding words based on their topic distributions in the hierarchical softmax, so as to introduce the prior semantic relevance information into the neural networks.The word vectors generated by our model are evaluated with respect to word relevance and the document relevance.Experimental results show that our approach is promising for further improving the quality of word vectors.