Learning Short Text Representation using Non-Negative Matrix Factorization and Word Semantic Correlations
Luepol Pipanmekaporn, Suwatchai Kamonsantiroj, Earn Suriyachay · 2019
With the emergence of online medias, short texts have been increasingly available. Because the short texts have a limited contextual information, they are sparse, noisy and ambiguous. Conventional models to text representation are hence not applicable. In this paper, we propose a representation learning model for short texts to tackle this challenge. Our proposed model is based on Non-Negative Matrix Factorization (NNMF). Due to the problem lacking word co-occurrences information, it effectively learns the representation model by incorporating the semantic relationship between words generated by unsupervised learning methods. Finally, the NNMF-based model is effectively solved by using a gradient descent-based algorithm. Experimental results on two benchmark data sets, including sentiment140 and 20newsgroups, demonstrate that our proposed approach achieved the improvement of classification accuracy to state-of-the-art topic discovery techniques.