Research of the text clustering based on LDA using in network public opinion analysis
Wang Shao-pen · Journal of Shandong University · 2014
For the problem that hidden information of the text may be ignored by the traditional text clustering algorithm based on words,a kind of text clustering algorithm based on the latent dirichlet allocation( LDA) topic model was proposed. The algorithm uses the TF-IDF algorithm and LDA topic model to calculate text similarity,through the cost function to determine the fusion coefficient of text similarity,through linear combination to get the similarity between texts and uses the F-measure value to evaluate the clustering result. In the constructing of the LDA model,the algorithm uses Gibbs sampling to estimate the parameter,and through the Bias statistical standard method to determine the optimal number of topics. Viewing from the accuracy and stability of clustering results,the simulation results show that the proposed algorithm has a better effect than the traditional text clustering algorithm.