A Distributed Approach For Chinese Micro-blog Hot Topic Detection
Xiang Zhang, Ruitao Lin, Lili Dong, Ru Wang · Advances in intelligent systems research/Advances in Intelligent Systems Research · 2014
In consideration of the features of micro-blogging content such as short text, sparse feature words and the huge scale, a method to detect micro-blogging hot topic was proposed in this paper based on MapReduce programming model.This method first employs the hidden topic analysis to solve the problem of short micro-blogging content and sparse feature words.Then the CURE algorithm is used to alleviate the problem that the Kmeans algorithm is sensitive to the initial points.Finally, the hot topic clustering results are obtained through the parallel Kmeans clustering algorithm based on the MapReduce programming model.The experimental results show that proposed method can effectively improve the micro-blogging hot topic detection efficiency.