A novel hybrid clustering algorithm for microblog topic detection
Xiao Geng, Yanmei Zhang, Yuhang Jiao, Yinan Mei · AIP conference proceedings · 2017
Microblog has the characteristics of large scale, various topics and too much topic-unrelated texts included. So we propose a three -layer hybrid clustering algorithm to replace the original ones used in the topic detection models which can hardly handle microblog. We apply the K-means algorithm in clustering the microblog texts by their topics in the first layer. And in the second layer, we use the agglomerative nesting algorithm to merge the small clusters consisting of texts of the same topic. The first two layers also remove most noise, reducing their further impact on the K-means in the third layer, which reassigns the texts assigned to the wrong cluster. Experiments show our algorithm outperforms some related traditional algorithms on the clustering of real dataset and functions perfectly in the topic detection.