Term weighting evaluation in bipartite partitioning for text clustering
Chao Qu, Li Yong, Jun Zhu, Peican Huang, Ruifen Yuan, Tianming Hu · 2008
Abstract. To alleviate the problem of high dimensions in text clus-tering, an alternative to conventional methods is bipartite partitioning, where terms and documents are modeled as vertices on two sides respec-tively. Term weighting schemes, which assign weights to the edges linking terms and documents, are vital for the final clustering performance. In this paper, we conducted an comprehensive evaluation of six variants of tf/idf factor as term weighting schemes in bipartite partitioning. With various external validation measures, we found tfidf most effective in our experiments. Besides, our experimental results also indicated that df factor generally leads to better performance than tf factor at moderate partitioning size. 1