A Comparative Study on Feature Selection in Chinese Text Categorization
Dai Liu · Zhongwen xinxi xuebao · 2004
This paper is a comparative study of feature selection methods in text categorization. Four methods were evaluated, including document frequency (DF), information gain (IG), mutual information (MI) and χ 2 test (CHI). A Support Vector Machine ( SVM) and a k nearest neighbor (KNN) were selected as the evaluating classifiers. We found IG, MI and CHI had poor performance in our test, though they behave well in English text categorization. We analyzed the reasons theoretically and put forwarded the possible solutions. A furthermore experiment proved that the combined feature selection method is effective.