Feature selection in SVM text categorization
Hirotoshi Taira, Masahiko Haruno · 1999
This paper investigates the effect of prior feature selection in Support Vector Machine (SVM) text categorization. The input space was gradually increased by using mutual information (MI) fil-tering and part-of-speech (POS) filtering, which determine the portion of words that are appro-priate for learning from the information-theoretic and the linguistic perspectives, respectively. We tested the two filtering methods on SVMs as well as a decision tree algorithm C4.5. The SVMs ’ re-sults common to both filtering are that 1) the opti-mal number of features differed completely across categories, and 2) the average performance for all categories was best when all of the words were used. In addition, a comparison of the two filter-ing methods clarified that POS filtering on SVMs consistently outperformed MI filtering, which in-dicates that SVMs cannot find irrelevant parts of speech. These results suggest a simple strategy for the SVM text categorization: use a full number of words found through a rough filtering technique like part-of-speech tagging.