A Novel Term Weighting Scheme for Automated Text Categorization
Hongzhi Xu, Chunping Li · Seventh International Conference on Intelligent Systems Design and Applications (ISDA 2007) · 2007
Term weighting is an important task for text classification. Inverse document frequency (IDF) is one of the most popular methods for this task; however, in some situations, such as supervised learning for text categorization, it doesn 't weight terms properly, because it neglects the category information and assumes that a term that occurs in smaller set of documents should get a higher weight. There have been several term weighting schemes that consider the category information. In this paper, we present a new term weighting scheme that considers more information provided by the term distribution among different categories. The experiments show that our method is more effective than three other popular schemes.