A new term‐weighting scheme for text classification using the odds of positive and negative class probabilities
Youngjoong Ko · Journal of the Association for Information Science and Technology · 2015
Text classification (TC) is a core technique for text mining and information retrieval. It has been applied to many applications in many different research and industrial areas. Term‐weighting schemes assign an appropriate weight to each term to obtain a high TC performance. Although term weighting is one of the important modules for TC and TC has different peculiarities from those in information retrieval, many term‐weighting schemes used in information retrieval, such as term frequency–inverse document frequency (tf–idf), have been used in TC in the same manner. The peculiarity of TC that differs most from information retrieval is the existence of class information. This article proposes a new term‐weighting scheme that uses class information using positive and negative class distributions. As a result, the proposed scheme, log tf–TRR, consistently performs better than do other schemes using class information as well as traditional schemes such as tf–idf.