Improving multi-class text classification with Naive Bayes
Jason D. M. Rennie · 2001
The Problem: There are billions of text documents available in electronic form. More and more are becoming available every day. The Web itself contains over a billion documents. Millions of people send e-mail every day. Academic publications and journals are becoming available inelectronicform. Thesecollections and many others represent a massive amount of information that is easily accessible. However, seeking value in this huge collection requires organization. Many web sites offer a hierarchically-organized view of the Web. E-mail clients offer a systems for filtering e-mail. Numerous academic communities have a Web site that allows searching on papers and shows an organization of papers. However, organizing documents by hand or creating rules for filtering is painstaking and labor-intensive. This can be greatly aided by automated classifier systems. The accuracy of such systems determines their usefulness. We propose to use the Support Vector Machine (SVM) in conjunction with Error-Correcting Output Codes (ECOC) to improve the state-of-the-art in text classification. Motivation & Previous Work: In 1998, Joachims published results on a set of binary text classification experiments using the SVM [4]. The SVM yielded lower error than many other classification techniques. Yang followed later with experiments of her own on the same data set [5]. She used improved versions of Naive Bayes (NB) and kNN but still found that the SVM performed at least as well as all other classifiers she tried. She also found that the linear SVM performed as well as polynomial and RBF versions. Both papers used the SVM for binary text classification,