A systematic approach to design of a text categorizer
Roger B. Bradford, John Pozniak · 2016
In this paper, we implement a systematic approach to text categorization using latent semantic indexing (LSI). A novel feature of our approach is that we iteratively refine the LSI space used for categorization. Using a verification set, we also employ LSI to determine the values of all parameters controlling the steps of the categorization process. Our approach is designed to scale to enterprise-level implementations. We test the categorizer using the standard Reuters 21578 test set. In order to accurately compare our results with other prior work, we carried out a review of over 500 previous reports of document categorization using the Reuters 21578 collection. At least within the scope of that review, the categorization performance reported here is the best yet attained for the single-label case on this standard test set.