Managing Content With Automatic Document Classification
Rafael A. Calvo, Jae-Moon Lee, Xiaobo Li · Texas Digital Library (University of Texas) · 2004
News articles and web directories represent some of the most popular and commonly accessed content on the web.Information designers normally define categories that model these knowledge domains (i.e.news topics or web categories) and domain experts assign documents to these categories.This paper describes how machine learning and automatic document classification techniques can be used for managing large numbers of news articles, or web page descriptions, lightening the load on domain experts.In this paper we use two datasets, one with with more than 800,000 Reuters news stories and another with over 41,000 web sites, and classify them using a Naive Bayes algorithm, into predefined categories.We discuss the different parameters and design decisions that normally appear when building automatic classifiers, including, stemming, stopwords, thresholding, amount of data and approaches for improving performance using the structure in XML documents.The methodology developed would enable web based applications or workflow systems to manage information more efficiently, i.e. by assigning documents to topics automatically or assisting humans in the process of doing so.