A probabilistic model of passage categorization
Makoto Iwayama, Takenobu Tokunaga · 1997
The difficulty in processing long documents is due to the variety of topics they contain. In this paper we study the use of probabilistic passage analysis in text categorization, assigning predefined topics to long documents. Unlike conventional text categorization that assigns topics to a whole document, passage categorization assigns topics to each passage in a document. The advantage of passage categorization is verified through experiments on the Ziff data set. We also discuss possible applications of passage categorization such as text summarization, text tiling, and passage clustering.