Extracting hot spots of basic and complex topics from time stamped documents
Wei Chen, Parvathi Chundi · 2009
Identifying time periods with a burst of activity related to a topic has been an important problem in analyzing time stamped documents. In this paper, we discuss methods to compute a hot spot of a given topic from a time stamped document set. We consider basic topics that contain one or more keywords as well as complex topics that contain topics connected by logical operators and, or, not. We use the temporal scan statistic to assign a discrepancy score to each of the intervals of the time period spanning the given document set. The hot spot of the given topic is the time interval with the highest discrepancy score. We describe efficient algorithms to compute the hot spots of both basic and complex topics. Our preliminary experiments using the SIGMOD/VLDB paper titles data set and the CNN/Reuters news article titles data set collected from the TDT-Pilot Corpus show that our methods to compute the measure and the hot spot of a topic work very well in practice.