Examining New Event Detection
Johannes Schanda, Mark Sanderson, Paul David Clough · 2014
We examine the accuracy of first story detection on traditional news collections and on a re-purposed source of academic material. The impact on accuracy of detecting an early rather than the first story is examined, showing that accuracy increases under a broader time window, however, the increases on some collections are small. Even on collections where the increase is large, many new events are still missed and there remains an underlying challenge to detecting new events. An analysis of temporal and vocabulary profiles of topics within their source collections is conducted. Analysis of the results establish the underlying causes of the patterns seen in the experimental results with respect to the different source types and performance. The usefulness of new criteria for new event detection and success across source types is discussed.