Adaptive Threshold Setting for Novelty Mining
Wenyin H. S. Tang, Flora S. Tsai · 2010
In the age of information, it is easy to accumulate various documents such as news articles, scientific papers, blogs, advertisements, etc. People who are interested in a certain topic may only want to track the new developments of an event or the different opinions on the topic. This motivates the study of novelty mining, or novelty detection, which aims to retrieve novel information, given a specific topic defined by a user. This chapter focuses on the novelty mining. It addresses the problem of setting an adaptive threshold by modeling the score distributions of both novel and non-novel documents. The novelty mining system combined with Gaussian-based adaptive threshold setting (GATS) algorithm, has been tested on both document-level and sentence-level data and compared to the novelty mining system using various fixed thresholds. The experimental results show that a good performance of GATS can be obtained at both levels. Controlled Vocabulary Terms Gaussian process