Outlier detection from massive short documents using domain ontology
Yongheng Wang, Shenghong Yang · 2010
With the rapid development of information technology, huge data is accumulated. A vast amount of such data appears as short documents such as paper summary or conversations in open chatting rooms. It is useful to detect outliers from those documents in intelligence analysis applications. However, traditional outlier detecting methods based on vector space model can not get acceptable accuracy because the key words appear at low frequency. On the other hand, traditional outlier detecting algorithms become very inefficient or even unavailable when processing massive data. In this paper a density-based outlier detecting method using domain ontology is presented. This algorithm uses domain ontology to calculate the semantic distance between short documents which improves the accuracy. Parallel method is also used to get better performance and scalability.