OnSeS: A Novel Online Short Text Summarization Based on BM25 and Neural Network
Jianwei Niu, Qingjuan Zhao, Lei Wang, Huan Chen, Mohammed Atiquzzaman, Fei Peng · 2016
The last decade has witnessed a dramatic growth of social networks, such as Twitter, Sina Microblog, etc. Messages/short texts on these platforms are generally of limited length, causing difficulties for machines to understand. Moreover, it is rarely possible for users to read and understand all the content due to the large quantity. So it is imperative to cluster and extract the viewpoints of these short texts. To solve this, the representation of a word is enriched with additional features from external, but it is demanding in terms of computational and time resources. In this paper, we proposed OnSeS, a novel short text summarization method which makes full use of word2vec to represent a word and utilizes neural network model to generate each word of the summary. OnSeS consists of three phrases: 1) clustering short texts using the K-means algorithm; 2) ranking content of each cluster by building a graph-based ranking model using BM25; 3) generating main point of each cluster with the help of neural machine translation model on the top ranked sentence. The experimental results reveal that our proposed fully data-driven approach outperforms state-of-the-art method.