Real Time Micro-blog Summarization Based on Hadoop/HBase
Sanghoon Lee, Sunny Shakya, Rajshekhar Sunderraman, Saeid O. Belkasim · 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) · 2013
Micro-blog is a medium of communication that allows users to communicate with each other via short contents. Using the micro-blog as a way of spreading information more broadly has gained much interest as a new social medium where the contents can be delivered in real-time. However, the users should take the trouble to read manually through the posts for understanding a specific topic since the posts have been sorted by time, not relevancy. In this paper, we present a real time application that summarizes the posts by relevancy, considering the time that the posts are written. We set Hadoop environment with HBase since the application needs to be scalable and also, fault-tolerant. Summaries that the application produces are evaluated by ROUGE metric which is a well-known summary evaluation method. The evaluation result indicates that the summaries produced by the application show better results comparing to summaries generated by a traditional summarization method.