Fast Knowledge Discovery in Social Media Data using Clustering via Ranking
Taufik Edy Sutanto, Richi Nayak · 2021 9th International Conference on Cyber and IT Service Management (CITSM) · 2021
To gain insight on a large amount of text data collected by social media outlets is challenging, even availed with the latest advancement of big data technology. In addition to the curse of dimensionality and computational complexity, users expect (near) real-time solutions. Instant analytic values are needed for making fast decisions in social media applications where data is constantly growing. This paper proposes a unique solution of generating insight on a large amount of data using an approximation of clustering via ranking approach. The solution provides a comprehensive insight without the need to scan entire data in the collection. Using a novel method, it determines the most relevant objects in the data to be included in the clustering process. Incremental clustering is done via the relevant clusters concept, instead of using the conventional window-based model. Experiments from publicly available social media datasets show that this approach is able to produce near real-time and accurate clustering results by merely using a standard computer setup instead of a costly big data machine setup.