Detection of Real-Time Consumer Group Changes and Behavior Analysis through Automated Crawling and Sentiment Analysis
Hyeonji Ko, Baekbin Ko, Tae‐Wan Kim · Journal of Multimedia Information System · 2025
This paper introduces a novel methodology for real-time detection of consumer cluster changes with enhanced precision and speed. By leveraging automated data crawling and sentiment analysis, it surpasses traditional dichotomous clustering based on gender and age, allowing for more nuanced cluster identification that captures diverse consumer characteristics. Conventional batch analysis methods, which retrospectively analyze data after marketing campaigns, struggle to capture rapidly evolving consumer trends. To address this limitation, our study emphasizes the importance of identifying consumer clusters through real-time data collection and analysis, facilitating swift strategic responses. We integrate automated crawling with machine learning techniques to analyze review data following the release of the film "Top Gun: Maverick." Our results show that emerging clusters during both the initial release and subsequent consumer influx can be classified in real-time. Additionally, we quantitatively and qualitatively assessed the characteristics, entry pathways, and emotional responses of each cluster, demonstrating the effectiveness of real-time classification. Notably, compared to batch analysis, our approach accelerates cluster change detection by at least 83% and achieves finer segmentation that incorporates contextual subtleties beyond frequency analysis. In conclusion, this study confirms that real-time analysis using streaming data effectively addresses omnivore consumption patterns, which are challenging to explain with traditional age- and gender-based criteria. It enhances the speed and significance of trend detection, enabling businesses to gain a competitive edge by promptly adapting to rapidly changing consumer needs and contributing to sustainable growth models.