Evaluating Disaster Warning Platforms on X Social Media Based on the IS Success Model: Multilabel Sentiment and Topic Modeling
Chairul Imam I'Zaaz, Asti Amalia Nur Fajrillah, Riska Yanu Fa’rifah · 2025
Effective disaster communication is critical for reducing risks in hazard-prone regions, as it ensures the timely and accurate dissemination of information that enables communities to take preventive actions. This study assesses the InfoBMKG disaster warning platform on social media$\mathbf{X}$using a comprehensive framework based on the Information Systems (IS) Success Model, combining multilabel classification, sentiment analysis, and BERTopic modeling to analyze public feedback. The analysis focused on extracting meaningful topics from both positive and negative sentiments across three key dimensions: Information Quality, Service Quality, and System Quality. The results showed that the multilabel topic classification model achieved an accuracy of 62.57 % and a macro-average F1-score of 0.81, reflecting balanced performance across all topics. Sentiment analysis showed robust detection of negative sentiments with accuracy rates of 84.79 %,$\mathbf{8 8. 7 3 \%}$, and 89.03% for Information, Service, and System Quality respectively, but positive sentiment detection remains limited, especially in Service and System Quality. The topic modeling on negative sentiment tweets revealed critical thematic issues such as incomplete and delayed information, errors in early warning management, and system instability. These findings inform targeted recommendations to improve platform responsiveness, reliability, and user trust. The study demonstrates the utility of combining multilabel sentiment classification and topic modeling to assess disaster communication effectiveness on social media and offers insights applicable to similar early warning systems