Machine Learning Algorithm Comparison with Class Imbalance Handling for Sentiment Analysis Satusehat Application Reviews on Play Store
Yulia Ery Kurniawati, Muhammad Firdaus, Tota Pirdo Kasih · 2025
This study aims to compare the machine learning algorithm to handling class imbalance learning for sentiment analysis SATUSEHAT Application Reviews on Play Store. SATUSEHAT was previously known as PeduliLindungi. We compare three algorithms-Multinomial Naïve Bayes (MNB), Support Vector Machine (SVM), and Random Forest (RFT)—using 10-fold cross-validation for more comprehensive performance evaluation, with and without applying SMOTE to handle class imbalance. The findings indicate that SVM achieved the best performance after applying SMOTE, reaching an accuracy of${9 4. 1 4 \%}$and an F1-Score of${9 4. 1 6 \%}$, followed by RFT with 91.29 % accuracy. The sentiment analysis revealed a prevalence of negative feedback, signifying user dissatisfaction driven by issues related to user registration, login, and email verification. New dissatisfaction drivers that were not prominent in previous studies on PeduliLindungi. Word cloud analysis further illustrated that unfavourable reviews frequently mentioned “vaksin,” “login,” and “email,” whereas favourable reviews included terms like “bagus” and “sehat.”