A Comparison of Oversampling and Undersampling Methods in Sentiment Analysis Regarding Indonesia Fuel Price Increase Using Support Vector Machine
Zaki Al Faridzi, Dita Pramesti, Riska Yanu Fa’rifah · 2023
On September 03, 2022, the Indonesian Government has decided to issue a policy to increase fuel prices. Fuel subsidies are converted into direct cash assistance (BLT) to protect the poor and vulnerable. However, the increase affects people's daily activities and eventually raises a debate in the community which is widely discussed in social media such as Twitter. There are various opinions expressed by the Indonesian people, such as approvals, criticisms, and insults. In this research, sentiment analysis was conducted on public opinion from Twitter to determine public sentiment regarding fuel prices increase policy. Sentiment analysis in this study uses a machine learning approach with Support Vector Machine (SVM) algorithm to classify each opinion into positive or negative sentiment. The split between training and testing data is carried out in a ratio of 70:30. A comparison was made between four imbalance handling methods, such as Synthetic Minority Over-Sampling Technique (SMOTE), Random Undersampling (RUS), Synthetic Minority Over-Sampling Technique-Edited Nearest Neighbors (SMOTE-ENN), and Synthetic Minority Over-Sampling Technique-Tomek Links (SMOTE-Tomek). The evaluation process uses a confusion matrix that will show accuracy, recall, precision, and F-1 score. The SMOTE method produces the highest F1-score of 57,42%, 0.02% superior to SMOTE-Tomek method and 7.03% compared to the RUS method. The best model based on this study is the SVM model using SMOTE for imbalance handling because it produces an accuracy score of 87,94%, precision score of 48,22% recall score of 70.95%, and F1-score of 57,42%.