Evaluating Sentiment Analysis Models: A Comparative Study of Traditional Machine Learning and Transformer-Based Approaches
Bahar Asgari, Hamid Rastegari, Vahid Nejati · Journal of Computer & Robotics · 2025
The rapid growth of digital information has elevated sentiment analysis to a critical subfield of natural language processing. A persistent challenge is identifying models that can process large-scale data while balancing accuracy, efficiency, and robustness. This study systematically compares traditional machine learning algorithms like Logistic Regression, Decision Trees, Support Vector Classification, Multilayer Perceptrons, Random Forests, and Extra Trees with transformer-based deep learning architectures, specifically BERT and DistilBERT, using the VADER lexicon as a sentiment knowledge base. Experiments conducted on the Sentiment140 dataset assess accuracy, precision, recall, F1 score, convergence time, robustness to noise, and class-wise error rates. Results show that deep learning models outperform traditional methods in accuracy and stability, with DistilBERT achieving the highest accuracy (88%) and BERT following at 85%. However, these gains come with increased computational demands, BERT requires 187 seconds to converge, compared to just 0.22 seconds for logistic regression. The findings emphasize the trade-offs between predictive performance and computational cost, offering practical guidance for selecting sentiment analysis models aligned with operational constraints.