A Weighted-Average Approach Evaluation of Multi-Class Emotion Analysis Model Using Text Mining and Machine Learning
Subhash Chandra Gupta, Noopur Goel · Cureus Journal of Computer Science. · 2025
Introduction Sentiment analysis is a text mining technique used to extract emotions from textual content into binary classes (positive or negative) or multi-class labels (more than two predefined class labels). In this paper, a multiclass sentiment analysis model is proposed to classify text into one of six predefined class labels. Methods The model utilizes the "Emotion Dataset for NLP" from Kaggle, comprising 18,000 samples. These samples underwent preprocessing steps including tokenization, stopword removal, and stemming, before being converted into text features using term frequency and inverse document frequency vectorization. The model was implemented in Python 3.9.12 using the scikit-learn library along with other required packages in Jupyter Notebook. It employed seven machine learning classifiers: k-nearest neighbor, multinomial naive Bayes, decision tree, random forest, logistic regression, support vector machine, and gradient boosting. Result and analysis Since the dataset is a multiclass imbalanced dataset, the model's performance has been evaluated using macro-averaged and weighted-average scores of precision, recall, F1 score, and class-wise area under the receiver operating characteristic curves (ROC-AUC) instead of only accuracy. The random forest classifier produced the best results, followed by the logistic regression and support vector machine classifiers. The random forest classifier achieved macro-averaged F1-score, precision, and recall of 82.26%, 83.64%, and 81.25%, respectively, based on optimal hyperparameters tuned via cross-validation. Its weighted-average scores were even better, at 85.87%, 86.06%, and 85.97%, respectively. Furthermore, the class-wise ROC-AUC observations showed that the random forest classifier was the best, scoring the highest AUC for all labels, ranging between 0.97 and 0.99.