Comparative Analysis of EmoRoBERTa and Distilled Zero-shot Student Model for Emotion Detection in Texts
Mary Joy P. Canon, Lany L. Maceda, Christian Y. Sy, Nancy M. Flores · 2024
Despite the rapid progress in the field of emotion detection using NLP models, there is a noticeable gap in the literature concerning direct and comprehensive comparison between models of existing benchmark approach in transfer learning and zero-shot learning. This paper presents a comparative analysis of emotion detection models by investigating the potential of a distilled zero-shot student model compared with EmoRoBERTa pretrained model in text-based emotion classification task. Our data points are specifically focused on academic-related text responses, which are manually labeled using GoEmotions taxonomy. A key methodology decision was retaining the stopwords in the training corpus, as their removal degrades the model’s performance, indicating their significance in understanding the context of emotional expressions. Finetuning results revealed that distilled zero-shot student model outperformed the EmoRoBERTa model across all performance metric scores, achieving an impressive 84.21% accuracy and a parallel 84.23% F1 score. Efficiency in training was also observed to be a significant advantage of the distilled zero-shot model. It demonstrated more stable training progress with less pronounced overfitting compared to the EmoRoBERTa model. Moreover, the distilled zero-shot student model demonstrates excellent predictive ability to distinguish between various emotions, particularly those with distinct expressions. It is proved to be effective in accurately identifying most emotions, particularly excelling at desire, gratitude, and neutral classes. However, the model encountered challenges in classifying emotions like optimism and realization, suggesting future improvements on how to better represent them in the corpus.