Fine-grained emotion recognition using RoBERTa with low-rank adaptation and retrieval-augmented context contrastive definition pretraining masked language modeling and multi-dataset transfer learning
Elakkiya E, J. Kiran Deepthi, Judith Chrysolite Sirigiri, Sai Lavanya Madam, Y. Swathi Priya · International Journal of Computers and Applications · 2026
Classification of emotions expressed in textual data poses significant challenges due to the complex and intersecting nature of human emotions as well as natural class imbalances in the underlying label distributions. Earlier Bidirectional Encoder Representations from Transformers (BERT) based baselines still show limited Macro-F1 performance for emotion classification with 28 fine-grained emotion classes, namely GoEmotions. In this work, we present a parameter-efficient multi-task framework for emotion classification based on RoBERTa with Low-Rank Adaptation (LoRA). Our model incorporates three complementary modules into a unified architecture: (i) augmented retrieval of semantically similar context using Sentence-BERT and Facebook AI Similarity Search (FAISS); (ii) Contrastive Definition Pre-training (CDP); and (iii) Masked Language Modelling (MLM) as an encoder regularization technique. Moreover, we use class-weighted Binary Cross Entropy (BCE) along with soft label smoothing and class-wise threshold tuning to mitigate class imbalance during inference. In experiments on three benchmark datasets, our model achieved a better Macro-F1 score and a better Macro ROC-AUC on GoEmotions, achieving the best F1 point gain compared to the BERT+CDP baseline while using less than 1.77 million parameters. In transfer evaluation on ISEAR with 7 emotion classes, our model obtained a better Macro-F1 than other baseline models.