Emotion Analysis in Texts Using Decoder Only Transformer Model
Ambati Shivani, Dokala Hrushikesh, Gali Jaya Chandra, Tipirneni Lahari · 2025
This paper proposes a new way to recognize emotion. Only the Transformer Decoder will do text-based research. Sentiment analysis has previously been worked on to quite an extent, either focused on encoder-decoder archi-tecture or vice versa, using both orientations of models. The current research is trying to demonstrate how a unidirectional, purely decoder-based network would effectively do the work for sentiment analysis. This lightweight network architecture will learn the subtleties of emotions embodied in texts. with accuracy comparable to that of more complex models, but at lower computational cost. Our proposed model is based on these very same fundamental concepts of the attention mechanism and self-supervised learning, particularly human-learning-specific adaptation for emotion detection. We have a new training objective based on traditional linguistic methods through fine-tuning the pre-trained decoder only transformer models. Modeling with emotion-specific token prediction so that simultaneously, the model picks up the representation of the emotional context, via an implicit strategy for the linguistic structure. We have done experiments with several benchmark datasets testing our method, was involving social media posts, literary texts, and conversational. Our experiments show that the decoder-only model is able to learn competitively on many emotion classification tasks and outperform even the most traditional LSTM-based, match or exceed the accuracy of larger, bidirectional Transformer architectures. The following research represents completely new effective and efficient analysis of emotions in text from different directions include sentiment analysis and follow-up customer feedback processing, to head tracking in mental health. The directions for transfer learning will be used in the future for domain adaptation, and incorporating more multimodal data to Emotion Detection.