Emotion Retrieval Using Generative Models: An Exploratory Study
Muhammad Luqman Jamil, Sebastião Pais, Andrés Caro Lindo, João Cordeiro · 2024
Recent developments in Large Language Models(LLMs) have amplified their usability across various tasks. Significant improvements have been made in their ability to understand text intelligently and provide correct results across many possible scenarios. Following this trend, we fine-tune commonly used open-source LLMs with smaller parameter sizes to test them for classifying specific text related to emotions. The objective is to explore them instead of using conventional classifications and their related advantages for the text classification task. We use zero-shot to test base models and fine-tune them using prompt tuning. QLoRA technique for reducing the computational requirements is used on emotion datasets to improve and compare their results with the zero-shot model. Results show that the pre-trained zero-shot models score lower for classifying specific text, with the phi-3 scoring the best accuracy of 53%. However, fine-tuning improves the results for almost all models, showing a perspective for future applications across various scenarios. Using lightweight LLMs with optimization techniques such as QLoRA can be a viable alternative to avoid the challenges of LLMs with larger parameter sizes, requiring high computational resources and extended training times.