Exploring Large Language Models in a Limited Resource Scenario
Anand Panchbhai, Smarana Pankanti · 2021
Generative Pre-trained Transformers (GPT) have gained a lot of popularity in the domain of Natural Language Processing (NPL). Lately, GPTs have been fine-tuned for tasks like sentiment analysis and text summarization. As the number of tunable parameters increases with larger language models (like GPT-3), it becomes resource-heavy to fine-tune these models on commercially available personal computer systems. In addition to that, GPT-3 is only available through an API which makes it even harder to fine-tune it for a specific task. This makes these models less accessible to the general public and researchers. Alternative ways are required to better understand the nature of these language models and employ them for challenging NLP tasks without explicit fine-tuning. This study capitalizes on the raw capabilities of GPT-2, it proposes and proves the efficacy of one such system in the task of sentiment analysis without explicit fine-tuning. It also sheds light into the nature of such generative language models and shows how explainability can be exploited to achieve good results with minimum resources. It was observed that the proposed system does a good job of capturing the sentiment of a given text. It reached an accuracy of 82% on a part of the IMDB Data set of Movie Reviews. The system performed better with natural language prompt when compared to symbol-based syntactic prompts.