A Comparative Study of Intent Classification Performance in Truncated Consumer Communication using GPT-Neo and GPT-2
Chanda Hirway, Enda Fallon, Paul Connolly, Kieran Flanagan, Deepak Yadav · 2023
This study presents a comparative analysis of intent classification performance using two widely used language models, GPT-Neo and GPT-2, in the context of truncated consumer communications. Generative Pre-trained Transformer (GPT) is a machine learning technique that has revolutionized the field of natural language processing (NLP). GPT uses a transformer-based neural network architecture that is pre-trained on large volumes of data to generate highly accurate and versatile NLP models capable of performing various tasks, such as language translation, question-answering, and text summarization. GPT's ability to generate natural language responses that closely resemble those of humans has greatly enhanced the potential of language processing in the future. The data used in this study was provided by Circana. This data is an essential resource as it includes real world consumer purchases, promotional and e-commerce activity from multiple retail markets, as well as related consumer messaging. The model is trained, evaluated and analyzed on their respective accuracies, precision, recall, and F1 scores. The results indicate that GPT- Neo with BCELoss function outperformed GPT-2 in terms of accuracy and F1 score, with a significant improvement in the reduction of false negative values. These findings demonstrate the potential of GPT-Neo for improving intent classification in consumer communication applications with limited text input.