A GPT-PERNIE Model for Short Text Sentiment Analysis
Jingyue Huang, Pinyao He, Chenxia Li, Yao He, Yi Yang · 2024
Recently, deep learning techniques have been widely used for text sentiment classification in the domain of natural language processing. Effective text representation plays a critical role in improving the classification performance of deep learning models in this field. Due to the limited emotional contents in short texts and the susceptibility to noise during training, we propose a GPT-PERNIE model for short text sentiment analysis. This model incorporates GPT with adversarial training (P). Initially, GPT is used to enrich the expressiveness of the text. Subsequently, a single-tower model is used to evaluate text similarity, followed by vectorization of input text through the ERNIE pre-training model, which enables preliminary extraction of emotional features from the text. Later, noise interference is introduced into the output vector of the ERNIE pre-training model, leading to the generation of adversarial samples by attacking the origin. Then, these adversarial samples are used for the adversarial training of the classification model, enhancing the model's robustness against noise attacks. Experimental results indicate that the GPT-PERNIE model demonstrates superior performance and generalization capability in text classification tasks. This achievement is not only significant within the realm of short text sentiment analysis but also paves the way for novel approaches to utilizing deep learning in natural language processing tasks.