Multi-task sentiment classification model based on DistilBert and multi-scale CNN
Guanghao Xiong, Ke Yan · 2021 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · 2021
Sentiment classification is an important topic in the field of natural language processing (NLP). The mainstream methods have also changed from the initial emotional dictionary to the later machine learning, and now most of them focus on deep learning. In sentiment classification tasks, comments in different fields have many similar expressions and relevant characteristics, which can effectively solve the problem of insufficient training data for current deep learning methods. In order to improve the classification effect of the model, we proposed a sentiment analysis model based on multi-task learning that Distillation Bidirectional Encoder Representations from Transformers combined with multi-scale convolution (MTL-DISTILBERT-MSCNN). It models comments in multiple fields together, and uses the advantages of the multi-task learning framework, the pre-training model DistilBert and the convolutional neural network (CNN) model to obtain the correlation between comments in multiple fields, the global features and local features of the text. We conduct experiments on 14 product review data sets and 2 movie review data sets. Using accuracy, F1-score and training speed as the evaluation criteria of the proposed model. By comparing single-task learning and other multi-task learning methods, the MTL-DISTILBERT-MSCNN model can be improved by 2%-5%, and the training efficiency is very high. In addition, we also adjusted the convolutional neural network of the model to further optimize the training effect of the model.