Chinese News Classification Based on ERNIE and Attention Fusion Features
Siju Zhu, Chengwan He · 2023
In order to improve the classification performance of Chinese news and correlate the features extracted by multiple network models at the same time, we propose a deep network model based on the pre-trained model ERNIE and attention fusion features. Use ERNIE as the word embedding layer to obtain dynamic word vectors, use the self-attention mechanism SAT and DPCNN network to obtain the long-distance dependence of the text, and use the bidirectional gating unit BIGRU and the soft attention mechanism AT to obtain contextual timing features. Combine the label CLS processed by ERNIE, the hidden layer state of BIGRU at the last moment, and the above two features into a feature sequence, and use the attention mechanism to assign the weight ratio of each feature. Finally, the weighted and summed feature vectors are classified and output using the fully connected layer. Trained on the public Chinese dataset THUCNews, the experimental results show that this model can effectively improve the accuracy of text classification compared with other comparison models.