Research on multi-feature fusion text classification model based on self-attention mechanism
Xiaoxia Luo, XuanHao Wang · Journal of Physics Conference Series · 2020
Abstract For complex long texts, critical semantic information will be weakened and non-critical features will be forgotten. This paper proposes a multi-feature fusion text classification model based on autoattention mechanism. The preprocessed text data is represented by character-level vectors through the Bert model. First of all, self attention mechanism (Self-Attention) is used to learn the dependence of text words to capture the internal structural information of the text, Secondly, according to Deep superposition convolutional neural network (DSCNN) and Bi-directional Gated Recurrent Unit (BiGRU) based on soft attention mechanism (Soft-Attention), the semantic features of text data are extracted separately, and two different feature extraction results are combined. Finally, the Softmax layer is used to classify the deep-extracted features, and the accuracy of the classification model is improved by adding a uniform distribution item to the cross-entropy loss function. The text is verified in the medical consultation data set, and the results show that the F1 value of the text classification of the model is as high as 88.91%, Text semantic understanding is better than mainstream models.