Named Entity Recognition Based on BERT-MBiGRU-CRF and Multi-head Self-attention Mechanism
Xiaoni Yang, Yuelei Xiao · 2022 4th International Conference on Natural Language Processing (ICNLP) · 2022
When processing Chinese named entity recognition, the traditional algorithm model have been having the ambiguity of expressive words and the singleness of the word vector, and the training consequence of the algorithm model was not well. To solve this problem, a BERT-MBiGRU-CRE model was proposed to increase the accuracy of Named Entity Recognition (NER). This model used a Multilayer Bidirectional Gated Recurrent Unit (MBiGRU) network to replace the Bidirectional Long Short-Term Memory (BiLSTM) network in the BERT-BiLSTM-CRE model, which can extract global context semantic features more effectively. Next, a BERT-MBiGRU-MS-CRF model was suggested based on the model. It added a layer of Multi-Head Selfttention (MS) mechanism behind the MBiGRU layer, which can efficiently extract multiple semantic features and overcome the deficiencies of MBiGRU to get local features. Finally, the experimental results on the MSRA dataset showed that the training time of the two models was significantly lower than that of the BERT-BiLSTM-CRE model, but the accuracy, recall and F1 value of them were significantly higher than those of the BERT-BiLSTM-CRE model, reaching 97.26%, 97.53% and 97.39% respectively.