A Named Entity Recognition Method For Chinese Winter Sports News Based On RoBERTa-WWM
Pingshan Liu, Yuan Cao · 2022 3rd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE) · 2022
In the task of Named Entity Recognition (NER) of winter sports text news, for the problem that common Pre-trained Language Model (PLM)trained with a common corpus have limited semantic representation capabilities for sentence and words in the winter sports domain, and the problem that Chinese BERT PLM, trained based on a single character masking approach, cannot understand Chinese semantic features well, this paper proposes a Named Entity Recognition method for Chinese winter sports news based on RoBERTa-WWM model. Firstly, we add many proper nouns and terminologies in the domain of winter sports to lexicon of the LTP word segmentation tool used for pre-training of RoBERTa-WWM model, so that the LTP can segment the text more accurately, thus the RoBERTa-WWM model can achieve more accurate Whole Word Masking (WWM) pre-training. After that, we pre-train the RoBERTa-WWM model on abundant unlabeled winter sports text news data with the premise of more accurate WWM pre-training. In the end, the pre-trained RoBERTa-WWM model is used as the word embedding layer to connect with the downstream task model BILSTM-CRF for fine-tuning the NER task by using a hierarchical learning approach, to obtain better accuracy of entity recognition results. The experimental results show that the NER model trained by the method proposed in this paper achieves an F1 value of 93.56% in the NER test of actual winter sports textual news, which is 0.77% higher than the F1 value of the common RoBERTa-WWM-BiLSTM-CRF model, 1.81% higher than the F1 value of the common BERT-BiLSTM-CRF model, and 4.88% higher than the F1 value of the BiLSTM-CRF benchmark model.