Pre-Trained Language Model Transfer on Chinese Named Entity Recognition
Huan Zhao, Mingquan Xu, Jie Cao · 2019
The challenges of natural language processing (NLP) lie in the polysemy and insufficiency of human-labeled training data. Bidirectional Encoder Representations from Transformers (BERT) facilitates pre-training deep bidirectional representations on large-scale unannotated text on the web and has created state-of-the-art performance on various NLP tasks after simple fine-tuning. The representation and application of semantic knowledge are important steps in the entire process of NLP tasks. In this work, for the encoder of our model, we encode an input sequence into contextual representations using pre-trained language model and design a new model that combines neural network with BERT. Experimental results show that our method achieves new state-of-the-art.