Correcting Chinese Spelling Errors with Phonetic Pre-training
Ruiqing Zhang, Chao Pang, Chuanqiang Zhang, Shuohuan Wang, Zhongjun He, Yu Sun, Hua Ren Wu, Haifeng Wang · 2021
Chinese spelling correction (CSC) is an important yet challenging task.Existing state-ofthe-art methods either only use a pre-trained language model or incorporate phonological information as external knowledge.In this paper, we propose a novel end-to-end CSC model that integrates phonetic features into language model by leveraging the powerful pre-training and fine-tuning method.Instead of conventionally masking words with a special token in training language model, we replace words with phonetic features and their sound-alike words.We further propose an adaptive weighted objective to jointly train error detection and correction in a unified framework.Experimental results show that our model achieves significant improvements on SIGHAN datasets and outperforms the previous state-of-the-art methods.