Text Proofreading Method for Tibetan Sound Similarity Syllables Based on Soft-Masked BERT
Rengong Zheng, Jie Zhu, Ciren Dunzhu · 2023
Error detection and error correction are two very important elements in the field of text proofreading research. In this paper, we propose an algorithm to construct a Tibetan sound similarity syllables confusion set, which generates a large amount of tagged data by adding noise to the correct Tibetan text, and use the soft-masked technique to construct a model for Tibetan text proofreading, which consists of an error detection network and an error correction network. Experimental results on two different test sets show that the method can effectively detect and correct the wrong syllables in the text.