A Tibetan Error Correction Method Based on Syllable-Based Levenshtein Distance
Xiaodong Liu, Jie Yu, Qingbo Wu, Zibo Yi, Long Peng, Jun Ma, Bingshan Chang, Jianfeng Li, Min Liu, Nyima Tashi, Duojie Rengzeng · 2021 2nd International Conference on Electronics, Communications and Information Technology (CECIT) · 2021
Accurately and effectively find and correct errors in Tibetan is of great significance to Tibet's informatization construction. However, due to the linguistic specificity of Tibetan, previous methods proposed for general Language were unable to accurately correct Tibetan errors. Consequently, in this paper, we present TEC_SLD, a Tibetan error correction method which is able to accurately identify incorrect syllables in Tibetan and provide candidate words to correct the errors. Exploiting syllable-based Levenshtein distances method, TEC_SLD calculate the edit distance between two strings with syllable as the smallest unit. Experimental results demonstrate that, compared with the traditional Levenshtein distance method, the average hit rate of TEC_SLD proposed in this paper is 70.9% higher. And the first hit rate of TEC_SLD achieves up to 18x increment.