A Joint Statistical Model for Word Spacing and Spelling Error Correction Simultaneously
Hyungjong Noh, Jeong-Won Cha, GaryGeun-Bae Lee · Jeongbo gwahaghoe nonmunji. so'peuteuweeo mich eung'yong · 2007
In this paper, we present a preprocessor which corrects word spacing errors and spelling correction errors simultaneously. The proposed expands noisy-channel model so that it corrects both errors in colloquial style sentences effectively, while preprocessing algorithms have limitations because they correct each error separately. Using Eojeol transition pattern dictionary and statistical data such as n-gram and Jaso transition probabilities, it minimizes the usage of dictionaries and produces the corrected candidates effectively. In experiments we did not get satisfactory results at current stage, we noticed that the proposed methodology has the utility by analyzing the errors. So we expect that the preprocessor will function as an effective error corrector for general colloquial style sentence by doing more improvements.