A Hybrid Part-of-Speech Tagging using Complemental Characteristics of Stochastic Information and Linguistic Knowledge
임희석, 김진동, 임해창 · Jeongbo gwahaghoe nonmunji. so'peuteuweeo mich eung'yong · 1998
The rule-based part-of-speech(POS) tagger is very accurate where rules are applied but its coverage is often limited to some core linguistic phenomena. To the contrast, while the probabilistic POS tagger is robust and extensive to various linguistic phenomena, the accuracy is lower than the rule-based POS tagger. In this paper, we propose a hybrid POS tagging method combining those complemental characteristics of the rule-based and the probabilistic POS tagger. By preferring the linguistic knowledge to stochastic information, the proposed method can keep the accuracy of the rule-based tagger. Also it has robustness and coverage of the probabilistic POS tagger by assigning a unique tag for the word whose ambiguity is not fully resolved by linguistic knowledge. A Korean POS tagger suing the proposed method was implemented and experimented on the 20K Eojeol-sized open test corpus. As a result, the experiment result showed that the tagger had the accuracy of 95.43%, which reduced the error rate of the probabilistic tagger about 32.70%. Also, it is proved that the tagger is as robust and extensive as the probabilistic POS tagger.