Symbol-and-Statistics Decoding Model and Its Application in Chinese POS Tagging
Zhiyong Luo · Zhongwen xinxi xuebao · 2010
A statistical language model named Symbol-and-Statistics Decoding(SSD) language model is presented in this article.The 2-gram SSD model is applied to the Chinese POS tagging task with a quite good result.The precision is as high as 97.08% in the closed test and 95.67% in the open test is,which are both significantly higher than the HMM at 95.56% and 94.70%,respectively.Although the performance of SSD model is not as good as the conditional models such as Maximum Entropy Model and CRF model,the training time of SSD is much less than the conditional models,which makes SSD model more applicable to certain tasks in natural language processing.