Applying class triggers in Chinese POS tagging based on maximum entropy model
Yan Jun Zhao, Xiao-Long Wang, Bingquan Liu, Yi Guan · 2005
A method of applying class triggers in Chinese POS tagging based on maximum entropy model is proposed in this paper. First of all, feature template of "word-> word/tag" is used to extract the triggers from corpus and the triggers that we extracted are added into the maximum entropy model as a new kind of feature. Then, the average mutual information is applied to make feature selection and the semantic lexicon is used to build class triggers to overcome sparseness problem. Meanwhile, a solution based on experience to deal with over-fitting problem in model training is presented. Finally, the performance of the system is evaluated on a manually annotated POS tag corpus. The experiment demonstrates that the method can provide increase of accuracy of POS tagging from 94% to 96%, compared our new model with HMM model that is smoothed by absolute smoothing.