Semantic tagging using HMM with semantic induction
LI Xiang-yang, Pla Uni · Journal of PLA University of Science and Technology · 2005
It is difficult for hidden Markov models to get precise parameter estimation when applied to semantic tagging as the number of semantic tags is large and training data is insufficient. Different from classic method for solving data sparsity problem, a method which takes advantage of the hierarchical structure of Synonymy Thesaurus was presented to improve the quality of HMM parameter estimation by semantic induction. Restrictive selection policy was used to reverse the decline of model discriminability caused by the method. Tests indicate that the method is feasible to semantic tagging and tuning according to training data with size can greatly improve semantic tagging.