Combining Statistical Model and Dictionary for Domain Adaption of Chinese Word Segmentation
Ting Liu · Zhongwen xinxi xuebao · 2012
Generally,statistical methods for Chinese Word Segmentation don't have good domain adaptability owing to the specific training corpus.In practice,domain dictionaries are more easily achieved than humanly annotated segmentation corpus,and it contains plenty of domain information.We propose an approach which integrates dictionary information into statistical models(i.e.,CRF model in this paper) to realize domain adaption for Chinese Word Segmentation.Experimental results show that our approach have good domain adaption.When the test corpus is identical to the domain of training corpus,the F-measure value increases 2%;when test corpus is in a different domain of the training corpus,the F-measure value increases 6%.