The Character-based CRF Segmenter of MSRA&NEU for the 4th Bakeoff
Zhenxing Wang, Changning Huang, Jingbo Zhu · International Joint Conference on Natural Language Processing · 2008
This paper describes the Chinese Word Segmenter for the fourth International Chinese Language Processing Bakeoff. Base on Conditional Random Field (CRF) model, a basic segmenter is designed as a problem of character-based tagging. To further improve the performance of our segmenter, we employ a word-based approach to increase the in-vocabulary (IV) word recall and a post-processing to increase the out-of-vocabulary (OOV) word recall. We participate in the word segmentation closed test on all five corpora and our system achieved four second best and one the fifth in all the five corpora.