Chinese Word Segmentation: A Decade Review
Hai Yan Zhao · Zhongwen xinxi xuebao · 2007
During the last decade,especially since the First International Chinese Word Segmentation Bakeoff was held in July 2003,the study in automatic Chinese word has been greatly improved.Those improvements could be summarized as following:(1) on the computation sense Chinese words in real text have been well-defined by segmentation guidelines + lexicon + segmented corpus;(2) practical results show that performance of statistic systems outperforms that of handcrafted rule-based systems;(3) the evaluation in terms of Bakeoff data shows that the accuracy drop caused by outof-vocabulary(OOV) words is at least five times greater than that of ambiguities;(4) the better performance of OOV recognition the higher accuracy of the system in whole,and the accuracy of statistic systems with character-based tagging approach outperforms any other word-based system.