Chinese Word Segmentation Based on Dictionary and Statistics
Wanli Zuo · Journal of Chinese Computer Systems · 2006
Proposed a method based on dictionary integrated with statistics.The method uses the segmentation method based on dictionary in the first step and then employs segmentation based on statistics to resolve ambiguity and unregistered words left over in the first step.An improved data structure of dictionary is employed to accelerate dictionary looking up speed in the first step,and during the second step,statistics integrated with rules is adopted in order to improve accuracy of crossing ambiguity division and to deal with the unregistered words.The integrity of Dsfenci System which is realized on the method proposed by this paper is 99.52%,the accuracy is 98.52%.