Chinese word segmentation dictionary using two-level index
Sheng Chai · Computer Engineering and Applications Journal · 2009
As the basis of Chinese information processing,Chinese word segmentation plays a very important role in the fields of searching engine,automatic and so on.Chinese word dictionary is the basis of mechanic segmentation algorithm,it tells the algo-rithm what is a Chinese word.Because the algorithm needs the content of dictionary in order to match the string in the text,the storage structure of the dictionary will decide the method of the algorithm and its performance.Through making research into the existed theory and refinement,this paper adds multi-level index for the dictionary,and based on this formulates a new mechanism of Chinese word segmentation dictionary—dictionary based on two-level index.On the basis of this new theory,this paper also improves the positive matching method,reduces the complexity of matching process,moreover,elevates the speed of the segmentation.