A New Dictionary Mechanism for Chinese Word Segmentation
Jiaguang Sun · Zhongwen xinxi xuebao · 2003
Chinese word segmentation is the preparation for Chinese Information Processing. As one basic component of Chinese word segmentation systems, the dictionary mechanism influences the speed and efficiency of segmentation significantly. In this paper, we provide a new dictionary mechanism named double-character-hash-indexing (DCHI). Compared with existing typical dictionary mechanisms (i.e. binary-seek-by-word, TRIE indexing tree and binary-seek-by-characters), DCHI improves the speed and efficiency of segmentation without increasing the space and time complication and maintenance difficulty.