A Method of Sentence Segmentation That Check All Overlapping Ambiguity
Limin Du · Dianzi xuebao · 2004
We proposed a new method of Chinese automatic segmentation that can check all overlapping ambiguity in sentence. This algorithm is based on the principle of “Choose Longer Word”. It solves the problem that the count of segmentation way is exponentially increasing with the sentence length,and provides a method to handle overlaying ambiguity and overlapping ambiguity separately. The time complexity of this algorithm is O(N),where N is the length of sentence. Its output can greatly decrease the computing cost of post processing.