Combining Multi-knowledge for Chinese Word Segmentation Disambiguation
Ying Hua Qin, Suxiang Zhang, Xiaojie Wang · 2006
In the task of Chinese word segmentation, there are two main segmentation ambiguities, overlapping ambiguity and combination ambiguity. The paper analyzes properties of ambiguities and supposes multi-knowledge approach to disambiguate. Multi-knowledge refers to the knowledge from statistic of large corpus and syntactic, semantic or discourse information about ambiguous words. Class based N-gram and maximum entropy model are applied to combining multi-knowledge and disambiguation