Research on Combinational Ambiguity in Chinese Word Segmentation
Ying Hua Qin · Zhongwen xinxi xuebao · 2007
One of challenges in Chinese Word Segmentation is the combinational ambiguity problem with two main obstacles: the detection of combinational ambiguities and ambiguity resolution.This paper investigate the structures of combinational ambiguities and proposes a new approach for automatically detecting this type of ambiguities.The experimental result reveals the approach is effective in the tagged corpus of 1998-01 People Daily with about 1 million words,we have detected more than 400 combinational ambiguities,far more than that detected by common approaches.Then the resolutions of 60 combinational ambiguities are carried out using the maximum entropy model.The effect of six kinds of features,as well as their combination,on the performance of disambiguation is further studies.The average accuracy of disambiguation reaches 88.05%.