Chinese Word Segmentation Based on an Approach of Maximum Entropy Modeling
Yan Song, Jiaqing Guo, Dongfeng Cai · 2006
In this paper, we described our Chinese word segmentation system for the 3rd SIGHAN Chinese Language Processing Bakeoff Word Segmentation Task. Our system deal with the Chinese character sequence by using the Maximum Entropy model, which is fully automatically generated from the training data by analyzing the character sequences from the training corpus. We analyze its performance on both closed and open tracks on Microsoft Research (MSRA) and University of Pennsylvania and University of Colorado (UPUC) corpus. It is shown that we can get the results just acceptable without using dictionary. The conclusion is also presented.