Chinese Word Segmentation by Mining Maximized Substrings
Mo Shen, Daisuke Kawahara, Sadao Kurohashi · International Joint Conference on Natural Language Processing · 2013
A major problem in the field of Chinese word segmentation is the identification of out-ofvocabulary words. We propose a simple yet effective approach for extracting maximized substrings, which provide good estimations of unknown word boundaries. We also develop a new semi-supervised segmentation technique that incorporates retrieved substrings using discriminative learning. The effectiveness of this novel approach is demonstrated through experiments using both in-domain and out-ofdomain data.