Integrated Chinese word segmentation and part-of-speech tagging based on the divide-and-conquer strategy
Maosong Sun, Dongliang Xu, Benjamin K. Tsou · 2004
In this paper, various ways of integration of Chinese word segmentation and part-of-speech tagging, including the so-called true-integration and pseudo-integration, are tested and compared based on a test corpus consisting of 367,114 Chinese characters. A novel true-integration approach, named 'the divide-and-conquer integration', is originally proposed. Preliminary experiments show that this true integration achieves 98.72% accuracy of word segmentation, 95.65% accuracy of part-of-speech tagging, and 94.43% accuracy of word segmentation and part-of-speech tagging, outperforming all other kinds of combinations to some extent (though not very significant). The results demonstrate the potential for further improving the performance of Chinese word segmentation and part-of-speech tagging.