Two Knives Cut Better Than One: Chinese Word Segmentation with Dual Decomposition
Mengqiu Wang, Rob Voigt, Christopher D. Manning · 2014
There are two dominant approaches to Chinese word segmentation: word-based and character-based models, each with re-spective strengths. Prior work has shown that gains in segmentation performance can be achieved from combining these two types of models; however, past efforts have not provided a practical technique to allow mainstream adoption. We pro-pose a method that effectively combines the strength of both segmentation schemes using an efficient dual-decomposition al-gorithm for joint inference. Our method is simple and easy to implement. Ex-periments on SIGHAN 2003 and 2005 evaluation datasets show that our method achieves the best reported results to date on 6 out of 7 datasets. 1