Discriminative Word Alignment Over Multiple Word Segmentations
Ning Xi, Xinyu Dai, Shujian Huang, Jiajun Chen · Chinese Journal of Electronics · 2014
Conventional bilingual word alignment is conducted on sentence pairs with single word segmentation for languages such as Chinese, viz. Single-segmentation-based word alignment (SSWA). However, SSWA may run the risk of losing optimal word segmentation granularities or causing data sparseness in word alignment. This paper proposes Multiple-segmentation-based word alignment (MSWA). In MSWA, diverse and complementary knowledge in multiple word segmentations can be employed to lower the above risks in word alignment. Given$k$word segmentations of a Chinese sentence, a skeleton segmentation is firstly constructed. The alignment between the skele-ton segmentation and the parallel English sentence is log-linearly modeled, where various features defined over multiple word segmentations are incorporated. The Viterbi alignment, the alignment with the highest score, is mapped back to$k$word alignments based on$k$segmentations respectively. Experimentally, MSWA outperformed SSWA on all$k$segmentations in both alignment quality and translation performance.