Unsupervised Segmentation for Statistical Machine Translation
Siriwan Sereewattana · 2003
An unsupervised approach is applied to segment German-English and French-English parallel corpora for statistical machine translation. The approach requires no language-nor domain-specific knowledge whatsoever. Segmentation is shown to effectively re-duce the number of unknown words and singletons in the corpora which helps improve the translation model. As a result, word error rates are lowered by 0.37 % and 2.15% in the translation of German to English and French to English respectively. The ben-efits of segmentation to statistical machine translation are more pronounced when the training data size is small. i Acknowledgements I would like to thank Miles Osborne and Chris Callison-Burch for their guidance; and Pronab Saha for his much-needed moral support. ii Declaration