Phrase Model Training for Statistical Machine Translation with Word Lattices of Preprocessing Alternatives
Joern Wuebker, Hermann Ney · 2012
In statistical machine translation, word lattices are used to represent the ambiguities in the preprocessing of the source sentence, such as word segmentation for Chinese or morphological analysis for German. Several approaches have been proposed to define the probability of different paths through the lattice with external tools like word segmenters, or by applying indicator features. We introduce a novel lattice design, which explicitly distinguishes between different preprocessing alternatives for the source sentence. It allows us to make use of specific features for each preprocessing type and to lexicalize the choice of lattice path directly in the phrase translation model. We argue that forced alignment training can be used to learn lattice path and phrase translation model simultaneously. On the newscommentary portion of the German→English WMT 2011 task we can show moderate improvements of up to 0.6 % BLEU over a stateof-the-art baseline system. 1