Using Factored Word Representation in Neural Network Language Models
Jan Niehues, Thanh-Le Ha, Eunah Cho, Alex Waibel · 2016
Neural network language and translation models have recently shown their great potentials in improving the performance of phrase-based machine translation.At the same time, word representations using different word factors have been translation quality and are part of many state-of-theart machine translation systems.used in many state-of-the-art machine translation systems, in order to support better translation quality.In this work, we combined these two ideas by investigating the combination of both techniques.By representing words in neural network language models using different factors, we were able to improve the models themselves as well as their impact on the overall machine translation performance.This is especially helpful for morphologically rich languages due to their large vocabulary size.Furthermore, it is easy to add additional knowledge, such as source side information, to the model.Using this model we improved the translation quality of a state-of-the-art phrasebased machine translation system by 0.7 BLEU points.We performed experiments on three language pairs for the news translation task of the WMT 2016 evaluation.