Quality estimation for translation selection.
Kashif Ur Rehman Shah, Lucia Specia · 2014
We describe experiments on quality esti-mation to select the best translation among multiple options for a given source sen-tence. We consider a realistic and chal-lenging setting where the translation sys-tems used are unknown, and no relative quality assessments are available for the training of prediction models. Our findings indicate that prediction errors are higher in this blind setting. However, these errors do not have a negative impact in performance when the predictions are used to select the best translation, compared to non-blind settings. This holds even when test condi-tions (text domains, MT systems) are dif-ferent from model building conditions. In addition, we experiment with quality pre-diction for translations produced by both translation systems and human translators. Although the latter are on average of much higher quality, we show that automatically distinguishing the two types of translation is not a trivial problem.