TRANSLATION ERROR SPOTTING FROM A USER'S POINT OF VIEW
Thomas Meyer · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · 2012
The evaluation of errors made by Machine Translation (MT) systems still needs hu-man effort despite the fact that there are automated MT evaluation tools, such as the BLEU metric. Moreover, assuming that there would be tools that support hu-mans in this translation quality checking task, for example by automatically mark-ing some errors found in the MT sys-tem output, there is no guarantee that this actually helps to achieve a more correct or faster human evaluation. The paper presents a user study which found statisti-cally significant interaction effects for the task of finding MT errors under the con-ditions of non-annotated and automatically pre-annotated errors, in terms of the time needed to complete the task and the num-ber of correctly found errors.