Machine Translation for Subtitling: A Large-Scale Evaluation
Thierry Etchegoyhen, Lindsay Bywood, Mark Fishel, Panayota Georgakopoulou, Jie Jiang, Gerard van Loenhout, Arantza del Pozo, Mirjam Sepesy Maučec, Anja Turner, Martin Volk · 2014
This article describes a large-scale evaluation of the use of Statistical Machine Translation for professional subtitling.The work was carried out within the FP7 EU-funded project SUMAT and involved two rounds of evaluation: a quality evaluation and a measure of productivity gain/loss.We present the SMT systems built for the project and the corpora they were trained on, which combine professionally created and crowd-sourced data.Evaluation goals, methodology and results are presented for the eleven translation pairs that were evaluated by professional subtitlers.Overall, a majority of the machine translated subtitles received good quality ratings.The results were also positive in terms of productivity, with a global gain approaching 40%.We also evaluated the impact of applying quality estimation and filtering of poor MT output, which resulted in higher productivity gains for filtered files as opposed to fully machine-translated files.Finally, we present and discuss feedback from the subtitlers who participated in the evaluation, a key aspect for any eventual adoption of machine translation technology in professional subtitling.