Neural Machine Translation Quality and Post-Editing Performance
Vilém Zouhar, Martin Popel, Ondřej Bojar, Aleš Tamchyna · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
We test the natural expectation that using MT in professional translation saves human processing time.The last such study was carried out by Sanchez-Torron and Koehn (2016) with phrase-based MT, artificially reducing the translation quality.In contrast, we focus on neural MT (NMT) of high quality, which has become the state-of-the-art approach since then and also got adopted by most translation companies.Through an experimental study involving over 30 professional translators for English→Czech translation, we examine the relationship between NMT performance and post-editing time and quality.Across all models, we found that better MT systems indeed lead to fewer changes in the sentences in this industry setting.The relation between system quality and post-editing time is however not straightforward and, contrary to the results on phrase-based MT, BLEU is definitely not a stable predictor of the time or final output quality.