Automatic Reference-Based Evaluation of Pronoun Translation Misses the Point
Liane Guillou, Christian Hardmeier · 2018
We compare the performance of the APT and AutoPRF metrics for pronoun translation against a manually annotated dataset comprising human judgements as to the correctness of translations of the PROTEST test suite.Although there is some correlation with the human judgements, a range of issues limit the performance of the automated metrics.Instead, we recommend the use of semiautomatic metrics and test suites in place of fully automatic metrics.