Meta-evaluation of machine translation evaluation methods
Lifeng Han · Dublin City University Open Access Institutional Repository (Dublin City University) · 2021
Starting from 1950s, Machine Transla- tion (MT) was challenged from different scientific solutions which included rule- based methods, example-based and sta- tistical models (SMT), to hybrid models, and very recent years the neural mod- els (NMT). While NMT has achieved a huge quality improvement in comparison to conventional methodologies, by taking advantages of huge amount of parallel corpora available from internet and the recently developed super computational power support with an acceptable cost, it struggles to achieve real human parity in many domains and most language pairs, if not all of them. Alongside the long road of MT research and development, qual- ity evaluation metrics played very impor- tant roles in MT advancement and evo- lution. In this tutorial, we overview the traditional human judgement criteria, automatic evaluation metrics, unsupervised quality estimation models, as well as the meta-evaluation of the evaluation methods.