Evaluation of machine translation output in context of inflectional languages
Daša Munková, Michal Munk, Jozef Kapusta, Jaroslav Reichel · 2016
Objective of the paper is to evaluate metrics of automatic evaluation of machine translation output using manual metrics — fluency and adequacy. We tried to answer the question to which extent the manual evaluation correlates with the automatic evaluation of MT output from/to Slovak to/from English. We focused on metrics based on the similarity and statistical principles (WER, PER, CDER and BLEU-n). We found out, that the manual evaluation, namely fluency and adequacy metrics correlates with automatic metrics of MT evaluation for less spoken language and low resource language such as Slovak. The contribution also consists of system proposal for both, manual (based on POS tagging) and automatic (based on reference) evaluation of MT output.