Direct Application of A Language Learner Test To MT Evaluation
Florence M. Reeder · 2006
This paper shows the applicability of lan-guage testing techniques to machine trans-lation (MT) evaluation through one of a set of related experiments. One straightfor-ward experiment is to use language testing exams and scoring on MT output with little or no adaptation. This paper describes one such experiment, the first in a set. After an initial test (Vanni & Reeder, 2000), we ex-panded the experiment to include multiple raters and a more detailed analysis of the surprising results. Namely that unlike with humans, MT systems perform more poorly at both level zero and one than at level two and three. This paper presents these results as an illustration of both the applicability of language testing techniques and also the caution that needs to be applied. 1