Evidence for Increased Discriminability in Judging the Acceptability of Machine Translations: The Case for Magnitude Estimation
James D. Walrath · 2009
Abstract : An earlier experiment, in which magnitude estimation (ME) was used as the method for judging the acceptability of machine translations, was replicated with one exception. The current study used a four-point Likert scale as the measurement methodology. In the earlier work, using ME, judges easily discriminated between two sets of machine translations, one containing 25% more correctly translated names than the other. In this study, using a Likert scale, judges were not able to make the same discrimination. These results support the theory that ME may be a superior measurement methodology for assessing the acceptability of machine translations.