Automatic Evaluation of Machine Translation Based on Rate of Accomplishment of Sub-Goals
Kiyotaka Uchimoto, Katsunori Kotani, Yujie Zhang, Hitoshi Isahara · 2007
The quality of a sentence translated by a machine translation (MT) system is dif-ficult to evaluate. We propose a method for automatically evaluating the quality of each translation. In general, when translating a given sentence, one or more conditions should be satisfied to maintain a high translation quality. In English-Japanese translation, for example, prepo-sitions and infinitives must be appropri-ately translated. We show several proce-dures that enable evaluating the quality of a translated sentence more appropriately than using conventional methods. The first procedure is constructing a test set where the conditions are assigned to each test-set sentence in the form of yes/no questions. The second procedure is devel-oping a system that determines an answer to each question. The third procedure is combining a measure based on the ques-tions and conventional measures. We also present a method for automatically gener-ating sub-goals in the form of yes/no ques-tions and estimating the rate of accom-plishment of the sub-goals. Promising re-sults are shown. 1