Quantitative evaluation of machine translation using two-way MT.
Shoichi Yokoyama, Akira Kumano, Masaki Matsudaira, Yoshiko SHIROKIZAWA, Mutsumi Kawagoe, Shuji Kodama, Hideki Kashioka, Terumasa Ehara, Shinichiro Miyazawa, Yasuo Nakajima · 1999
One of the most important issues in the field of ma-chine translation is evaluation of the translated sen-tences. This paper proposes a quantitative method of evaluation for machine translation systems. The method is as follows. First, an example sentence in Japanese is machine translated into English us-ing several Japanese-English machine translation sys-tems. Second, the output English sentences are ma-chine translated into Japanese using several English-Japanese machine translation systems (different from the Japanese-English machine translation systems). Then, each output Japanese sentence is compared with the original Japanese sentence in terms of word identi-fication, correctness of the modification, syntactic de-pendency, and parataxes. An average score is calcu-lated, and this becomes the total evaluation of the ma-chine translation of the sentence. From this two-way machine translation and the cal-culation of the score, we can quantitatively evaluate the English machine translation. For the present study, we selected 100 Japanese sen-tences from the abstracts of scientific articles. Each of these sentences has an English translation which was performed by a human. Approximately half of these sentences are evaluated and the results are given. In addition, a comparison of human and machine trans-lations is also performed and the trade-off between the two methods of translation is discussed.