Hiroshima City University at Evaluation Subtask in the NTCIR-8 Patent Translation Task
Hidetsugu Nanba, Kazuho Hirahara, Toshiyuki Takezawa · 2010
The evaluation of computer-produced texts is an important research problem for automatic text summarization and machine translation. Traditionally, computer-produced texts were evaluated automatically by n-gram overlap with human-produced texts. However, these methods cannot evaluate texts correctly, if the n-grams do not overlap between computer-produced and human-produced texts, even though the two texts convey the same meaning. We explore the use of paraphrases for the refinement of traditional automatic methods for text evaluation. In our previous work, we devised an evaluation method for text summarization using multiple paraphrase methods. Our goal in NTCIR-8 is to confirm the effectiveness of our method for machine translation. We evaluated 1200 computer-produced translations by six proposed methods and two baseline methods, and confirmed the effectiveness of our methods.