Overview of NTCIR-9 RITE : Recognizing Inference in TExt
Hideki Shima, Hiroshi Kanayama, Cheng‐Wei Lee, Chuan‐Jie Lin, Teruko Mitamura, Yusuke Miyao, Shuming Shi, Koichi Takeda · 2011
This paper introduces an overview of the RITE (Recognizing Inference in TExt) task in NTCIR-9. We evaluate systems that automatically recognize entailment, paraphrase, and contradiction between two texts written in Japanese, Simplified Chinese, or Traditional Chinese. The task consists of four subtasks: Binary classification of entailment (BC); Multi-class classification including paraphrase and contradiction (MC); and two extrinsic application-oriented datasets: Entrance Exam and RITE4QA. This paper also describes how we built the test collection, evaluation metrics, and evaluation results of the submitted runs.