Overview of the NTCIR-11 Recognizing Inference in TExt and Validation (RITE-VAL) Task.
Suguru Matsuyoshi, Yusuke Miyao, Tomohide Shibata, Chuan‐Jie Lin, Cheng-Wei Shih, Yotaro Watanabe, Teruko Mitamura · 2014
This paper describes an overview of Recognizing Inference in TExt and Validation (RITE-VAL) task in NTCIR-11. We evaluated systems that automatically recognize semantic re-lations between sentences such as entailment, contradiction and independence in Japanese (JA), English (EN), Simpli-ed Chinese (CS) and Traditional Chinese (CT). RITE-VAL task has the following two subtasks: Fact Validation subtask (FV) and System Validation subtask (SV). SV consists of bi-nary classication subtask (SVBC) and multi-classication subtask (SVMC). We had 23 active participating teams, and received 170 formal runs (59 Japanese runs, 9 English runs, 53 Simplied Chinese runs and 49 Traditional Chinese runs). This paper also describes how the datasets for RITE-VAL had been developed, how the systems were evaluated, and reports RITE-VAL formal run results.