Textual Entailment at EVALITA 2009
Johan Bos, Fabio Massimo Zanzotto, Marco Pennacchiotti · 2009
Abstract. The first shared task for recognising textual entailment for Italian was organised during the EVALITA 2009 campaign. The task definition followed that of its English counterpart, and consists of the task of when given a pair of texts, determining whether the first entails the second. A corpus of 800 examples pairs (400 for development/training and 400 for testing) was constructed on the basis of Wikipedia revision histories. Two groups of researchers participated, submitting eight runs in total, with a variation in performance ranging from 0.50 (conform a baseline system that always predicts entailment) to 0.71 accuracy. 1