Recognizing Textual Entailment With a Modified BLEU Algorithm
Terrence Szymanski · 2005
The BLEU algorithm was proposed as a baseline technique for the task of recognizing textual entailment (RTE) by (Perez & Alfonseca, 2005) in the first PASCAL RTE challenge. However, because the BLEU algorithm was designed as a metric for measuring the accuracy of automatically generated translations, certain features of the algorithm are not appropriate for RTE. Specifically, BLEU penalizes brevity both explicitly and implicitly in its scoring algorithm; since entailment hypothesis are very often short phrases, this behavior is not desirable in RTE. Therefore, this paper proposes a variation on the BLEU scoring algorithm that does not penalize brevity and can consistently outperform both the unmodified BLEU algorithm and a dumb baseline.