Predictive and objective evaluation of speech understanding: the “challenge” evaluation campaign of the I3 speech workgroup of the French CNRS
Jean-Yves Antoine, Caroline Bousquet-Vernhettes, Jérôme Goulian, Mohamed-Zakaria Kurdi, Sophie Rosset, Nadine Vigouroux, Jeanne Villaneau · 2002
This paper presents a new paradigm of "challenge" evaluation of Spoken Language Understanding.This methodology aims at a quantitative assessment with a high diagnostic power, by opposition with standard ATIS-like frameworks.This paper details the methodology as well as the results of an evaluation campaign held by the French CNRS research agency.The benefits of this methodology are also discussed.