The French MEDIA/EVALDA Project: the Evaluation of the Understanding Capability of Spoken Language Dialogue Systems
Laurence Y. Devillers, Hélène Bonneau-Maynard, Sophie Rosset, Patrick Paroubek, Kevin McTait, Djamel Mostefa, Khalid Choukri, Laurent Charnay, Caroline Bousquet, Nadine Vigouroux, Frédéric Béchet, Laurent Romary, Jean-Yves Antoine, Jeanne Villaneau, Myriam Vergnes, Jérôme Goulian · 2004
This paper presents and reports on the progress of the EVALDA/MEDIA project, focusing on the recording and annotating protocol of the reference dialogue corpus.The aim of this project is to design and test an evaluation methodology to compare and diagnose the context-dependent and independent understanding capability of spoken language dialogue systems.Systems from both academic organisations (IRIT, LIA, LIMSI, LORIA, VALORIA, CLIPS) and industrial sites (FRANCE TELECOM R&D, TELIP) will be tested.ELDA is the coordinator of the Technolangue/EVALDA multi-campaign evaluation project, of which MEDIA is a subcampaign.LIMSI is the scientific coordinator of the project.MEDIA began in January 2003.