Overview of QA4MRE Main Task at CLEF 2013
Richard F. E. Sutcliffe, Anselmo Peñas, Eduard H. Hovy, Pamela Forner, Álvaro Rodrigo, Corina Forăscu, Yassine Benajiba · 2015
Abstract. This paper describes the Question Answering for Machine Reading (QA4MRE) Main Task at the 2013 Cross Language Evaluation Forum. In the main task, systems answered multiple-choice questions on documents con-cerned with four different topics. There were also two pilot tasks, Machine Reading on Biomedical Texts about Alz-heimer's disease, and Japanese Entrance Exams. This paper describes the preparation of the data sets, the definition of the background collections, the metric used for the evaluation of the systems ’ submissions, and the results. We intro-duced two novelties this year: auxiliary questions to evaluate systems level of inference, and a portion of questions where none of the options were correct. Nineteen groups participated in the task submitting a total of 77 runs in five languages. 1