CSGS: Adapting a Short Answer Scoring System for Multiple-choice Reading Comprehension Exercises.
Simon Ostermann, Nikolina Koleva, Alexis Palmer, Andrea Horbach · CLEF (Working Notes) · 2014
This paper describes our system submission to the CLEF Question Answering Track 2014 Entrance Exam shared task competi- tion, where the task is to correctly answer multiple choice reading com- prehension exercises. Our system is a straightforward adaptation of a model originally designed for scoring short answers given by language learners to reading comprehension questions. Our model implements a two step procedure, where both steps use the same set of metrics for evaluating similarity between pairs of input sentences/questions. In the rst step, we automatically select the sentence of the reading text that best matches the question. In the second step, the selected sentence is compared to each of the four answers, and the answer with the highest similarity score is chosen as the correct answer. Although the model has not been tuned to this specic task, we obtain scores that are compet- itive with other top-performing systems in the challenge. Additionally, we make no use of the training material but rather treat the task as one of general determination of semantic similarity between text sentences and provided answers.