Evaluation Dataset (DT-Grade) and Word Weighting Approach towards Constructed Short Answers Assessment in Tutorial Dialogue Context
Rajendra Banjade, Nabin Maharjan, Nobal Bikram Niraula, Dipesh Gautam, Borhan Samei, Vasile Rus · 2016
Evaluating student answers often requires contextual information, such as previous utterances in conversational tutoring systems.For example, students use coreferences and write elliptical responses, i.e. incomplete but can be interpreted in context.The DT-Grade corpus which we present in this paper consists of short constructed answers extracted from tutorial dialogues between students and an Intelligent Tutoring System and annotated for their correctness in the given context and whether the contextual information was useful.The dataset contains 900 answers (of which about 25% required contextual information to properly interpret them).We also present a baseline system developed to predict the correctness label (such as correct, correct but incomplete) in which weights for the words are assigned based on context.