Short answer grading: When sorting helps and when it doesn’t
Ulrike Padó, Cornelia Kiefer · Fachbereich Informatik (University of Stuttgart) · 2015
Automatic short-answer grading promises improved student feedback at reduced teacher effort both during and after in- struction. Automated grading is, how- ever, controversial in high-stakes testing and complex systems can be difficult to set up by non-experts, especially for fre- quently changing questions. We propose a versatile, domain-independent system that assists manual grading by pre-sorting an- swers according to their similarity to a ref- erence answer. We show near state-of- the-art performance on the task of auto- matically grading the answers from CREG (Meurers et al., 2011). To evaluate the grader assistance task, we present CSSAG (Computer Science Short Answers in Ger- man), a new corpus of German computer science questions answered by natives and highly-proficient non-natives. On this cor- pus, we demonstrate the positive influence of answer sorting on the slowest-graded, most complex-to-assess questions.