Automatically Scoring Freshman Writing: A Preliminary Investigation
Courtney Napoles, Chris Callison-Burch · 2015
In this work, we explore applications of automatic essay scoring (AES) to a corpus of essays written by college freshmen and discuss the challenges we faced.While most AES systems evaluate highly constrained writing, we developed a system that handles open-ended, long-form writing.We present a novel corpus for this task, containing more than 3,000 essays and drafts written for a freshman writing course.We describe statistical analysis of the corpus and identify problems with automatically scoring this type of data.Finally, we demonstrate how to overcome grader bias by using a multi-task setup, and predict scores as well as human graders on a different dataset.Finally, we discuss how AES can help teachers assign more uniform grades.