Automated Essay Evaluation at Scale
Corey Palermo, Arianto Wibowo · 2024
This chapter presents a progressive human–machine hybrid scoring approach designed to address limitations of automated scoring applications in large-scale, summative assessment programs. The chapter begins with context about modern summative assessment scoring programs, including current stakeholder expectations and constraints. Next, the Project Essay Grade (PEG) automated scoring engine is described, along with a summary of evidence for its use. Engine training and validation requirements in support of the progressive hybrid scoring approach are then proposed, followed by a discussion of the training process, including fairness considerations. Next, the deployment of PEG by Measurement Incorporated in a multi-state summative assessment context is described to illustrate the approach. The chapter concludes with implications for the field and future directions.