A Transformer-Based Approach for Enhancing Automated Essay Scoring
Ravi Kishore Reddy Chavva, Sharath Reddy Muthyam, Meghana Sadwini Seelam, Narsing Nalliboina · 2024
Automated essay scoring (AES) is a crucial technology for modern educational systems, providing a consistent and efficient method for evaluating written assignments. Traditional manual grading is often time-consuming, subjective, and prone to inconsistency, which can impact the quality of education and feedback. To address these challenges, we developed an AES system using the RoBERTa model, utilizing its advanced natural language processing capabilities. Our approach involved training and fine-tuning the RobERTa model on the Automated Student Assessment Prize (ASAP) dataset, which consists of diverse student essays graded by human raters. The model was trained to understand and replicate the scoring patterns and criteria used in the ASAP dataset, enabling it to predict accurate scores for new essays. The performance of our AES system was evaluated using the quadratic weighted kappa (QWK) metric, a standard measure for assessing agreement between predicted scores and human grades. Our system achieved a QWK score of 0.815, indicating a high level of accuracy and reliability in essay scoring. By ensuring consistent and objective grading, our AES system can support educators in delivering better learning outcomes. This result demonstrates the potential of the RoBERTa-based AES system to enhance the educational process through timely and fair assessments of student writing.