Automated Formal Register Scoring of Student Narrative Essays Written in Portuguese
Moésio Wenceslau da Silva Filho, André Câmara Alves do Nascimento, Pericles B. C. de Miranda, Luiz Rodrigues, Thiago Cordeiro, Seiji Isotani, Ig Ibert Bittencourt, Rafael Ferreira Mello · 2023
Automated essay scoring (AES) is the task of automatically assigning scores (i.e., grades) to written texts. Although AES has been widely studied in the literature (e.g., informational and argumentative essays), specific types of texts still need more attention. Narrative essays are characterized by texts describing personal experiences and stories, either real or fictional. In this work, we describe a study on scoring student essays written in Portuguese under the aspect of Formal Register, which evaluates aspects related to the use of Brazilian Portuguese formal grammar and proficiency. The dataset created in this study provides a rich corpus of narrative essays produced in the context of a motivational situation, with a diverse set of language proficiency levels annotated by two professional graders. Different machine learning algorithms were evaluated using a diverse set of handcrafted linguistic features, and their results were compared against manual scores by the two human annotators. The results of the proposed analysis demonstrated that the AES model proposed achieved an equivalent agreement to that of the two human annotators.