Scoring Essays Written in Persian Using a Transformer-Based Model
Tahereh Firoozi, Mark J. Gierl · 2024
The automated scoring of student essays is now recognized as a significant development in both the research and practice of educational assessment. The majority of the published studies on automated essay scoring (AES) focus on outcomes in English. Studies on languages other than English are, by comparison, practically nonexistent. The purpose of this chapter is to describe and evaluate the first AES system for scoring essays in the Persian language using multilingual Bidirectional Encoder Representation for Transformers (mBERT). mBERT is a transformer-based encoder model for language representation that uses an attention mechanism to learn the contextual relations between words and sentences in a text. mBERT is pre-trained on 104 languages, including Persian. mBERT was used to evaluate 2,000 holistically scored essays written in Persian by non-native language learners in Iran using a five-point scale that ranged from Elementary to Advanced. The performance of the mBERT transformer model was examined against a baseline model that only included a Word2Vec word embedding layer. The mBERT model performed with high classification consistency compared to the baseline model. These results demonstrate that the mBERT model can be used with a high degree of precision to predict the Persian essay scores produced by human raters. The methods described in this study can be easily adapted and readily used to score essays written in the remaining 103 languages in mBERT, thereby supporting the application and widespread use of multilingual AES.