Should You Fine-Tune BERT for Automated Essay Scoring?
Elijah Mayfield, Alan W. Black · 2020
Most natural language processing research now recommends large Transformer-based models with fine-tuning for supervised classification tasks; older strategies like bag-ofwords features and linear models have fallen out of favor.Here we investigate whether, in automated essay scoring (AES) research, deep neural models are an appropriate technological choice.We find that fine-tuning BERT produces similar performance to classical models at significant additional cost.We argue that while state-of-the-art strategies do match existing best results, they come with opportunity costs in computational resources.We conclude with a review of promising areas for research on student essays where the unique characteristics of Transformers may provide benefits over classical methods to justify the costs.