Aligning machine-generated feedback on speaking performance with human judgments
Lin Gu, Pam Mollaun, Jeremy Lee, Yuan Wang, Saerhim Oh, Fred S. Tsutagawa, Jorge Luis Beltrán Zúñiga · 2024
Feedback is a crucial component of successful language learning. Recent advances in technology have increased the prospects for automated systems to provide feedback on spoken language produced by second language (L2) learners. This study examined the similarities and differences between teacher feedback and machine feedback with the goal of guiding the development of automated systems for evaluating speaking performance. Forty-eight participating teachers were asked to provide feedback on L2 spoken performance elicited by TOEFL iBT speaking tasks. Results indicated the following opportunities for future development of automated feedback systems: (a) including a wider range of linguistic foci, especially those that tend to draw attention from teachers, (b) incorporating targeted feedback, and (c) prioritizing feedback depending on a learner’s proficiency level. These results will have implications for developing automated systems capable of providing feedback that is aligned with human intelligence and beneficial for learning.