Exploring the potential of automated speech recognition for scoring the Korean Elicited Imitation Test
Daniel R. Isbell, Kathy MinHye Kim, Xiaobin Chen · Research Methods in Applied Linguistics · 2023
Elicited Imitation Tests (EITs) are often used as a practical tool for measuring L2 proficiency. In this study, we explore the potential of applying three commercial automated speech recognition (ASR) tools—Amazon, Google, and Naver—in combination with several transcription scoring metrics to automate Korean EIT scoring. Two hundred and four participants’ previously human-scored EITs (Isbell & Lee, 2022) were compared with automated EIT scores based on ASR transcriptions. Nearly all combinations of ASR platforms and transcription metrics correlated with original human EIT total scores above .90, with the Naver and Google transcriptions yielding the strongest correlations (r = .94-.96). Moreover, examining the item-level relationships between transcription metrics and original EIT scores yielded moderate-to-large correlations near .80 for the Google and Naver transcriptions, thereby providing additional support for the validity of automated EIT scoring. We conclude with implications for prospective users and developers of machine-scored EITs.