Evaluating automatic speech recognition for L2 pronunciation feedback: a focus on Google Translate
Paul St. John, Walcir Cardoso, Carol Johnson · 2022
This study examines the L2 pronunciation feedback provided by the Automatic Speech Recognition (ASR) functionality in Google Translate (GT). We focus on three Quebec Francophone (QF) errors in English: th-substitution, h-deletion, and h-epenthesis. Four hundred and fifty male and female QF recordings of sentences with correctly and incorrectly pronounced final items (e.g. I don’t know who to thank versus tank) were played into GT. Errors were equally divided between mispronunciations leading to real word (thank → tank) and nonword output (thief → tief). As anticipated, we found greater transcription accuracy for correct pronunciations and, among incorrect pronunciations, for real words versus nonwords. Overall, our findings suggest ASR can be highly effective for pronunciation feedback. We also examined transcriptions for gender bias, since ASR systems are often trained on corpora with more male voices, but our concerns proved unfounded: surprisingly, higher transcription accuracy was found for female recordings.