P1‐449: AUTOMATIC TRANSCRIPTION OF VERBAL FLUENCY RECORDINGS
David Glenn Clark, Justin Bushnell, Fredrick W. Unverzagt, Viriginia G. Wadley · Alzheimer s & Dementia · 2019
Novel scoring methods for verbal fluency word lists, such as clustering and switching, are valuable for risk stratification in dementia. However, manual transcription of these recordings is costly and state-of-the-art automatic methods do not produce sufficiently accurate transcriptions for scientific study. Approximately 150,000 verbal fluency recordings are available through the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study. We focus here on a subset of 1,202 animal fluency recordings, half from individuals who subsequently suffered from incident cognitive impairment. All recordings were automatically transcribed by Amazon Web Services (AWS). A separate program identified intervals of voice activity within each recording and AWS transcriptions were augmented to include untranscribed intervals of voicing. Two hundred AWS files were manually corrected using a custom Matlab tool that tracks onset and offset of spoken material. Differences between original and corrected transcriptions were quantified, including common transcriptions errors. Automatic correction of one problematic word (“giraffe”) was piloted. Out of 1,202 AWS transcriptions, 1,288 intervals of untranscribed voice activity were identified (1.07 missed items per file). Manual correction of 200 files resulted in 1,119 changes. The most frequently unrecognized animal words were “giraffe,” “dog,” “cat,”, “lion,” and “cow.” The most common erroneously transcribed animal words were “deer,” “doe,” “bee,” “ewe,” and “owl.” Automatic correction of “giraffe” mistranscriptions was undertaken with sensitivity 0.871 and specificity 0.977. At a threshold yielding perfect specificity, the sensitivity was 0.419, indicating that approximately 2 in 5 incorrect transcriptions of “giraffe” could be automatically corrected without adding errors to the transcriptions. Analysis of errors made by a state-of-the-art automatic transcription service will provide a means of automatic correction that will, at the very least, accelerate manual correction of the transcriptions. Future work will incorporate a language model comprising transition probabilities for target words.