Evaluation of speech recognizers for speech training applications
S. Anderson, Diane Kewley-Port · IEEE Transactions on Speech and Audio Processing · 1995
The use of speech recognition technology for speech training represents an important and potentially very large application of speech technology. However, speech training places unique demands on recognizer performance that have not been well-characterized. In this research, a database and testing procedures were developed to evaluate two facets of recognizer performance integral to speech training: utterance identification and speech quality assessment. Using these materials, three commercial speech recognizers that employ different types of recognition algorithms were evaluated. In general, the recognizer, based on hidden Markov models (HMM's), provided better identification scores for normal and disordered speech than the two template-based recognizers. A recognizer's identification performance on normal speech often predicted its identification performance on disordered speech. For each recognizer, analysis using phonological features revealed classes of speech sounds that are poorly discriminated. Procedures were developed to provide human ratings of the quality of disordered speech for comparison to recognizer performance. Recognizers were compared to speech-language pathologists with respect to the ability to judge speech quality. In contrast, with identification performance, the two speech recognizers based on template comparisons provided better measures of speech quality than the HMM-based recognizer.>