Performance-Based Measurement of Speech Quality with an Audio Proof-Reading Task
Mark A. Huckvale, Gaston Hilkhuysen, Deizom Frasi · 2010
The extent to which speech communication systems deliver speech signals which are ‘fit for purpose’ can be assessed through a range of evaluation techniques including intelligibility and quality measures. But when signal quality is high, performance-based scales such as intelligibility reach ceiling values, and evaluation turns instead to opinion-based rating scales. In this paper we propose that performance-based testing can be applied even when intelligibility is high by measuring the cognitive effort listeners employ to understand speech. We describe previous research that shows how behavioural measures of recall accuracy and reaction time can vary with signal quality in a listening task. We demonstrate a new listening task based on the correction of transcripts of fluent spontaneous dialogues, and evaluate it for six different signal qualities, including telephone, added noise and noise-reduced conditions. We show that signal quality has a significant effect both in terms of transcript error detection accuracy and in terms of processing speed. We also show that in this test noise-reduction did not have any beneficial effect, despite previous observations that noise reduction improves opinions of signal quality