Perceptual evaluation of the naturalness of synthetic speech
Merrilyn L. Gow · The Journal of the Acoustical Society of America · 1990
This paper reports an investigation that assessed the naturalness of speech produced by several text-to-speech (TTS) systems. Using a 9-point subjective rating scale of naturalness, a group of listeners assigned ratings to speech produced by DECtalk 2.0, three versions of STL's system, and a software system. Each system produced 12 short paragraphs, two of which were repeated in order to assess reliability. These 60 paragraphs were used as the speech stimuli. The paragraphs were recorded in random order onto digital audio tape. Listeners made judgments based on how natural they perceived the speech to be, and assigned a rating accordingly. The reliability with which listeners assigned naturalness ratings was assessed by comparing their judgments on the paragraphs that were repeated. The listeners performed the task with satisfactory levels of intralistener agreement. The results showed that the average naturalness ratings obtained for these TTS systems fell within the predicted range, but outside the range expected for natural human speech. These findings suggest that this procedure may be a reliable tool for perceptually evaluating the naturalness of synthetic speech. The limitations and implications will be discussed.