Information conveyed by voices
Bruce R. Gerratt, Jody E. Kreiman · The Journal of the Acoustical Society of America · 2007
Two approaches for dealing with interrater variability in voice quality judgments have recently been proposed. The first [Shrivastav et al., J. Speech Lang. Hear. Res. 48, 323–335 (2005)] advocates reducing rating errors after data collection by averaging many ratings of each voice by each rater and then standardizing the resulting means. The second [Gerratt and Kreiman, J. Acoust. Soc. Am. 110, 2560–2566 (2001)] controls rating error at the source by applying a method-of-adjustment task, which controls variability due to attention, variable internal standards, etc. Using this second method, we accounted for 87% of the variance in listeners perception of noise level for 40 voices. This low level of residual variance (errors) raises the following question: Does averaging ratings reduce the information within listeners responses along with measurement errors? To address this question, 20 listeners provided 10 breathiness ratings for 40 voices. The likelihood of exact listener agreement was calculated and compared to data from the method-of-adjustment task. We also compared amounts of variance accounted for by differences between voices in quality (a metric of information captured). We hypothesized that the averaging and standardizing approach would produce both lower interrater agreement and lower amounts of variance accounted for than the method of adjustment technique. [Work supported by the NIH/NIDCD.]