Comparison of Two Methods for the Subjective Evaluation of Speech
Gerhard Martin Sessler, James E. West · The Journal of the Acoustical Society of America · 1972
Speech samples were generated by bandpass filtering and reverberating a sentence recorded with a broad-band system in an anechoic chamber. The speech samples were rated by a large number of subjects utilizing two testing methods: (1) The standard forced-choice paired comparison test where all possible combinations of the samples were presented to the subjects. In this test each subject had to select from each pair the sample he found preferable. This test, although lengthy, gives very reliable results since each sample is compared with all other samples. (2) A rating method where the samples were presented to the subjects just once in random order. In this much shorter test, each subject has to assign a number expressing his preference to each sample. In both cases, the subjective responses were evaluated with a multidimensional scaling method on a digital computer. The results show that the two testing methods yield related representations of the samples. Thus, the second method, which requires considerably less testing, is often adequate for the subjective evaluation of speech samples.