Measures of Performance Sensitivity
Gordon W. Wilcox · The Journal of the Acoustical Society of America · 1967
Several writers have suggested fitting straight lines on double-probability paper to the empirically obtained receiver-operating-characteristic (ROC) curves for human observers. A simple theorem is proved which shows that in acoustical tests in which a fixed signal is presented, no observer of any kind can have a straight-line ROC with other than unity slope. This negative result suggests that some commonly used measures of performance sensitivity are inadequate. A new interpretation of Green's theorem relating the area under the single-interval ROC to the probability of correct response in two-interval forced-choice tasks is given. The interpretation permits a new and direct estimate of the area under the ROC curve in single-interval rating tasks. The estimator, denoted PA, (i) is unbiased, (ii) depends only upon obtained ROC points (i.e., no curve must be fitted to the data), and (iii) generalizes to certain multiple signal-alternative situations. It is further shown that PA is a special case of a more general measure of association between the performance of any two observers (human or electronic) in the same task. As such, it may be used in testing alternative models of sensory processing [This research was conducted at the Sensory Intelligence Laboratory, University of Michigan, and supported by a contract with NASA.]