Bootstrap method versus analytical approach for estimating uncertainty of measures in ROC analysis on large datasets
Jin Chu Wu, Alvin F Martin, Gregory A. Sanders, Raghu N. Kacker · 2018
The nonparametric two-sample bootstrap is employed to estimate uncertainties of measures in ROC analysis on large datasets with/without data dependency due to multiple use of the same subjects in many disciplines, based on our studies of bootstrap variability.On the other hand, it would seem that the analytical approach might be used for the same purpose.The differences between these two methods are noteworthy.The bootstrap method can intrinsically take account of how genuine scores and impostor scores are distributed, deal with data dependency, and solve the issue of the covariance occurred while the statistic is a weighted sum of two probabilities derived from two sets of data, respectively, in ROC analysis.The analytical approach cannot.The analytical approach generally underestimates the uncertainties of measures as opposed to the bootstrap method.The comparison was carried out using the real data obtained from the speaker recognition evaluations and the biometric evaluations, as well as the simulated data with normal distributions and nonparametric distributions, respectively.