Sampling variability of the success ratio in predictor‐based selection
Wilfried De Corte · British Journal of Mathematical and Statistical Psychology · 2002
In applied settings, the expected benefits of a predictor-based selection are often expressed in terms of the expected success ratio of the selection. Although Taylor and Russell proposed a formula to estimate its value, simple simulations show that their formula is often inadequate. Also, little is known at present about the sampling variability of the success ratio and, hence, about the accuracy with which the success ratio of any particular selection can be predicted. The above deficiencies are addressed for the three most popular single-stage selection scenarios: the restricted (or fixed) quota, the threshold (mastery or fixed cut-off) and the mixed quota/threshold decision. For each scenario it is shown how the sampling distribution of the success rate statistic can be derived. The problems related to the numerical evaluation of the distribution are also discussed, and it is shown by means of example applications that the sampling variability of the success ratio may be quite substantial in many real-world predictor-based selections.