Pseudorandom Sampling has a Real Effect on Test Size
Jeffrey H. Dorfman · The American Statistician · 1996
A dangerous trend toward nonrandom sampling mechanisms has been observed in several fields of the applied sciences. These sampling designs are attempts to avoid tail observations, resulting in smaller estimated standard errors and inflated claims of statistical significance. Often, the researchers go to some lengths to make the sampling mechanisms appear random; yet, they clearly are not. The effect of these pseudorandom sampling designs on the power and size of the hypothesis tests is investigated and shown to be substantial.