Exact level and power of permutation, bootstrap, and asymptotic tests of trend
Christopher D. Corcoran, Cyrus R. Mehta · Journal of Modern Applied Statistical Methods · 2002
We develop computational tools that can evaluate the exact size and power of three tests of trend (e.g., permutation, bootstrap and asymptotic) without resorting to large-sample theory or simulations. We then use these tools to compare the operating characteristics of the three tests. It is seen that the bootstrap test is ultra-conservative relative to the other two tests and as a result suffers from a severe deterioration in power. The power of the asymptotic test is uniformly larger than that of the other two tests, but it fails to preserve the Type I error for most of the range of the baseline response probability. The permutation test, being exact, is guaranteed to preserve the Type I error throughout the range of the baseline response probability. The price paid for this guarantee is a loss of power relative to the asymptotic test. The power loss is, however, small in most situations.