The Missing Theorem: Can sampling error be estimated?
Michael Cader Nelson · 2020
If statistics is about sorting information from error, then, arguably, half of the theory of statistical estimation is missing. Estimation theory, at its beginning, paid sampling error (also known as random error or discrepancy, as opposed to bias) little consideration, other than to state that it exists and must be overcome. Nowhere, it seems, did our statistical forebearers consider so fundamental a question as: Is it possible to estimate the amount of sampling error in a statistic from a single sample?” Once the issue did appear in the literature, it seems to have been immediately and summarily answered in the negative, without formal justification, a status that has remained unchanged and, apparently, unquestioned. In this paper, I argue that the omission of a theorem on sampling error estimation diminishes estimation theory, and that our collective blind spot in this respect represents a persistent limitation on our reason and imagination. I propose that a good start would be to restate the question as a scientific query, in unambiguous and falsifiable terms, and I provide my own effort to that end.