Statistical Distances as Loss Functions in Assessing Model Adequacy
Bruce G. Lindsay · 2004
This chapter takes on the problem of model adequacy and makes an argument for reformulating the way model-based statistical inference is carried out. In the new formulation, it does not treat the model as “truth.” It is instead an approximation to truth. Rather than testing for model fit, an integral part of the proposed statistical analysis is to assess the degree to which the model provides adequate answers to the statistical questions being posed. One method for doing so is to create a single overall measure of inadequacy that evaluates the degree of departure between the model and truth. The chapter argues that there are two components of errors in any statistical analysis. One component is due to model misspecification; that is, the working model is different from the true data-generating process. The chapter compares confidence intervals on model misspecification error with external knowledge of the scientific relevance of prediction variability to address the issue of scientific significance. The chapter also analyzes several familiar measures of statistical distances in terms of their possible use as inadequacy measures.