Minimum description length model selection criteria for generalized linear models
Mark H Hansen, Bin Yu · Lecture notes-monograph series · 2003
This paper derives several model selection criteria for generalized linear models (GLMs) following the principle of Minimum Description Length (MDL). We focus our attention on the mixture form of MDL. Normal or normal-inverse gamma distri-butions are used to construct the mixtures, depending on whether or not we choose to account for possible over-dispersion in the data. In the latter case, we apply Efron's [6] double exponential family characterization of GLMs. Standard Laplace approximations are then employed to derive computationally tractable selection rules. Each of the criteria we construct have adaptive penalties on model complexity, either explicitly or implicitly. Theoretical results for the normal linear model, and a set of simulations for logistic regression illustrate that mixture MDL can \\bridge " the selection \\extremes " AIC and BIC in the sense that it can mimic the performance of either criterion, depending on which is best for the situation at hand.