Predictive Assessment of Bayesian Hierarchical Models
Daniel Sabanés-Bové · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · 2009
Bayesian hierarchical models are increasingly used in many applications.In parallel, the desire to check the predictive capabilities of these models grows.However, classic Bayesian tools for model selection, as the marginal likelihood of the models, are often unavailable analytically, and the models have to be estimated with MCMC methodology.This also renders leave-one-out cross-validation of the models infeasible for realistically sized data sets.In this thesis we therefore propose approximate cross-validation sampling schemes based on work by Marshall and Spiegelhalter (2003), for two model classes: conjugate change point models are applied to time series, while normal linear mixed models are used to analyze longitudinal data.The quality of the models' predictions for the left-out data is assessed with calibration checks and proper scoring rules.In several case studies we show that the approximate cross-validation results are typically close to the exact cross-validation results, and are much better suited for predictive model assessment than analogous posterior-predictive results, which can only be used for goodness-of-fit checks.