VARIATIONAL BAYESIAN LOGISTIC REGRESSION MODEL SELECTION: AN IMPROVEMENT OVER LAPLACE?
Liang Zhang, David B. Dunson · Journal of Statistical Research · 2010
Increasingly, statisticians are faced with the problem of identifying interesting subsets of predictors from among a large number of candidates. Existing methods for variable selection, such as stochastic search algorithms, tend to explore the model space too slowly in large dimensions. Shotgun stochastic search (SSS) algorithms have been proposed as an efficient alternative. As current SSS algorithms rely on conjugacy, they are not appropriate for generalized linear models without use of approximation methods. This article compares the frequently used Laplace approximation with two alternatives based on Variational Bayes methods. The comparison is illustrated using several simulated data examples and an application to the problem of predicting conception using data on timing of intercourse in the menstrual cycle. This application also illustrates the problem of selection of interactions.