Bayesian Model Scoring in Markov Random Fields
Sridevi Parise, Max Welling · The MIT Press eBooks · 2007
Scoring structures of undirected graphical models by means of evaluating the marginal likelihood is very hard. The main reason is the presence of the parti-tion function which is intractable to evaluate, let alone integrate over. We propose to approximate the marginal likelihood by employing two levels of approximation: we assume normality of the posterior (the Laplace approximation) and approxi-mate all remaining intractable quantities using belief propagation and the linear response approximation. This results in a fast procedure for model scoring. Em-pirically, we find that our procedure has about two orders of magnitude better accuracy than standard BIC methods for small datasets, but deteriorates when the size of the dataset grows. 1