An Introduction to Bayesian Inference Via Variational Approximations: Supplemental Notes
Justin Grimmer · 2010
1.1 The Tractability-Fit Tradeoff in Variational Approximations The goal of a variational approximation is to approximate a posterior, p(β|Y ) by making an approximating distribution, q(β), as close as possible to the true posterior (Bishop, 2006). We search over the space of approximating distributions in order to find the particular distribution with the minimum KL-divergence with the actual posterior. Formally, we search over the set of approximating distributions q(β) to minimize