Auxiliary variable transformations for intractable distributions
Paul Vanetti · cIRcle (University of British Columbia) · 2013
Expectations over probability distributions can be approximated by Markov chain Monte Carlo methods when the density can be evaluated up to a normalizing constant. However, there exist cases where this density takes on the form of an intractable integral and therefore cannot be computed exactly. We explore a class of auxiliary variable methods which allow correct sampling from such distributions. In some cases, existing approaches which employ these methods can be inefficient, requiring long computation times. We identify causes for this inefficiency and demonstrate how this can be improved when we can develop a reasonable importance sampling estimate of the integral. We discuss applications for our methods, placing a particular focus on the Dirichlet process with a non-conjugate base distribution. We show how the auxiliary variables can be interpreted in this non-parametric context, and how we can develop proposals which provide greater computational efficiency.