Generating Efficient MCMC Kernels from Probabilistic Programs
Lingfeng Yang, Patrick Hanrahan, Noah D. Goodman · 2014
Universal probabilistic programming lan-guages (such as Church [6]) trade perfor-mance for abstraction: any model can be rep-resented compactly as an arbitrary stochas-tic computation, but costly online analy-ses are required for inference. We present a technique that recovers hand-coded lev-els of performance from a universal proba-bilistic language, for the Metropolis-Hastings (MH) MCMC inference algorithm. It takes a Church program as input and traces its execution to remove computation overhead. It then analyzes the trace for each proposal, using slicing, to identify the minimal compu-tation needed to evaluate the MH acceptance probability. Generated incremental code is much faster than a baseline implementation (up to 600x) and usually as fast as hand-coded MH kernels. 1