Collapsed Variational Bayesian Inference for Hidden Markov Models
Pengyu Wang, Phil Blunsom · 2013
Approximate inference for Bayesian models is dominated by two approaches, variational Bayesian inference and Markov Chain Monte Carlo. Both approaches have their own ad-vantages and disadvantages, and they can complement each other. Recently researchers have proposed collapsed variational Bayesian inference to combine the advantages of both. Such inference methods have been success-ful in several models whose hidden variables are conditionally independent given the pa-rameters. In this paper we propose two col-lapsed variational Bayesian inference algo-rithms for hidden Markov models, a pop-ular framework for representing time series data. We validate our algorithms on the nat-ural language processing task of unsupervised part-of-speech induction, showing that they are both more computationally efficient than sampling, and more accurate than standard variational Bayesian inference for HMMs. 1