An Efficient Markov Chain Monte Carlo Method for Mixture Models by Neighborhood Pruning
Youyi Fong, Jon Wakefield, Kenneth Rice · Journal of Computational and Graphical Statistics · 2011
Inference on both finite mixture models and infinite mixture models involves a partition parameter, which describes the clustering of the observations into groups. Because this partition parameter is discrete and can take a massive number of values, making inference about it is challenging. In this article, we focus on devising efficient Metropolis–Hastings methods for models in which parameters other than the partition parameter can be integrated out. By drawing an analogy between the Metropolis–Hastings (MH) and stochastic local search (SLS) algorithms, we introduce several concepts from the SLS to guide the design of MH algorithms, including neighborhood pruning. We propose a new neighborhood pruning method for the clustering problem based on bottom-up hierarchical clustering, and use it to design a Markov chain Monte Carlo method for mixture models. Through two sets of examples, we show that our method improves the mixing significantly for cases that are challenging for the existing samplers. Online supplemental materials are available.