Flexible Priors for Infinite Mixture Models
Max Welling · 2006
Most infinite mixture models in the current literature are based on the Dirichlet process prior. This prior on partitions implies a very specific (a priori) distribution on cluster sizes. A slightly more general prior known as the Pitman-Yor process prior generalizes this to a two-parameter family. The latter is the most general exchangeable partition probability function (EPPF) as defined by Pitman (Pitman, 2002) known to date. I want to argue that it is desirable to have more flexibility in expressing our prior beliefs over cluster sizes. EPPFs as defined by Pitman satisfy 3 conditions, exchangeability over objects, exchangeability over cluster-labels and consistency. In this contribution I explore the possibility to relax some of these conditions. In particular, I will discuss the consequences of relaxing exchangeability over cluster-labels and consistency. In both cases it turns out that one can formulate proper and efficient Gibbs sampling algorithms but with the added flexibility of having more control to design one’s prior.