Incremental Tree-Based Inference with Dependent Normalized Random Measures
Juho Lee, Seungjin Choi · 2014
Normalized random measures (NRMs) form a broad class of discrete random measures that are used as priors for Bayesian nonparametric models. Dependent normalized random mea-sures (DNRMs) introduce dependencies in a set of NRMs, to facilitate the handling of data where the assumption of exchangeability is violated. Various methods have been developed to con-struct DNRMs; of particular interest is mixed normalized random measures (MNRMs), where DNRM is represented as a mixture of underlying shared normalized random measures. Emphasis in existing works is placed on the construction methods of DNRMs, but there is a little work on efficient inference for DNRMs. In this pa-per, we present a tree-based inference method for MNRM mixture models, extending Bayesian hi-erarchical clustering (BHC) which was originally developed as a deterministic approximate infer-ence for Dirichlet process mixture (DPM) mod-els. We also present an incremental inference for MNRM mixture models, building a tree incre-mentally in the sense that the tree structure is par-tially updated whenever a new data point comes in. The tree, when constructed in such a way, allows us to efficiently perform tree-consistent MAP inference in MRNM mixture models, de-termining a most probable tree-consistent parti-tion, as well as to compute a marginal likelihood approximately. Numerical experiments on both synthetic and real-world datasets demonstrate the usefulness of our algorithm, compared to MCMC methods.