A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models
Mohammad Emtiyaz Khan, Shakir Mohamed, Benjamin M. Marlin, Kevin P. Murphy · 2012
The development of accurate models and effi-cient algorithms for the analysis of multivari-ate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using La-tent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate lin-ear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to ex-isting algorithms for multinomial logit/probit likelihoods on several problems, including in-ference in multinomial Gaussian process clas-sification and learning in latent factor mod-els. Our extensive comparisons demonstrate that our stick-breaking model effectively cap-tures correlation in discrete data and is well suited for the analysis of categorical data. 1