Admixture of Poisson MRFs: A Topic Model with Word Dependencies
David I. Inouye, Pradeep Ravikumar, Inderjit S. Dhillon · 2014
This paper introduces a new topic model based on an admixture of Poisson Markov Random Fields (APM), which can model dependencies between words as opposed to previous inde-pendent topic models such as PLSA (Hof-mann, 1999), LDA (Blei et al., 2003) or SAM (Reisinger et al., 2010). We propose a class of admixture models that generalizes previous topic models and show an equivalence between the conditional distribution of LDA and independent Poissons—suggesting that APM subsumes the modeling power of LDA. We present a tractable method for estimating the parameters of an APM based on the pseudo log-likelihood and demon-strate the benefits of APM over previous models by preliminary qualitative and quantitative exper-iments. 1.