Dual online inference for latent Dirichlet allocation
Khoat Quang Than, Tung Doan · Asian Conference on Machine Learning · 2014
Latent Dirichlet allocation (LDA) provides an ecient tool to analyze very large text collections. In this paper, we discuss three novel contributions: (1) a proof for the tractability of the MAP estimation of topic mixtures under certain conditions that might t well with practices, even though the problem is known to be intractable in the worse case; (2) a provably fast algorithm (OFW) for inferring topic mixtures; (3) a dual online algorithm (DOLDA) for learning LDA at a large scale. We show that OFW converges to some local optima, but under certain conditions it can converge to global optima. The discussion of OFW is general and hence can be readily employed to accelerate the MAP estimation in a wide class of probabilistic models. From extensive experiments we nd that DOLDA can achieve signicantly better predictive performance and semantic quality, with lower runtime, than stochastic variational inference. Further, DOLDA enables us to easily analyze text streams or millions of documents.