Pachinko Allocation: Scalable Mixture Models of Topic Correlations
Wei Li, Andrew McCallum · 2008
Statistical topic models are increasingly popular tools for summarization and manifold discovery in discrete data. However, the majority of existing approaches capture no or limited correlations between topics. In this paper, we propose the pachinko allocation model (PAM), which captures arbitrary topic correlations using a directed acyclic graph (DAG). The leaves of the DAG represent individual words in the vocabulary, while each interior node represents a correlation among its children, which may be words or other interior nodes (topics). As we have observed, topic correlations are usually sparse. By taking advantage of this property, we develop a highly-scalable inference algorithm for PAM. In our experiments, we show improved performance of PAM in document classification, likelihood of held-out data, topical keyword coherence, and the ability to support a great number of fine-grained topics in very large datasets. Keywords: pachinko allocation, topic models, Gibbs sampling