On Modelling Non-linear Topical Dependencies
Zhixing Li, Siqiang Wen, Juanzi Li, Peng Zhang, Jie Tang · 2014
Probabilistic topic models such as Latent Dirich-let Allocation (LDA) discover latent topics from large corpora by exploiting words ’ co-occurring relation. By observing the topical similarity be-tween words, we find that some other relation-s, such as semantic or syntax relation between words, lead to strong dependence between their topics. In this paper, sentences are represent-ed as dependency trees and a Global Topic Ran-dom Field (GTRF) is presented to model the non-linear dependencies between words. To infer our model, a new global factor is defined over all edges and the normalization factor of GRF is proven to be a constant. As a result, no in-dependent assumption is needed when inferring our model. Based on it, we develop an efficien-t expectation-maximization (EM) procedure for parameter estimation. Experimental results on four data sets show that GTRF achieves much lower perplexity than LDA and linear dependen-cy topic models and produces better topic coher-ence. 1.