Spectral Methods for Supervised Topic Models
Yining Wang, Jun Zhu · 2014
Supervised topic models simultaneously model the latent topic structure of large collections of documents and a response variable associated with each docu-ment. Existing inference methods are based on either variational approximation or Monte Carlo sampling. This paper presents a novel spectral decomposition algo-rithm to recover the parameters of supervised latent Dirichlet allocation (sLDA) models. The Spectral-sLDA algorithm is provably correct and computationally efficient. We prove a sample complexity bound and subsequently derive a suffi-cient condition for the identifiability of sLDA. Thorough experiments on a diverse range of synthetic and real-world datasets verify the theory and demonstrate the practical effectiveness of the algorithm. 1