An Automatic Approach for Document-level Topic Model Evaluation
Shraey Bhatia, Jey Han Lau, Timothy J. Baldwin · 2017
Topic models jointly learn topics and document-level topic distribution.Extrinsic evaluation of topic models tends to focus exclusively on topic-level evaluation, e.g. by assessing the coherence of topics.We demonstrate that there can be large discrepancies between topic-and documentlevel model quality, and that basing model evaluation on topic-level analysis can be highly misleading.We propose a method for automatically predicting topic model quality based on analysis of documentlevel topic allocations, and provide empirical evidence for its robustness.