Attention-based Autoencoder Topic Model for Short Texts
Tian Tian, Zheng Fang · Procedia Computer Science · 2019
Due to sparse word co-occurrences, traditional topic models work poorly on short texts. Recent work aggregates short texts to augment word co-occurrence. However, aggregation has a constraint on topic distribution over a document. Thus, we propose an Attention-based Autoencoder Topic Model (AATM) in this paper. The attention mechanism of AATM emphasizes relevant information and improves topic coherence. We also incorporate externally well-trained word embeddings to introduce contextual semantic information. Moreover, AATM consists of a phrase model and a knowledge-based ranking model to discover phrase-level topics and inter-topic rankings, respectively. Experimental results on real-world data validate the efficacy of AATM.