Transferring Knowledge from Large Language Models for Short Text Topic Modeling
Sunjie Huang, Rui Wang, Jun Li, Mark Junjie Li, Lejun Liu, Lijuan He · 2024
Topic modeling of short texts is hindered by the sparsity of data.Existing approaches usually depend on metadata or features of short texts to indirectly increase word co-occurrence data.However, there has been little research on directly enriching co-occurrence information, such as short text expansion.To fill this research gap, we introduce a novelty Instructed-Expansion method, which generates an expanded pseudo-long document from the original short text by instructing Large Language Models (LLMs).The pseudo document retains the same topic consistency.Furthermore, we propose an Instructed-Expansion-Based Topic Model (IETM) that leverages enriched word co-occurrence in expanded documents and the uniqueness of short texts to improve the topic modeling performance.Extensive experimental results demonstrate the superiority of IETM over state-of-the-art baselines, as it produces higher-quality topics.