Unifying clustering and representation learning for unsupervised text analysis: A Bayesian knowledge-enhanced approach
Gianni Costa, Riccardo Ortale · Information Fusion · 2024
Clustering and representation learning are foundational tasks in natural language processing and text mining, aiming to structure and semantically encode text documents, respectively. While clustering organizes documents into cohesive groups based on similarity, representation learning generates low-dimensional embeddings that capture the nuances of document semantics. This paper presents a novel knowledge-enhanced approach that synergistically integrates these two tasks, improving performance in both areas. Our method employs a latent-factor Bayesian generative model, named MINING ( docuMent clusterINg and embeddING ), along with a specialized collapsed Gibbs sampling algorithm. We enrich the learned representations by incorporating external knowledge from word and entity embeddings, enhancing their semantic and syntactic richness. Our approach treats clustering and representation learning as interdependent tasks, allowing them to inform and refine one another. Extensive experiments on benchmark datasets demonstrate that our integrated approach outperforms traditional methods that carry out clustering and representation learning as separate tasks. • New approach to text analysis unifies clustering and representation learning. • Novel mathematical foundation laid for the principled integration of both tasks. • Innovative Bayesian latent-factor model of text corpora formalizes their interplay. • Document representations enhanced via external knowledge graph and text collections. • MCMC inference allows for both tasks to be performed jointly and cooperatively.