Towards a More Efficient Sinkhorn Distance Computation in Neural Topic Models
Pierre Dardouillet, Kavé Salamatian, Hervé Verjus, Faiza Loukil, David Télisson, Olivier Le Van · 2025
In natural language processing, topic modeling aims to extract a corpora latent structure. In recent years, optimal transport distances have improved the topic extraction capabilities of Neural Topic Models (NTMs). More precisely, the Sinkhorn-Knopp algorithm is used to compute the blurred Wasserstein distance with relatively low complexity and is fully differentiable. This algorithm ease of implementation and advantages are thus particularly interesting for enforcing desired properties in NTMs. However, the algorithm can be unstable and inefficient under low blur setups, hence hindering overall topic model performances. In this article, we first assess the stability and efficiency of the Sinkhorn-Knopp algorithm in NTM scenarios. We compare five of the most relevant variations of this algorithm, and three distinct usages in NTMs. We evaluate each specific Sinkhorn-Knopp algorithm variation and topic model architecture independently, under various quantitative and qualitative metrics. Furthermore, we propose a novel method that focuses on the Sinkhorn-Knopp algorithm initialization, by reusing its dual variables from previous model updates as warm-start values. Our experiments reveal that our method can drastically improve the computation efficiency of the algorithm by reducing its number of iterations by up to 70%, and is easily applicable to any topic model using the Sinkhorn distance.