HarpLDA+: Optimizing latent dirichlet allocation for parallel efficiency
Bo Peng, Bingjing Zhang, Langshi Chen, Mihai Avram, Robert Henschel, Craig A. Stewart, Shaojuan Zhu, Emily Mccallum, Lisa Smith, Zahniser Tom, Omer Jon, Judy Qiu · 2017
Latent Dirichlet Allocation (LDA) is a widely used machine learning technique in topic modeling and data analysis. Training large LDA models on big datasets involves dynamic and irregular computation patterns and is a major challenge to both algorithm optimization and system design. In this paper, we present a comprehensive benchmarking of our novel synchronized LDA training system HarpLDA+ based on Hadoop and Java. It demonstrates impressive performance when compared to three other MPI/C++ based state-of-the-art systems, which are LightLDA, F+NomadLDA, and WarpLDA. HarpLDA+ uses optimized collective communication with a timer control for load balance, leading to stable scalability in both shared-memory and distributed systems. We demonstrate in the experiments that HarpLDA+ is effective in reducing synchronization and communication overhead and outperforms the other three LDA training systems.