Topic Modelling: A Comparison of The Performance of Latent Dirichlet Allocation and LDA2vec Model on Bangla Newspaper
Md Umaid Hasan, Md. Motaher Hossain, Adnan Ahmed, Mohammad Shahidur Rahman · 2019
Topic modeling is a statistical data mining method for organizing the documents with variable contents under similar topics.It is utilized to reveal the concealed topics from an enormous collection of articles or documents.In this research, we have conducted an experiment on the effectiveness of two trending topic modeling methods on Bangla news corpus.Our target is to find an effective way of analyzing Bangla news documents and categorize them automatically for a recommendation system and searching. Different Models have been found effective for different languages because of their unique morphological structure.In this research LDA and its hybrid with word2vec known as lda2vec have been chosen as techniques to extract topics from Bangla news documents.LDA is a matrix factorization technique and as our observation in this research, this model is effective on Bangla despite the syntax difference with English.For the testing and implementation purpose, we developed a technique to find topics for not factorized documents from LDA and lda2vec.As our finding, lda2vec gave 85.66% accuracy over topics for test documents where accuracy for LDA was 62.45% Which makes lda2vec more reliable for the implementation purpose.