IMPACTS OF VARIOUS TEXT PREPROCESSING METHODS FOR TOPIC MODELING TECHNIQUES
Sankari Alagukumar, R. Lawrance · 2023
Text preprocessing is crucial for topic modeling as it helps to remove noise, correct typos, standardize word usage and apply topic modeling. It involves cleaning and transforming the data to make it suitable for analysis, such as removing stop words, stemming, lemmatization and converting all texts to lowercase. Preprocessing can improve the accuracy of topic modeling by removing noise and irrelevant information. Preprocessed data canimprove the performance of the topic modeling algorithm by providing high-quality input data. In this work, it has been analyzed that the mixture of text data is evaluated with various preprocessing namely, removing stop words, eliminating punctuation, finding the root word using lemmatization, tokenization, creating document term matrix and Latent Dirichlet Allocation. This work can be used to discover hidden topics or themes in a large corpus of text data.