OptiNLP: Leveraging Natural Language Processing for Optimal Feature Selection in Dimensionality Reduction
Nadimpallli Madana Kailash Varma, Lingampalli Anjali, Macharla Vishnu Vardhan Babu, Shaik Ismail, Mohammed Sharfuddin Sahel, Gagandeep Arora · 2024
In today’s data-driven world, the exponential growth of data has led to the emergence of high-dimensional datasets, often causing complexity and overload in machine learning models. To address these challenges, dimensionality reduction techniques such as feature selection and feature extraction have become pivotal in enhancing model performance and interpretability. This paper focuses on exploring various feature selection methods to assist researchers in identifying the most suitable technique for sentiment analysis tasks. Utilizing Natural Language Processing (NLP), we conduct a comprehensive evaluation of different feature selection models. Our experimental findings show that, in the context of sentiment analysis, filter method consistently tends to be more effective than other evaluated feature selection techniques. Filter performs better than other methods in terms of precision, effectiveness, and model interpretability, which makes them ideal compared to other feature selection techniques. By leveraging insights from our comparative analysis, researchers can make informed decisions on feature selection techniques.