Feature Selection Using Ensemble Lasso Regression, Random Forest and Recursive Feature Elimination Methods in Breast Cancer Classification
Wilda Royhan, Sutarman Sutarman, Amalia Amalia · 2025
Healthcare datasets, especially those used in cancer diagnosis, often present challenges such as high dimensionality, redundancy, and irrelevant features, which can reduce the performance and reliability of machine learning models. This research proposes a robust ensemble feature selection method to address these challenges, combining Lasso Regression, Random Forest, and Recursive Feature Elimination (RFE). By leveraging the complementary strengths of these algorithms, the ensemble approach aims to improve feature selection stability and enhance classification accuracy. Additionally, Shannon entropy is employed to assess data complexity and guide the feature selection process. The proposed method is applied to the Breast Cancer Wisconsin (Diagnostic) dataset, and its performance is evaluated using metrics such as accuracy, precision, recall, and F1-score. Experimental results demonstrate that the ensemble method outperforms individual feature selection techniques, achieving higher classification accuracy and robustness in handling complex and imbalanced datasets. This study advances machine learning-based diagnostic tools by offering a reliable framework for high-dimensional medical data analysis. The findings underscore the potential of ensemble feature selection in improving interpretability, reducing computational overhead, and enhancing predictive accuracy in breast cancer diagnosis, paving the way for more effective clinical decision support systems.