A robust logistic regression approach enhanced by hyperparameter optimization techniques through swarm intelligence and genetic algorithms: Advancing cancer diagnosis
Salsabila Benghazouani, Said Nouh, Abdelali Zakrani · High-Confidence Computing · 2025
Logistic regression (LR) stands out as a prevalent classification technique in machine learning, valued for its simplicity, efficiency, and suitability for deployment across diverse domains, from medical to social sciences. Despite the availability of more complex machine learning models, logistic regression (LR) remains a fundamental approach due to its well-established theoretical foundation, interpretability, and ease of implementation. However, its predictive performance can be limited by suboptimal weight estimation, particularly when dealing with high-dimensional datasets containing noisy or redundant features. To address these limitations, recent research has focused on enhancing LR by integrating optimization techniques. Acknowledging the effectiveness of Nature-Inspired Optimization Algorithms (NIOA) in solving complex problems, this study leverages the strengths of various NIOA methods, amalgamating them to create a robust technique that surpasses alternative approaches in challenging domains. This paper introduces a novel hybrid methodology, Enhanced GASI-RL (EGASI-RL), which integrates Genetic Algorithms (GA) with Swarm Intelligence (SI) algorithms to optimize the weight parameters of logistic regression models and enhance feature selection. By combining the global search capability of GA with the adaptive optimization power of SI, the proposed approach improves the predictive accuracy of LR while enhancing feature selection and robustness. Considering the significant threat cancer poses to human health, our experiments focus on breast and lung cancer datasets to evaluate the performance of EGASI-RL. The study consists of two experimental phases. In the initial phase, without feature selection, the proposed GASI-RL model demonstrated exceptional performance, surpassing many robust NIOA-based approaches. Specifically, in breast cancer classification, GASI-RL achieved an accuracy of 98.25%, precision of 97.3%, recall of 97.94%, F1-score of 97.61%, and an AUC of 98.18%. For lung cancer classification, the model achieved an accuracy of 97.53%, precision of 96.39%, recall of 98.77%, F1-score of 97.56%, and AUC of 97.55%. In the second phase, EGASI-RL was combined with various Nature-Inspired Optimization Algorithms for feature selection, with Differential Evolution (DE) emerging as the most effective. In the lung cancer dataset, EGASI-RL combined with DE attained the highest accuracy at 99.57%, along with perfect precision at 100 %, high recall at 99.16 %, a balanced F1-score of 99.58 %, and an AUC of 99.57. For the breast cancer dataset, the DE algorithm similarly delivered top-tier performance, achieving the highest accuracy at 99.73 %, precision at 99.8 %, recall at 99.4 %, F1-score at 99.59 %, and AUC at 99.75. Our findings indicate that EGASI-RL, particularly when combined with Differential Evolution for feature selection, offers a novel and highly accurate approach to healthcare analytics and cancer diagnosis, achieving superior accuracy and efficiency. This hybrid model demonstrates the potential of NIOA in enhancing logistic regression, advancing machine learning applications in critical domains.