Performance with Feature Selection Based Machine Learning: Combination of Low Variance Filter Pearson Correlation for Bots and Brute Force
Andri Saputra, Kalamullah Ramli, Anto Satriyo Nugroho · 2025
This research examines the efficacy of selection feature selection based machine-learning by combining low variance filter, Pearson correlation low variance filter algorithms to detect Bot and Brute Force attacks. The research uses CSE-CIC-IDS2018 dataset and employs a systematic methodology that includes normalization, data cleaning, and equal-size sampling. Through the implementation of low variance filtering with a 0.016 threshold followed by Pearson correlation analysis, the research achieved significant feature reduction while maintaining high classification accuracy. The experimental results demonstrate that for Bot detection, reducing features from 80 to 21 yielded 100% accuracy on threshold 0.1 and threshold 0.2. In Brute Force-Web attack detection, utilizing 57 selected features achieved 99.3% accuracy on threshold 0.4, while SSH-Brute Force detection maintained overall 100% accuracy with only 15 features. Performance evaluation using Decision Tree classification revealed consistently high precision and recall values across all attack types. The findings confirm that the proposed feature selection methodology effectively optimizes intrusion detection systems by identifying the most relevant features while reducing computational complexity without compromising detection capabilities. Future work is recommended to experiment develop real-time predictions for combine ML models with deep learning in the future.