Performance Impact of Feature-Selection and Hyperparameter Tuning in IIoT Attack Detection
Prathibha Keshavamurthy, Erin Colleen Smith, Aasif Majeed, Sarvesh S. Kulkarni · 2025
Industrial Internet of Things (IIoT) has brought convenience to businesses but has also increased their visibility and thus the attack surface of corporate networks. Machine learning (ML) methods for attack detection and prevention have been studied extensively and have shown great cyber-defense potential for IIoT devices. During the ML model training process, a careful selection of important features and parameter tuning is essential for accurate traffic classification. This paper investigates extensively the impact of feature selection and hyperparameter (HP) tuning on the performance of ML algorithms in the detection of attacks on IIoT devices. Using an ensemble of feature selection algorithms, we identify the most relevant features in the Edge-IIoT dataset. Based on the ranking of features thus obtained, we create three distinct datasets. We then demonstrate that the datasets that retain smaller subsets of the most important features greatly reduce the computational cost compared to the original, cleansed dataset from which they are derived. However, this computational speed-up comes with only a very minor decrease in detection efficacy. Next, we test ML algorithms to determine what gains, if any, can be obtained with HP tuning over the same models without such tuning. We demonstrate that HP tuning enhances the correctness of predictions of an ML algorithm by an average of less than 1% at the cost of a multi-fold increase in computation time. Finally, we recommend a selection of the optimum number of features and hyperparameters to effectively balance the performance and the complexity and computation cost of the model.