Redefining Trust: Assessing Reliability of Machine Learning Algorithms in Intrusion Detection Systems
Hossein Sayadi, Zhangying He, Tahereh Miari, Mehrdad Aliasgari · 2024
The performance limitations of conventional software-based Intrusion Detection Systems (IDSs) have paved the way for the emergence of hardware-oriented approaches. These approaches harness the power of Machine Learning (ML) algorithms applied to processors’ hardware-related data, thereby enhancing the overall system’s security and efficiency. However, ensuring the dependability of ML models’ decisions is crucial, yet this aspect has been largely overlooked in previous studies. In this paper, we delve into the reliability of machine learning algorithms within hardware-oriented intrusion detection systems, focusing specifically on malware detection. Our investigation aims to bridge the existing gap by shedding light on the tradeoffs between performance vs. reliability and robustness levels exhibited by ML models in intrusion detection systems. We conduct a thorough evaluation of ML algorithms in hardware-oriented IDSs, considering factors such as training data size, number of hardware events used, and internal data separability (malware stealthiness). Additionally, we incorporate an effective model observer module to assess prediction probabilities in real-time; thereby, employing a threshold to determine the ML model’s confidence for enhanced reliable intrusion detection.