Beyond Accuracy: Cybersecurity Resilience Evaluation Of Intrusion Detection System Against Dos Attacks Using Agent-Based Simulation
Jeongkeun Shin, Larry Richard Carley, Geoffrey B. Dobson, Kathleen M. Carley · 2023
Machine Learning has become increasingly popular in developing Intrusion Detection Systems (IDS) for cybersecurity. However, the focus has mainly been on achieving high detection accuracy rather than evaluating the impact on cybersecurity resiliency. In this paper, we use agent-based simulation to investigate the impact of different IDS algorithms on the cybersecurity resiliency of organizations under DoS attacks. Our simulation includes a server agent equipped with either Naive Bayes or SMO-based IDS, and a cybercriminal agent capable of launching different types of Denial of Service attacks. Our results suggest that the choice of IDS algorithm can significantly affect an organization’s cybersecurity resiliency against DoS attacks. Specifically, while SMO shows better overall accuracy on the KDD Cup 1999 dataset, Naive Bayes-based IDS proves more effective in practice due to its better-balanced detection rates across different types of DoS attacks. Our findings have important implications for improving organizations’ cybersecurity posture.