Critical-Threat-Alert Detection using Online Machine Learning
Samuel Ndichu, Tao Ban, Takeshi Takahashi, Daisuke Inoue · 2022 IEEE International Conference on Big Data (Big Data) · 2022
Today’s network security solutions, such as security information and event management systems, security analytics, and log management tools, tend to generate a large volume of threat alerts. The dynamic nature of threat alerts necessitates regular retraining or model updates for adequate detection performance. Meanwhile, common batch machine-learning-based threat alert analyzers are (i) time-consuming and prone to irregular updates, (ii) require massive alert data storage infrastructure, and (iii) poorly scalable for real-world applications. This paper introduces an online learning scheme for critical threat alert detection to address these challenges. This scheme treats threat alert data as a stream of items passed to the learning model, enforcing a quicker and easier response to emergent threat alerts. Furthermore, adopting a focal loss function in learning can effectively cope with the skewness commonly found in threat alert analysis scenarios. The proposed scheme is evaluated on a benchmark dataset collected in the security operation center of a large-scale enterprise network to identify potentially critical threat alerts. With 99.762% recall and 0.008% false positive rate, the proposed scheme yielded promising results for critical threat alert detection and thus points out an effective way to fight a gainst n etwork i ntrusions a nd s ecurity a lert fatigue.