Improving Real-Time Anomaly Detection using Multiple Instances of Micro-Cluster Detection
Rafael Copstein, Nur Zincir-Heywood, Malcolm Iain Heywood · 2024
Analysis of incoming packets in deployed systems is one of the main methods used for detection of anomalous behaviour. Techniques utilizing supervised learning subject to the need of retraining if the observed behaviour in the system changes over time. Unsupervised techniques mitigate this problem but are not always capable of real-time analysis. Real-time unsupervised techniques bring to the table both the adaptability to dynamic behaviour as well as the ability to detect and alert about anomalies in real-time. A recent state-of-the-art technique, MIDAS, shows real-time capabilities while being unsupervised, but recent works have showed that it still had some shortcomings regarding its performance over more specific datasets. An alternative method has been proposed, namely MIMC, that builds on the foundation set by MIDAS. In this paper it is shown that, for the datasets of interest, there is always a way to setup MIMC that yields a higher performance than MIDAS. Furthermore, a method for determining parameters for the technique is also presented, and it is shown that it improves the yielded performance even further in a majority of cases.