Cold Start for Cloud Anomaly Detection
Yonatan Katz, Danny Raz · 2023
Cloud providers need to constantly monitor their network and provide accurate timely alerts when the service level degrades. In order to do so, many providers use Anomaly Detection (AD) systems that detect deviation of the network parameters from the normal pattern. However, the accuracy of such systems strongly depends on acquiring enough data to learn the normal behavior. This problem, known as cold start, limits the ability to detect anomalies of newly created objects and thus significantly reduces the coverage of VM anomaly detection systems.In this paper we address this problem in the context of modeling VM traffic patterns in a big cloud provider setting. We first observe that the models of the deployed VMs are clustered into a relatively small number of clusters. Thus, a small number of appropriately selected models can provide an accurate modeling for a large fraction of the VMs. We then turn to the algorithmic problem and show how to efficiently find an appropriate model for a specific newly created VM. Our evaluation, based on a large set of VMs from a major cloud provider, indicates that using our algorithms, one can significantly improve anomaly detection coverage while maintaining a compatible accuracy level.