Virtual-SRE For Monitoring Large Scale Time-series Data
Wei Dong Zhang, Chris Challis · 2021 IEEE International Conference on Big Data (Big Data) · 2021
Monitoring online time-series data and providing real-time alerts are crucial for a variety of industry applications. But doing it at a large scale is a challenging task. In this paper, we introduce Virtual-SRE, an anomaly detection framework which utilizes human knowledge and is suitable for large-scale applications. The proposed approach starts without any human labeling. Instead, human knowledge is obtained through feedbacks. A joint data and model optimization process is proposed to ensure that accurate models can be built. This makes it easily applicable to large-scale and heterogeneous tasks while enjoying the benefits of supervised learning. The proposed framework has been deployed to monitor thousands of time-series data streams and obtained favorable results.