Unsupervised Online Anomaly Detection With Parameter Adaptation for KPI Abrupt Changes
Guang Wei Yu, Zhiping Cai, Siqi Wang, Haiwen Chen, Fang Liu, Anfeng Liu · IEEE Transactions on Network and Service Management · 2019
IT companies need to monitor various Key Performance Indicators (KPIs) and detect anomalies in real time to ensure the quality and reliability of Internet-based services. However, due to the diversity of KPIs, the ambiguity and scarcity of anomalies and the lack of labels, anomaly detection for various KPIs has been a great challenge. Existing KPI anomaly detection methods have not explored the properties of anomalies in KPIs in detail to our best knowledge. Therefore, we explore anomalies in KPIs and recognize a common and important form of anomalies namedabrupt changes, which often indicate potential failures in the relevant services. Forabrupt changesin various KPIs, we proposeDDCOL, an unsupervised online anomaly detection algorithm with parameter adaptation from the perspective of anomalies for the first time. We propose three techniques: high order${D}$ifference extraction and combination,${D}$ensity-based${C}$lustering with parameter adaptation and${O}\text{n}{L}$ine detection with subsampling (DDCOL). Compared with traditional statistical methods and unsupervised learning methods, extensive experimental results and analysis on a large number of public KPIs show the competitive performance ofDDCOLand the significance ofabrupt changes. Furthermore, we provide an interpretation for the promising results, which shows thatDDCOLcan be robust to KPI expected concept drifts, and obtain a good feature distribution of normal data in KPIs.