Ensemble based unsupervised anomaly detection with concept drift adaptation for time series data
Danlei Li, Nirmal Sukumaran Nair, Kevin I‐Kai Wang, Kouichi Sakurai · 2024
Anomaly detection is the identification of instances that substantially deviate from the majority of the data and do not conform to a well-defined normal behavior. Investigating time series anomalies has become increasingly popular in recent years due to widely deployed IoT devices and sensors. Although most existing methods apply image anomaly detection algorithms directly on time series and commonly assume the data is IID (independent and identically distributed). However, in most realworld applications, the underlying data distribution alters over time. Changes in data over time are referred to as concept drift and can significantly degrade the efficacy of an anomaly detection model. Existing research also indicates that the majority of anomaly detection frameworks ignored concept drift, resulting in poor long-term model performance. This paper presents an ensemble-based, unsupervised anomaly detection framework that is adaptive to the concept drift in time series data. Both distribution and performance-based techniques are used to detect concept drift and update the model to ensure the framework’s longterm performance reliability. The proposed method is evaluated using a real-world univariate time series data that captures sea surface temperature data over a 2 -year period. By comparing our proposed framework to other state-of-the-art algorithms, we demonstrate our proposed method has obvious benefits in accuracy and long-term stability.