Sequential Model-Free Anomaly Detection for Big Data Streams
Mehmet Necip Kurt, Yasin Yılmaz, Xiaodong Wang · 2019
We study sequential anomaly detection for big data streams where the nominal and anomalous high-dimensional probabilistic data models are unknown. We propose a model-free solution approach in that we firstly compute a set of univariate summary statistics from a nominal dataset in an offline phase where the summary statistics are useful to distinguish anomalous data from nominal data. We then evaluate whether the online summary statistics deviate from the nominal case via a cumulative sum-like detector. Our experiments with real-world data illustrate the advantages of the proposed detector in early and reliable anomaly detection in big data settings compared to the existing alternatives.