RealDriftGenerator: A Novel Approach to Generate Concept Drift in Real World Scenario
Borong Lin, Chao Huang, Xiaohui Zhu, Nanlin Jin · 2024
Concept drift refers to the probability distribution of data generation changes over time in a data stream environment. In recent years, there has been an increasing interest in drift detection models. However, due to the lack of labeled concept drift datasets, most researchers tend to using synthetic drift data generators for model training. These generators only have relatively simple feature distributions, which fail to capture the complexity found in real-world scenarios. This paper introduces a real scenario concept drift label generator (RealDriftGenerator). This generator aims to preserve the complexity and temporal correlation of real-world scenario while generating concept drifts with user defined drift positions and drift widths. The validation result show that the temporal correlation coefficients of RealDriftGenerator is significantly higher than benchmark drift generators. Additionally, the ability of RealDriftGenerator to capture the complexity in real-world scenarios is 20% higher than benchmark drift generators(measured by model performance). The source code of RealDriftGenerator has been published on https://github.com/sniperrifle71/realDriftGenerator.