Detection of Real Concept Drift Under Noisy Data Stream
Sirvan Parasteh, Samira Sadaoui, Mohammad Sadegh Khosravani · 2023
Concept drift detection in noisy data streams is challenging yet essential. This paper introduces NPRDD, a new concept drift detection algorithm that is robust to noise and accurately identifies Real drifts. NPRDD operates on a moving window of recent data, utilizing predicted class probabilities and cross-entropy-based surprise measures to weigh real drift candidates. In line with the Bayesian definition of Real concept drift, NPRDD considers a sample as a drift candidate when the classifier makes an error but is highly confident in its judgment. We evaluate NPRDD on synthetic datasets by varying the noise levels and comparing its performance with other well-established methods. Our results show that NPRDD outperforms other methods regarding ROC-AUC and Accuracy metrics.