Efficient Feature Drift Detection in Multidimensional Time Series Using PCA
Gábor Szücs, Marcell Németh, Gábor Benedek · 2024
This paper introduces an efficient method for detecting feature drift in multidimensional time series using Principal Component Analysis (PCA). Traditional approaches rely on applying univariate drift detectors to each feature individually, which is both resource-intensive and time-consuming due to the high dimensionality of data. To address this challenge, we focus on reducing the data to a lower-dimensional space using PCA, which captures the most significant variance while decreasing computational complexity. In our method, we apply drift detectors (ADWIN, KSWIN, and Page-Hinkley) to the principal components derived from PCA. A drift is detected if any two detectors signal a drift within a specified time window. The ensemble decision strategy consolidates these detections to identify baseline drift points accurately. We then use a factor loading model to map detected drifts back to the original features, quantifying the drift strength for each feature over time and reducing false positives by introducing a threshold on drift strength values. Our approach was validated on multiple datasets, demonstrating that PCA-based dimensionality reduction maintains high drift detection accuracy while significantly reducing computational costs. The results highlight the trade-offs between detection performance and computational efficiency, showcasing the practical benefits of our proposed method.