Improving Real-Time Concept Drift Detection using a Hybrid Transformer-Autoencoder Framework
Namburu Chaitanya Harshit, K Mounvik · 2025
In applied machine learning, concept drift, which is either gradual or abrupt changes in data distribution, can significantly reduce model performance. Typical detection methods like statistical tests or reconstruction-based models are generally reactive and not very sensitive in early detection. Our study proposes a hybrid framework consisting of Transformers and Autoencoders to model complex temporal dynamics and provide online drift detection. We create a distinct Trust Score methodology, which includes signals on (1) statistical and reconstruction-based drift metrics (more specifically, PSI, JSD, Transformer-AE error, (2) prediction uncertainty, (3) rules violations, and (4) trend of classifier error) aligned with the combined metrics defined by the Trust Score. Using a time-sequenced airline passenger data set with synthetic drift, our proposed model allows for a better detection of drift using as a whole and at different detections thresholds for both sensitivity and interpretability compared to baseline methods and provides a strong pipeline for drift detection in real time for applied machine learning. We evaluated performance using a time-sequenced airline passenger dataset having the gradually injected stimulus of drift in expectations, e.g., permuted ticket prices in later batches, broken into 10 time segments [1]. In the data, our results support that the Transformation-Autoencoder detected drift earlier and with more sensitivity than the autoencoders commonly used in the literature, and provided improved modelling above more error rates and logical violations. Therefore, a robust framework was developed to reliably monitor concept drift.