Interpretability-Based Virtual Drift Detection and Adaptation Algorithm: A Case Study on Tetouan’s Energy Data
P Shahad, Ebin Deni Raj · 2024
The feature impact in multilinear regression models can vary significantly with changes in learning environments. Identifying feature importance and building models with minimal possible features helps to streamline models and reduce costs. However, in many domains, the importance of features changes due to various factors, which can decrease model performance or render the model obsolete. This leads to the phenomenon known as concept drift.Proper monitoring of changes in feature importance can help mitigate it. Interpretability techniques can determine feature importance and help manage these shifts, especially in streaming data where virtual drifts are challenging to detect and address. The proactive approach we introduced here is to handle concept drift arising from frequent virtual drifts. Our method identifies the most suitable model for recent data, ensuring more accurate forecasting and minimizing the impact of concept drifts. This approach enhances model performance by continuously adapting to changes in feature importance, providing a robust solution for dynamic data environments.