ReFIN: Recurrent Feature Importance Network for Dynamic Feature Selection
Md. Jahangir Alam, Asif Al Zawad, Huu-Hoa Nguyen, Dewan Md. Farid · 2024
Dynamic feature selection is critical for improving the flexibility and efficiency of predictive models in machine learning, particularly when dealing with sequential data streams. In this study, we have introduced a novel framework, the Recurrent Feature Importance Network (ReFIN), which integrates Recurrent Neural Networks (RNNs) with decision tree principles to dynamically select and adjust feature importance over time. This approach begins by randomly selecting subsets of features and evaluating their performance using a Random Forest Classifier with attribute bagging. The performance of each subset is assessed, and the best-performing feature sets are clustered into categories of Good, Average, and Poor importance. These selected features are then utilized to train the ReFIN model, which leverages an LSTM-based RNN architecture to continuously update feature importance based on evolving data contexts. Through systematic iteration, the proposed method adjusts feature importance dynamically, ensuring that the most relevant features are prioritized while less important ones are down-weighted or replaced as data patterns shift. This adaptability is particularly effective in handling concept drift, a common challenge in data stream classification, where data distributions change over time. Experimental results demonstrate that ReFIN significantly enhances model accuracy and efficiency by optimizing feature selection and improving the interpretability of the classification process in dynamic environments.