Conformal Recursive Feature Elimination
Marcos López-De-Castro, Alberto García‐Galindo, Rubén Armañanzas · Pattern Recognition · 2026
In this work, we introduce a novel feature selection method that leverages the conformal prediction framework. This new method, named Conformal Recursive Feature Elimination (CRFE), recursively identifies and removes features that increase the non-conformity of a dataset measured under additively decomposable non-conformity functions, re-estimating feature relevance after each elimination step analogously to classical Recursive Feature Elimination (RFE). We also introduce a geometrically motivated stopping criterion for recursive feature selectors. CRFE outputs are compared with the classical RFE algorithm and the results of other state-of-the-art feature selectors across several benchmark and real-world datasets. Our experiments show that CRFE generally improves prediction-set efficiency and subset stability relative to RFE, while maintaining empirical coverage close to the nominal level. Finally, we show how the automatic stopping criterion effectively halted the recursive selection of features, selecting subsets of effective and non-redundant features. • A novel feature selector named CRFE is presented. • CRFE leverages conformal prediction to recursively remove features. • CRFE improves standard RFE in terms of performance and stability. • A data-driven stopping criterion is presented and validated.