Beyond conformal predictors: Adaptive Conformal Inference with confidence predictors
Johan Hallberg Szabadváry, Tuve Löfström · Pattern Recognition · 2025
Adaptive Conformal Inference (ACI) provides finite-sample coverage guarantees, enhancing the prediction reliability under non-exchangeability. This study demonstrates that these desirable properties of ACI do not require the use of Conformal Predictors (CP). We show that the guarantees hold for the broader class of confidence predictors, defined by the requirement of producing nested prediction sets, a property we argue is essential for meaningful confidence statements. We empirically investigate the performance of Non-Conformal Confidence Predictors (NCCP) against CP when used with ACI on non-exchangeable data. In online settings, the NCCP offers significant computational advantages while maintaining a comparable predictive efficiency. In batch settings, inductive NCCP (INCCP) can outperform inductive CP (ICP) by utilising the full training dataset without requiring a separate calibration set, leading to improved efficiency, particularly when the data are limited. Although these initial results highlight NCCP as a theoretically sound and practically effective alternative to CP for uncertainty quantification with ACI in non-exchangeable scenarios, further empirical studies are warranted across diverse datasets and predictors. • ACI ensures finite-sample coverage guarantees without conformal predictors. • NCCPs can match or exceed CP performance on non-exchangeable data. • Inductive NCCP avoids data splitting, improving predictive efficiency. • NCCP’s online benefit is speed; in batch settings, it’s data usage. • Nested prediction sets are vital for meaningful confidence predictions.