Refining Fidelity Metrics for Explainable Recommendations
Mikhail Baklanov, Veronika Bogina, Yehonatan Elisha, Yahlly Schein, Liron Allerhand, Oren Barkan, Noam Koenigstein · 2025
Counterfactual evaluation provides a promising framework for assessing explanation fidelity in recommender systems, but perturbation metrics adapted from computer vision suffer three key limitations: (1) they conflate explaining and contradictory features, (2) they average over entire user histories instead of prioritizing concise, high-impact explanations, and (3) they use fixed-percentage perturbations, leading to inconsistencies across users.We introduce refined counterfactual metrics that focus on the most relevant explaining features, exclude contradictory elements, and assess fidelity at a fixed explanation length, ensuring a more consistent and interpretable evaluation.Our code is at: https:// github.com/DeltaLabTLV/FidelityMetrics4XRec