The Trade-off Between Privacy & Quality for Counterfactual Explanations
Sjoerd Berning, Vincent Dunning, Dayana Spagnuelo, Thijs Veugen, Jasper van der Waa · 2024
Counterfactual explanations are a promising direction of explainable AI in many domains such as healthcare. These explanations produce a counterexample from the dataset that shows, for example, what should change about a patient to reduce their risk of developing diabetes type 2. However, this poses a clear privacy risk when the dataset contains information about people. Recent literature shows that this risk can be mitigated by using k-anonymity to generalise the explanation, such that it is not about a single person. In this paper, we investigate the trade-offs between privacy and explanation quality in the medical domain. Our results show that for around 40% of the explained cases, the real gain in privacy is limited as the generalisation increases while the explanations continue decreasing in quality. These findings suggest that this can be an unsuitable strategy in some situations, as its effectiveness depends on characteristics of the underlying dataset.