Visualizing Privacy-Utility Trade-Offs in Differentially Private Data Releases
Priyanka Nanayakkara, Johes Bater, Xi Qin He, Jessica R. Hullman, Jennie Rogers · Proceedings on Privacy Enhancing Technologies · 2022
Abstract Organizations often collect private data and release aggregate statistics for the public’s benefit. If no steps toward preserving privacy are taken, adversaries may use released statistics to deduce unauthorized information about the individuals described in the private dataset. Differentially private algorithms address this challenge by slightly perturbing underlying statistics with noise, thereby mathematically limiting the amount of information that may be deduced from each data release. Properly calibrating these algorithms—and in turn the disclosure risk for people described in the dataset—requires a data curator to choose a value for a privacy budget parameter,ɛ. However, there is little formal guidance for choosingɛ, a task that requires reasoning about the probabilistic privacy–utility tradeoff. Furthermore, choosingɛin the context of statistical inference requires reasoning about accuracy trade-offs in the presence of both measurement error and differential privacy (DP) noise. We presentVisualizingPrivacy (ViP), an interactive interface that visualizes relationships betweenɛ, accuracy, and disclosure risk to support setting and splittingɛamong queries. As a user adjustsɛ, ViP dynamically updates visualizations depicting expected accuracy and risk. ViP also has an inference setting, allowing a user to reason about the impact of DP noise on statistical inferences. Finally, we present results of a study where 16 research practitioners with little to no DP background completed a set of tasks related to settingɛusing both ViP and a control. We find that ViP helps participants more correctly answer questions related to judging the probability of where a DP-noised release is likely to fall and comparing between DP-noised and non-private confidence intervals.