A Quantifying Approach for Evaluating Differential Privacy in Deep Learning
Lihua Yin, Yang Lv, Zhe Sun, Fuqiang Tao, Ran Li · 2022
Differential privacy is a strong notion of privacy. The privacy risk of the differentially private machine learning algorithm can be quantified by privacy loss. Deep learning makes privacy loss difficult to measure accurately, making it difficult for model designers to make trade-offs between utility and privacy. In this paper, we propose a membership inference-based evaluation method. This method estimates privacy risk of differentially private algorithms by inserting label samples into the target dataset. Our method can obtain more accurate evaluation results than previous evaluation methods by examining differential distributions in training datasets. Additionally, we found that in certain application scenarios, adding too much noise during model training could reduce privacy effectiveness. This result can guide model designers in the parameter configuration of differentially private machine learning.