DP-Discriminator: A Differential Privacy Evaluation Tool Based on GAN
Y. Y. Zhang, Xiaoyan Liang, Ruizhong Du, Junfeng Tian · 2024
Differential privacy has become increasingly popular in private machine learning applications due to its provable ability to limit information leakage. However, there are often vulnerabilities in the practical implementation of differentially private algorithms, making it necessary to have effective tools for evaluating them before deployment. Unfortunately, the current state of the art classifier-based evaluation tools for differential privacy are still weakly distinguishable and need to be improved. In this paper, we propose a DP-Discriminator to automatically detect the ξ-differential distinguishability (ξ-DD) for specific algorithms, which is able to efficiently discover violations of differential privacy. Specially, we give a new attack definition of ξ-DD, based on a mathematical observation, which conduce to find a larger ξ. In addition, the proposed DP-Discriminator learns the overall distribution of features across samples depending on the ability of the generating adversarial network to capture the latent features. Benefiting from powerful classifiers, DP-Discriminator is able to automatically and accurately evaluate differential privacy with minimal time consumption. The experimental results demonstrate the effectiveness of the proposed method in estimating the ξ-DD for various practical randomized algorithms. For example, the latest work that detects the ξ-DD of the algorithm RAPPOR(0.4-DP) is 0.301, whereas our tool detects ξ-DD=0.369, with an error that is one order of magnitude lower.