Computing Tight Differential Privacy Guarantees Using FFT
Antti Koskela, Joonas Jälkö, Antti Honkela · Työväentutkimus Vuosikirja · 2019
Computing privacy parameters for the differentially private stochastic gradient descent method (DP-SGD) is equivalent to analysing one dimensional mechanisms. We propose a numerical accountant for evaluating the (ε, δ)-privacy loss for mech- anisms with continuous one dimensional output. The proposed method is based on a numerical approximation of an integral formula which gives the tight (ε, δ)-values. The approximation is carried out by discretising the integral and by evaluating the resulting discrete convolutions using the fast Fourier transform algorithm. We focus on the subsampled Gaussian mechanism which underlies DP-SGD. We give both theoretical error bounds and numerical error estimates for the approximation. Experimen- tal comparisons with state-of-the-art techniques demonstrate significant improvements in bound tightness and/or computation time. Python code for the method can be found in Github (https://github.com/DPBayes/PLD-Accountant/).