Differentially Private Federated Learning: An Information-Theoretic Perspective
Shahab Asoodeh, Weining Chen, Flávio P. Calmon, Ayfer Özgür · 2021
We propose a new technique for deriving the differential privacy parameters in federated learning (FL). We consider the setting where a machine learning model is iteratively trained using stochastic gradient descent (SGD) and only the last update is publicly released. In this approach, we interpret each training iteration as a Markov kernel. We then quantify the impact of the kernel on privacy parameters via the contraction coefficient of the$E_{\gamma}$-divergence that underlies differential privacy. To do so, we generalize the well-known Dobrushin's ergodicity coefficient, originally defined in terms of total variation distance, to a family of$f$-divergences. We then analyze the convergence rate of SGD under the proposed private FL framework.