On Multi-Round Privacy in Federated Learning
August Deer, Ramy E. Ali, A. Salman Avestimehr · 2022
Secure Aggregation is essential in federated learning to ensure that, in any given round, the server learns nothing about the local models of the users beyond their aggregate. Secure aggregation, however, does not protect the privacy of the users over multiple training rounds due to the partial user participation at each round. To quantify such long-term privacy leakage, a new metric termed as multi-round privacy has been introduced recently that requires that the server cannot reconstruct any individual model using the aggregate models from any number of training rounds. In addition, a privacy-preserving structured user selection strategy known as Multi-RoundSecAgg has been developed to ensure multi-round privacy while taking into account the convergence rate and the fairness in the user selection. Multi-RoundSecAgg, however, provides a trade-off between the multi-round privacy guarantee and the convergence rate in the sense that stronger multi-round privacy requires a larger number of training rounds. In this paper, we consider a weaker notion of multi-round privacy termed as weak multi-round privacy that still requires that the server cannot get any individual model. We show that considering this weaker notion allows for better convergence rates compared to Multi-RoundSecAgg while still protecting the privacy of the individual users in a weaker sense.