POSEIDON: Privacy-Preserving Federated Neural Network Learning
Sinem Sav, Apostolos Pyrgelis, Juan Ramón Troncoso-Pastoriza, David Froelicher, Jean-Philippe Bossuat, João Sá Sousa, Jean‐Pierre Hubaux · 2021
Furthermore, the trusted party becomes a single point of failure, thus both data and model privacy could be compromised by data breaches, hacking, leaks, etc.Hence, solutions originating from the cryptographic community replace and emulate the trusted party with a group of computing servers.In particular, to enable privacy-preserving training of NNs, several studies employ multiparty computation (MPC) techniques and operate on the two [83], [28], three [82], [110], [111], or four [26], [27] server models.Such approaches, however, limit the number of parties among which the trust is split, often assume an honest majority among the computing servers, and require parties to communicate (i.e., secret share) their data outside their premises.This might not be acceptable due to the privacy and confidentiality requirements and the strict data protection regulations.Furthermore, the computing servers do not operate on their own data or benefit from the model training; hence, their only incentive is the reputation harm if they are caught, which increases the possibility of malicious behavior.A recently proposed alternative for privacy-preserving training of NNs -without data outsourcing -is federated learning.Instead of bringing the data to the model, the model is brought (via a coordinating server) to the clients, who perform model updates on their local data.The updated models from the parties are averaged to obtain the global NN model [75], [63].Although federated learning retains the sensitive input data locally and eliminates the need for data outsourcing, the model, that might also be sensitive, e.g., due to proprietary reasons, becomes available to the coordinating server, thus placing the latter in a position of power with respect to the remaining parties.Recent research demonstrates that sharing intermediate model updates among the parties or with the server might lead to various privacy attacks, such as extracting parties' inputs [53], [113], [120] or membership inference [78], [86].Consequently, several works employ differential privacy to enable privacy-preserving exchanges of intermediate values and to obtain models that are free from adversarial inferences in federated learning settings [67], [101], [76]. Although differentially private techniques partially limit attacks to federated learning, they decrease the utility of the data and the resulting ML model.Furthermore, training robust and accurate models requires high privacy budgets, and as such, the level of privacy achieved in practice remains unclear [55].Therefore, a distributed privacy-preserving deep learning approach requires strong cryptographic protection of the intermediate model updates during the training, as well as of the final model weights.Recent cryptographic approaches for private distributed learning, e.g., [119], [42], not only have limited ML functionalities, i.e., regularized or generalized linear models, but also employ traditional encryption schemes that make them vulnerable to post-quantum attacks.This should be cautiously considered, as recent advances in quantum computing [47], [87], [105], [116], increase the need for deploying quantum-resilient cryptographic schemes that eliminate Abstract-In this paper, we address the problem of privacypreserving training and evaluation of neural networks in an N-party, federated learning setting.We propose a novel system, POSEIDON, the first of its kind in the regime of privacy-preserving neural network training.It employs multiparty lattice-based cryptography to preserve the confidentiality of the training data, the model, and the evaluation data, under a passive-adversary model and collusions between up to N -1 parties.To efficiently execute the secure backpropagation algorithm for training neural networks, we provide a generic packing approach that enables Single Instruction, Multiple Data (SIMD) operations on encrypted data.We also introduce arbitrary linear transformations within the cryptographic bootstrapping operation, optimizing the costly cryptographic computations over the parties, and we define a constrained optimization problem for choosing the cryptographic parameters.Our experimental results show that POSEIDON achieves accuracy similar to centralized or decentralized non-private approaches and that its computation and communication overhead scales linearly with the number of parties.POSEIDON trains a 3-layer neural network on the MNIST dataset with 784 features and 60K samples distributed among 10 parties in less than 2 hours.