Hensel’s Compression-Based Dimensionality Reduction Approach for Privacy Protection in Federated Learning
Ahmed El Ouadrhiri, Ahmed Abdelhadi, Phu H. Phung · 2023
Differential privacy (DP) is considered a de-facto standard for protecting users’ privacy in data analysis, machine, and deep learning. Existing DP-based privacy-preserving approaches, in federated learning, consist of adding noise to the clients’ gradients before sharing them with the server. However, implementing DP on the gradient is inefficient as the privacy leakage increases by increasing the synchronization training epochs due to the composition theorem. Recently, researchers were able to recover images of the training dataset using a Generative Regression Neural Network (GRNN). In this work, we propose a novel approach using two layers of privacy protection to overcome the limitations of the existing DP-based methods. The first layer leverages Hensel’s Lemma to reduce the training dataset’ s dimension. The new dimensionality reduction method reduces the dimension of a dataset without losing information since Hensel’s Lemma guarantees uniqueness. The second layer applies DP to the compressed dataset generated by the first layer. The proposed approach overcomes the problem of privacy leakage due to composition by applying DP only once before the training. Therefore, clients train their local model on the privacy-preserving dataset generated by the second layer. Experimental results show that the proposed approach ensures strong privacy protection while achieving high accuracy. In particular, the new dimensionality reduction method achieves an accuracy of 97%, with only 25% of the original dataset size.