Dynamic Differential Privacy in Hierarchical Federated Learning: A Layerwise Adaptive Framework
Zhongyuan Qin, Dinglian Wang, Minghua Wang · 2024
With the growing emphasis on data privacy, Federated Learning (FL) has emerged as a novel distributed machine learning approach. It allows multiple participants to collaboratively train models without sharing their local data. However, ongoing research has shown that even without direct data exchange, sharing model parameters can still lead to privacy breaches. Differential Privacy (DP) helps mitigate this by adding noise to model parameters, ensuring privacy. While effective, this noise can degrade model performance. Therefore, finding a balance between noise injection and model accuracy remains a key challenge.To address this issue, we propose a dynamic noise addition method for hierarchical federated learning. Our method stratifies the model and injects different levels of noise based on each parameter’s contribution to the global model. We introduce a layer-wise asynchronous noise addition strategy. In this strategy, Deep Neural Networks (DNNs) are divided into shallow and deep layers. Deeper layers receive a larger privacy budget to enhance their protection. Additionally, we prioritize important features in the input layer, applying stricter noise to parameters with higher significance.We validated our algorithm on DNNs using two distinct datasets. The results confirm the effectiveness of our method in balancing privacy and model performance.