Fed-DSP: Federated Learning with Dynamic Sparse Perturbation
Zhenshen Liu, Mingmeng Zhang, Qiaoyu Fu, Long Li · 2024
The rapid development of artificial intelligence and big data has enabled the deployment of federated learning (FL) in the Industrial Internet of Things (IIoT). In FL, IIoT devices (as clients) can collaboratively train a global model by sharing parameters through a central server. However, the limited resources of heterogeneous devices and high concerns for data privacy pose significant challenges to the application of FL in IIoT. To address these issues, FL with dynamic sparse perturbation (Fed-DSP) is proposed in this paper. First, a dynamic sparse adjustment strategy is proposed to reduce the risk of adversaries deducing the model through a fixed sparsity rate, thus preventing privacy leakage. As the number of training rounds increases, the sparsity effect decreases, preserving more parameters. Second, Gaussian noise is added to the sparsely perturbed model parameters without compromising the level of privacy protection, avoiding both the waste of privacy budget and potential distortion caused by adding noise to less important parameters. Finally, experimental evaluations are conducted on the MNIST and CIFAR-10 datasets under both IID and non-IID datasets. Compared to similar schemes, Fed-DSP demonstrates faster convergence, an increase in model accuracy by 2.06%, and a reduction in uplink traffic by up to 32%.