A Study on Adaptive Gradient Clipping Algorithms for Differential Privacy: Enhancing Cyber Security and Trust
Siman Chen, Wei Liang · 2024
Privacy protection in federated learning is addressed by proposing an adaptive gradient clipping algorithm that simplifies the parameter tuning challenges associated with traditional gradient clipping methods. Existing privacy protection technologies are reviewed, highlighting the strengths and limitations of current differential privacy and gradient clipping techniques, and the trade-offs between managing data heterogeneity and model performance are examined. The paper introduces the mathematical principles of differential privacy and elaborates on the implementation of the proposed algorithm. By dynamically adjusting the clipping threshold, the algorithm adaptively optimizes gradient clipping parameters throughout various training stages and under different data heterogeneity conditions. Experimental validation on the NSL-KDD dataset, using both Gaussian and Laplacian mechanisms for privacy protection, evaluates the algorithm's performance and privacy protection efficacy. Results comparing the traditional DPSGD with the Ada-DPSGD algorithm demonstrate the benefits of the adaptive gradient clipping approach, showing that Ada-DPSGD consistently achieves higher model accuracy and more stable training outcomes under various noise parameter settings.