Research on Improving The Deep Learning Model of Differential Privacy Using Normalization Method
Boyu Jin · 2023
With the accumulation and processing of large amounts of personal data, privacy protection has become an important issue in the field of deep learning today. The current mainstream privacy protection method is the DPSGD mechanism that combines differential privacy and stochastic gradient descent. It protects data privacy by adding a random noise to the gradient during the training process, but it also brings certain flaws. The first is the complexity of adding noise and the choice of noise parameters. Choosing appropriate noise parameters is critical to maintaining model utility while protecting privacy, but this often requires extensive experimentation and tuning. On the other hand, the performance of DPSGD may be affected by the dependence and distribution of data. If the data is not independently and identically distributed, or the data distribution changes significantly at different points in time, this may affect the effect of differential privacy protection. In response to the problem of model performance degradation caused by adding noise, this paper studies the impact of the normalization layer on the performance of DPSGD and integrates batch normalization and DPSGD without causing additional privacy loss.