Optimal Differential Privacy for Deep Learning Model Training
Hlib Kokin, Oleksandr Lytvyn, Giang Nguyen · Procedia Computer Science · 2024
Differential Privacy proved itself as an optimal choice for protecting sensitive data during the machine learning lifecycle. However, achieving optimal model performance while using differential privacy remains a challenging task, where the balance between privacy level and data utility must be achieved. This work focuses on exploring the impact of differential privacy on machine learning algorithms that use sensitive data. The aim is to discover multiple tuning patterns and subsequently investigate their effects on model performance. Experiments are performed to establish the baseline and compare it with the tuning of differentially private models. The main factors of optimal differential privacy are the correct size of microbatches and batches, proper noise multiplier setting in combination with moderate learning rate. These parameters have the greatest impact on the performance of the final model in the context of medical data on heart disease. The results show the possibility of achieving comparable inference performance while preserving data privacy by the proposed approach efficiently.