VPP: Privacy Preserving Machine Learning via Undervolting
Md. Shohidul Islam, Behnam Omidi, Ihsen Alouani, Khaled N. Khasawneh · 2023
Machine Learning (ML) systems are susceptible to membership inference attacks (MIAs), which leak private information from the training data. Specifically, MIAs are able to infer whether a target sample has been used in the training data of a given model. Such privacy breaching concern motivated several defenses against MIAs. However, most of the state-of-theart defenses such as Differential Privacy (DP) come at the cost of lower utility (i.e, classification accuracy). In this work, we propose Privacy Preserving Volt $(V_{PP})$, a new lightweight inference-time approach that leverages undervolting for privacy-preserving ML. Unlike related work, VPPmaintains protected models’ utility without requiring re-training. The key insight of our method is to blur the MIA differential analysis outcome by comprehensively garbling the model features using random noise. Unlike DP, which injects noise within the gradient at training time, VPPinjects computational randomness in a set of layers’ during inference through carefully designed undervolting Specifically, we propose a bi-objective optimization approach to identify the noise characteristics that yield privacypreserving properties while maintaining the protected model’s utility. Extensive experimental results demonstrate that VPPyields a significantly more interesting utility/privacy tradeoff compared to prior defenses. For example, with comparable privacy protection on CIFAR-10 benchmark, VPPimproves the utility by 32.93% over DP-SGD. Besides, while related noisebased defenses are defeated by label-only attacks, VPPshows high resilience to such adaptive MLA. More over, VPPcomes with a by-product inference power gain of up to 61%. Finally, for a comprehensive analysis, we propose a new adaptive attacks that operate on the expectation over the stochastic model behavior. We believe that VPPrepresents a significant step towards practical privacy preserving techniques and considerably improves the state-of-the-art.