Private Convex Optimization in General Norms
Sivakanth Gopi, Yin Tat Lee, Daogao Liu, Ruoqi Shen, Kevin Tian · Society for Industrial and Applied Mathematics eBooks · 2023
We propose a new framework for differentially private optimization of convex functions which are Lipschitz in an arbitrary norm ||·||x. Our algorithms are based on a regularized exponential mechanism which samples from the density ∞ exp(-k(F + μr)) where F is the empirical loss and τ is a regularizer which is strongly convex with respect to ||·||x, generalizing a recent work of [GLL22] to non-Euclidean settings. We show that this mechanism satisfies Gaussian differential privacy and solves both DP-ERM (empirical risk minimization) and DP-SCO (stochastic convex optimization), by using localization tools from convex geometry. Our framework is the first to apply to private convex optimization in general normed spaces, and directly recovers non-private SCO rates achieved by mirror descent, as the privacy parameter ε → ∞. As applications, for Lipschitz optimization in ℓp norms for all p ∈ (1, 2), we obtain the first optimal privacy-utility tradeoffs; for p = 1, we improve tradeoffs obtained by the recent works [AFKT21, BGN21] by at least a logarithmic factor. Our ℓp norm and Schatten-p norm optimization frameworks are complemented with polynomial-time samplers whose query complexity we explicitly bound.