Estimating Sparse Distributions Under Joint Communication and Privacy Constraints
Surin Ahn, Weining Chen, Ayfer Özgür · 2022 IEEE International Symposium on Information Theory (ISIT) · 2022
We consider the problem of estimating a d-dimensional, s-sparse discrete distribution from independent samples subject to a joint b-bit communication constraint and ε-local differential privacy constraint. As an intermediate step, we introduce the Privatized Random Hashing (PRH) scheme, which concatenates a hashing-based quantization strategy with the randomized response privacy mechanism. Despite its simplicity, PRH turns out to achieve the order-optimal minimax estimation error and sample complexity in the standard (non-sparse) estimation setting, for all communication and privacy regimes. We then address the sparse case by developing a two-stage, non-interactive estimation scheme based on PRH in which the first half of samples are used to localize the unknown support of the distribution, and the remaining samples are used to obtain precise estimates of the individual probabilities. Using this scheme, we characterize the minimax sample complexity of the sparse case up to logarithmic factors, unifying existing results in the literature that considered communication and privacy constraints separately.