StreamDP: Continual Observation of Real-world Data Streams with Differential Privacy
Shuailou Li, Wen Yu, Zhaoyang Wang, Wenbo Wang, Lisong Zhang, Dan Meng · 2024
The real-time collection and query analysis of dynamic data streams have become increasingly common and important, yet the protection of sensitive private information remains a pressing challenge. Differential privacy, as the gold standard for protecting personal data privacy, has been widely studied and applied. However, existing mechanisms mostly focus on static datasets and specific simple stream queries. This paper presents StreamDP, a novel framework designed to achieve differential privacy for complex real-world stream queries. We introduce the observation-prediction mechanism that predicts statistics such as join attribute frequency using observations and truncates the data stream based on the predicted threshold. Then we design operation-oriented recursive sensitivity calculation rules and employ a hierarchy algorithm for noise perturbation. Extensive experimental evaluations on multiple real-world datasets and distributed stream processing benchmarks show that StreamDP can support various complex real-world data stream queries/applications with high utility and low-performance overhead.