Differentially Private Significance Testing on Paired-Sample Data
Christine Task, Chris Clifton · 2016
Rigorous data mining results require measures of the statistical significance of the outcomes. The complexity of the data and models makes this a challenge; methods to protect privacy further complicate the issue. We demonstrate how to estimate statistical significance of results in the context of a social network analysis problem; the impact of the noise required to provide differential privacy is included in the significance measure. As a result, providing privacy does not complicate the use of the analysis. While demonstrated for social network analysis, the approach is general. The Wilcoxon signed-rank test used is appropriate for a wide variety of data with “before” and “after” measurements, and adapts well to differential privacy. We demonstrate on publicly available data with known privacy issues, showing that some apparently large differences are not significant, some small differences are, and that when the analysis is done using differential privacy, the same results can been achieved while protecting individual privacy.