Differential Privacy for Text Analytics via Natural Text Sanitization
Xiang Yue, Minxin Du, Tianhao Wang, Yaliang Li, Huan Sun, Sherman S. M. Chow · 2021
Texts convey sophisticated knowledge.However, texts also convey sensitive information.Despite the success of general-purpose language models and domain-specific mechanisms with differential privacy (DP), existing text sanitization mechanisms still provide low utility, as cursed by the high-dimensional text representation.The companion issue of utilizing sanitized texts for downstream analytics is also under-explored.This paper takes a direct approach to text sanitization.Our insight is to consider both sensitivity and similarity via our new local DP notion.The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacypreserving natural language processing over the BERT language model with promising utility.Surprisingly, the high utility does not boost up the success rate of inference attacks.