Destruction-Restoration Suppresses Data Protection Perturbations against Diffusion Models
Tianrui Qin, Xitong Gao, Juanjuan Zhao, Kejiang Ye · 2023
Diffusion models have become popular in computer vision applications due to their ability to generate high-quality images quickly, with some models achieving a high degree of realism. However, the use of these models for novel image manipulation and generation poses significant risks of being used for harmful purposes, such as the spread of misinformation or copyright infringement. Recent research has proposed techniques that involve adding imperceptible perturbations to images to mitigate privacy and copyright concerns. However, trivial image processing methods can largely destroy these protective perturbations. In this paper, we propose a new method called DiffCleaner, a destruction-restoration pipeline, that combines the advantages of image corruption and restoration methods. It can not only remove protective perturbations but also further preserve high quality generation, as well as fidelity of the original features in generated images. DiffCleaner produces specialized images with visualization and quantitative metrics closely matching unprotected results. We pitch a range of circumventing methods vs. two state-of-the-art data protection methods against diffusion models, and show that both are fragile and offer inadequate data protection. By providing a benchmark comprising a range of protective perturbations removal methods, we hope this paper will inspire and facilitate new research towards developing more resilient methods for data protection.