Model Reconstruction from Model Explanations
Smitha Milli, Ludwig Schmidt, Anca D. Dragan, Moritz Hardt · 2019
We show through theory and experiment that gradient-based explanations of a model quickly reveal the model itself. Our results speak to a tension between the desire to keep a proprietary model secret and the ability to offer model explanations.