Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search
Pengfei Xia, Ziqiang Li, Bin Li · Proceedings of the Genetic and Evolutionary Computation Conference Companion · 2024
How to accurately evaluate the adversarial robustness of Deep Neural Networks (DNNs) is critical for their deployment in real-world applications. An ideal indicator of robustness is adversarial risk. Unfortunately, since it involves maximizing the 0--1 loss, calculating the true risk is technically intractable. The most common solution for this is to compute an approximate risk by replacing the 0--1 loss with a surrogate, such as Cross-Entropy loss. However, these functions are all manually designed and may not be well suited for adversarial robustness evaluation. In this paper, we leverage AutoML to tighten the gap between the true and approximate risks. First, AutoLoss-AR, the first method to search for surrogate losses for adversarial risk is proposed. The experimental results on 10 adversarially trained models demonstrate the effectiveness of the proposed method: the risks evaluated using the best-discovered losses are 0.2% to 1.6% better than those evaluated with baselines. Second, 5 surrogate losses with clean and readable formulas are distilled out and tested on 7 unseen adversarially trained models. These losses outperform the baselines by 0.8% to 2.4%, indicating that they can be used individually as some kind of new knowledge. Our code is publicly available at https://github.com/xpf/Tightening-Approximation-Error.