SIAS-AT: Adversarial Training with Sample-Wise Individualized Adversarial Strategy
Haina Li, Zuming Zhang · 2024
Adversarial training is generally regarded as one of the most effective methods to heighten the adversarial robustness of Deep Neural Networks. Until now, most existing methods have focused on enhancing the robustness of the overall model with fixed parameters. They treat each sample equally during the training and testing phases, all adversarial samples are generated with manually specified identical adversarial strategy, such as the well-known Projected Gradient Descent. To address the problem of the disparity in robustness among classes and the limited overall robustness improvement due to ignoring sample variability, we propose Adversarial Training with Sample-wise Individualized Adversarial Strategy (SIAS-AT). In our study, we confirm the differences of robustness between classes and the overall model and explore the reasons for this. First of all, we systematically explore the preferences of different samples for adversarial strategies, involving perturbation magnitude, correlation weights, and regularization. Then we conclude that the least iteration numbers of the data point can be used to individualize the strategy of the sample and propose formulas for calculating the strategies. Finally, empirical experiments on benchmark datasets show that the method proposed in this study significantly outperforms other popular methods in performance. Our approach can be further integrated with other research results as plug-and-play components to enhance model performance.