Curriculum Defense: An Effective Adversarial Training Method
Huilin Yin, Xiaoyang Deng, Jun Yan · 2022 41st Chinese Control Conference (CCC) · 2022
To resolve the challenge of adversarial attacks, several methods have been proposed. Adversarial training is one of the most effective defense methods. However, most of the adversarial training methods sacrifice the accuracy on the clean testing dataset and demonstrate the mediocre performance under the multi-step attacks. In our opinion, building a trustworthy machine learning system is an iterative learning process. The process by which a machine builds intelligence is similar to the process by which the human brain builds intelligence, following a process from easiness to difficulty. Such a learning technology can be implemented by curriculum learning. In this paper, we propose a curriculum learning based method to improve the robustness of the visual model. We treat adversarial examples as augmented data whose fitting difficulties surpass the clean examples. The “curriculum dropout” regularization mechanism is added in the adversarial defense to improve the adversarial robustness under the attack. In our experiment, we first guarantee the generalization of the model without the perturbations and then try to train the neural visual classifier under the single-step Fast Gradient Signed Method (FGSM) attack and multi-step Projected Gradient Descent (PGD) attack. The considerable generalization ability and robustness on the CIFAR-10 dataset could be achieved via the curriculum defense method. Our finding in the curriculum defense can be highlighted that the wide neural network model with fewer forgetting defects would remember even more. The accuracy of most models on the clean testing dataset would rise if the new sub-task contains similar but varied data compared with data in old sub-tasks, and the accuracy growth of the wide neural network model is highest.