Gradient-based Adversarial Attacks against Text Transformers
Chuan Jun Guo, Alexandre Sablayrolles, Hervé Jeǵou, Douwe Kiela · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
We propose the first general-purpose gradientbased adversarial attack against transformer models.Instead of searching for a single adversarial example, we search for a distribution of adversarial examples parameterized by a continuous-valued matrix, hence enabling gradient-based optimization.We empirically demonstrate that our white-box attack attains state-of-the-art attack performance on a variety of natural language tasks, outperforming prior work in terms of adversarial success rate with matching imperceptibility as per automated and human evaluation.Furthermore, we show that a powerful black-box transfer attack, enabled by sampling from the adversarial distribution, matches or exceeds existing methods, while only requiring hard-label outputs.