Query-Efficient Two-Phase Reinforcement Learning Framework for Black-Box Adversarial Attacks
Zerou Ma, Tao Feng · Symmetry · 2025
Generating adversarial examples under black-box settings poses significant challenges due to the inaccessibility of internal model information. This complexity is further exacerbated when attempting to achieve a balance between the attack success rate and perceptual quality. In this paper, we propose QTRL, a query-efficient two-phase reinforcement learning framework for generating high-quality black-box adversarial examples. Unlike existing approaches that treat adversarial generation as a single-step optimization problem, QTRL introduces a progressive two-phase learning strategy. The initial phase focuses on training the agent to develop effective adversarial strategies, while the second phase refines the perturbations to improve visual quality without sacrificing attack performance. To compensate for the unavailability of gradient information inherent in black-box settings, QTRL designs distinct reward functions for the two phases: the first prioritizes attack success, whereas the second incorporates perceptual similarity metrics to guide refinement. Furthermore, a hard sample mining mechanism is introduced to revisit previously failed attacks, significantly enhancing the robustness and generalization capabilities of the learned policy. Experimental results on the MNIST and CIFAR-10 datasets demonstrate that QTRL achieves attack success rates comparable to those of state-of-the-art methods while substantially reducing query overhead, offering a practical and extensible solution for adversarial research in black-box scenarios.