Adversarial example‐based test case generation for black‐box speech recognition systems
Hanbo Cai, Pengcheng Zhang, Hai Ying Dong, Lars Grunske, Shunhui Ji, Tianhao Yuan · Software Testing Verification and Reliability · 2023
Abstract Test case generation techniques based on adversarial examples are commonly used to enhance the reliability and robustness of image‐based and text‐based machine learning applications. However, efficient techniques for speech recognition systems are still absent. This paper proposes a family of methods that generate targeted adversarial examples for speech recognition systems. All are based on thefirefly algorithm (F), and are enhanced withgaussmutations and / orgradientestimation (F‐GM, F‐GE, F‐GMGE) to fit the specific problem of targeted adversarial test case generation. We conduct an experimental evaluation on three different types of speech datasets, includingGoogle Command,Common VoiceandLibriSpeech. In addition, we recruit volunteers to evaluate the performance of the adversarial examples. The experimental results show that, compared with existing approaches, these approaches can effectively improve the success rate of the targeted adversarial example generation. The code is publicly available at https://github.com/HanboCai/FGMGE .