The Effect of Combinatorial Coverage for Neurons on Fault Detection in Deep Neural Networks
Ziyuan Wang, Jinwu Guo, Yanshan Chen, Feiyan She · 2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C) · 2021
Many test adequacy metrics for deep neural net-works (DNNs) were proposed to measure the quality of the testing of deep learning system. The combinatorial coverage of neurons was proposed because it considers the influence of neurons in the same layer on neurons in the next layer. However, it is still inconclusive whether the combinatorial coverage of neurons is beneficial to the testing of DNNs. We conduct an empirical study on MNIST dataset to answer how the combinatorial coverage of neurons affects the fault detection in DNNs. The experimental results show a medium or strong correlation between the fixed-strength combinatorial coverage of neurons and the number of adversarial examples for DNNs. Such a results suggests that it is feasible to use the combinatorial coverage of neurons to guide the testing of DNNs.