Automated Failure Detection in Computer Vision Systems
Haoxin Yan, A. Achkar, Akshaya Kumar Mishra, K. Naik · Journal of Computational Vision and Imaging Systems · 2016
Human validation of computer vision systems increase their operatingcosts and limits their scale. Automated failure detection canmitigate these constraints and is thus of great importance to thecomputer vision industry. Here, we apply a deep neural networkto detect computer vision failures on vehicle detection tasks. Theproposed model is a convolution neural network that estimates theoutput quality of a vehicle detector. We train the network to learnto estimate a pixel-level F1 score between the vehicle detector andhuman annotated data. The model generalizes well to testing data,providing a mechanism for identifying detection failures.