Deep convolutional networks do not classify based on global object shape
Nicholas A. Baker, Hongjing Lu, Gennady Erlikhman, Philip J. Kellman · PLoS Computational Biology · 2018
Deep convolutional networks (DCNNs) are achieving previously unseen performance in object classification, raising questions about whether DCNNs operate similarly to human vision.In biological vision, shape is arguably the most important cue for recognition.We tested the role of shape information in DCNNs trained to recognize objects.In Experiment 1, we presented a trained DCNN with object silhouettes that preserved overall shape but were filled with surface texture taken from other objects.Shape cues appeared to play some role in the classification of artifacts, but little or none for animals.In Experiments 2-4, DCNNs showed no ability to classify glass figurines or outlines but correctly classified some silhouettes.Aspects of these results led us to hypothesize that DCNNs do not distinguish object's bounding contours from other edges, and that DCNNs access some local shape features, but not global shape.In Experiment 5, we tested this hypothesis with displays that preserved local features but disrupted global shape, and vice versa.With disrupted global shape, which reduced human accuracy to 28%, DCNNs gave the same classification labels as with ordinary shapes.Conversely, local contour changes eliminated accurate DCNN classification but caused no difficulty for human observers.These results provide evidence that DCNNs have access to some local shape information in the form of local edge relations, but they have no access to global object shapes. Author summary"Deep learning" systems-specifically, deep convolutional neural networks (DCNNs)have recently achieved near human levels of performance in object recognition tasks.It has been suggested that the processing in these systems may model or explain object perception abilities in biological vision.For humans, shape is the most important cue for recognizing objects.We tested whether deep convolutional neural networks trained to recognize objects make use of object shape.Our findings indicate that other cues, such as surface texture, play a larger role in deep network classification than in human recognition.Most crucially, we show that deep learning systems have no sensitivity to the overall shape of an object.Whereas deep learning systems can access some local shape features,