Does fragile co-adaptation occur in small datasets?
Akbar Gumbira, Rajmund Kozuszek · 2018
Recent study using AlexNet architecture and ImageNet dataset has shown that the transferability of each layer in Convolutional Neural Network (CNN) can be quantified. One interesting finding from this study is that performance degradation when performing transfer learning without finetuning is not only caused by the specificity of the features, but also due to fragile co-adapted neurons between neighboring layers. This raises a question whether this phenomenon also occurs on smaller datasets and simpler network architectures. Using CIFAR-10 and three different CNN architectures, we reported that we have not seen this effect. The drop in performance is solely due to the specificity of the features learned for the source task.