SparseConnect: regularising CNNs on fully connected layers
Qi Xu, Gang Pan · Electronics Letters · 2017
Deep convolutional neural networks (CNNs) have achieved unprecedented success in many domains. The numerous parameters allow CNNs to learn complex features, but also tend to hinder generalisation by over‐fitting training data. Despite many previously proposed regularisation methods, over‐fitting is one of many problems in training a robust CNN. Among many factors that may lead to over‐fitting, the numerous parameters of fully connected layers (FCLs) of a typical CNN are a contributor to the over‐fitting problem. The authors propose SparseConnect, which alleviates over‐fitting by sparsifying connections to FCLs. Experimental results on three benchmark datasets MNIST and CIFAR10 show SparseConnect outperforms several state‐of‐the‐art regularisation methods.