DropWeak: A novel regularization method of neural networks
Anas El Korchi, Youssf Ghanou · Procedia Computer Science · 2018
We Introduce DropWeak a novel regularization method of neural networks. The training of a neural network with Dropout [1] [4] [5] consist of setting to zero a randomly subset of activations within each layer, in the other side the training of a neural network with DropConnect [2] consist of setting to zero a randomly subset of weights within each layer. DropWeak instead sets to zero all the weak weights within all the layers of a neural network. We study all methods Dropout, DropConnect and DropWeak. We then evaluate DropWeak on a range of datasets, comparing to Dropout, and show results on MNIST image recognition dataset.