ACCELERATING THE TRAINING PROCESS OF CONVOLUTIONAL NEURAL NETWORKS FOR CLASSIFICATION BY DROPPING TRAINING SAMPLES OUT
Naisen Yang, Hong Tang, Jianwei Yue, Xin Yang, Zhihua Xu · 2018
Stochastic gradient descent and other adaptive optimization methods such as RMSprop, and Adam have been proved effective for training deep neural networks [1], [2]. Within each epoch of these methods, the whole training set is involved. In general, large training datasets have data redundancy. In this paper, we investigate an algorithm that reduce the training time of CNN by dropping certain samples out. Thus, it is called DropSample. This method can be viewed as a special type of truncated cross-entropy loss with a finite margin. We design experiments on several datasets to demonstrate the effects of acceleration. The results show that this method could decrease the training time of multilayer perceptrons (MLPs) and convolutional neural networks (CNNs) significantly. Despite reduced number of training samples, the accuracies of networks are similar, or even better.