Stacked denoising autoencoder and dropout together to prevent overfitting in deep neural network
Jianglin Liang, Ruifang Liu · 2015
Deep neural network has very strong nonlinear mapping capability, and with the increasing of the numbers of its layers and units of a given layer, it would has more powerful representation ability. However, it may cause very serious overfitting problem and slow down the training and testing procedure. Dropout is a simple and efficient way to prevent overfitting. We combine stacked denoising autoencoder and dropout together, then it has achieved better performance than singular dropout method, and has reduced time complexity during fine-tune phase. We pre-train the data with stacked denoising autoencoder, and to prevent units from co-adapting too much dropout is applied in the period of training. At test time, it approximates the effect of averaging the predictions of many networks by using a network architecture that shares the weights. We show the performance of this method on a common benchmark dataset MNIST.