Training Deep Neural Networks with Adversarially Augmented Features for Small-scale Training Datasets
Masato Ishii, Atsushi Sato · 2019
In this paper, we propose a novel method to train deep neural networks for small-scale training dataset. Since we focus on the situation in which the training dataset is limited, increasing the number of training data by data augmentation is the most straight-forward solution. Instead of augmenting data at an input layer as in typical data augmentation, we adversarially augment features at hidden layer by adding small perturbations to the original features extracted from training data. We call the augmented features as adversarial features. To effectively avoid overfitting of the trained network, the perturbation is designed to be adversarial, which means that they are designed to significantly change the output of the network. Moreover, to induce the adversarial feature to be more reasonable as real data, we adopt a coefficient layer to constrain the adversarial feature to be represented by linear combination of the original features. Experimental results on several benchmark datasets show that our method substantially improves the performance of the deep neural network.