Attentive Manifold Mixup for Model Robustness
Zhengbo Zhou, Jianfei Yang · 2022
The robustness of deep neural networks becomes more and more significant since the performance of models degrades heavily in real life. The main reason behind that is discrepancy between training and testing distribution. Many state-of-art methods have been proposed to improve model robustness. However, previous methods conduct data augmentation at the input phase, but they only utilize random weights for augmentation, failing to learn an optimal weight combination. Inspired by manifold mixup, Attentive Manifold Mixup network integrates the feature maps of different augmented images and retrieves a learnable ratio between those feature maps by the attention module. We conduct extensive experiments on Cifar-10-C, which is manually added corruptions of noise, blur, weather and digit. AMM shows superior results on public datasets, improving model robustness. We also visualize the attention weights for our methods, providing some underlying interpretation behind the proposed AMM.