A New Cyclic Spatial Attention Module for Convolutional Neural Networks
Daihui Li, Zeng Shangyou, Wenhui Li, Yang Lei · 2019
The mechanism of attention mechanisms is ubiquitous in the biological world. In order to make the convolutional neural network improve the model recognition ability and robustness, this paper proposes a lightweight and efficient Cyclic Spatial Attention Module (CSAM) for convolutional neural networks. CSAM is dedicated to making neural networks more effective in screening useful features. It generates a attention map by spatial information filtering according to the output features of the current convolutional layer. Then the attention maps are multiplied to the input feature maps for adaptive feature refinement, and combined with the residual and batch normalization techniques to return to the current convolution layer. CSAM is lightweight and can be directly added to traditional convolutional networks for end-to-end training. We evaluate network performance on classical benchmark datasets cifar10 and GTSRB. Experiments show that this module can effectively improve the performance of convolutional neural networks. And CSAM has obvious advantages over the popular attention modules for convolutional neural networks in recent years.