Pixel Attention Activation Function for Single Image Super-Resolution
Xinchen Gao · 2023
In recent years, the development of deep neural networks leads significant advances in image restoration tasks, especially in image super-resolution. The widely used activation functions in deep super-resolution models like ReLU and MTLU are suboptimal. Both ReLU and MTLU provide static scaling factors for the input image feature. Therefore, in this paper, motivated by the channel attention and spatial attention mechanism, we propose a simple and effective Pixel Attention Activation Function (PAAF) for image super-resolution. Our method provides dynamic element-wise (i.e., spatial-wise and channel-wise) scaling factors for input image features. Our PAAF obtains a specific scaling factor for every single pixel in input features and can capture rich spatial and channel-wise interactions. The experiments on several standard benchmarks demonstrate our proposed method is superior to other activation functions for super-resolution.