Lightweight Dual Sparse Self-Attention Network for Image Super-Resolution
Yike Li, Ming Hao · 2023
In recent years, the Transformer-based methods show great potential in dealing with image super-resolution task. Inspired by it, we present an innovative lightweight dual sparse self-attention network (DSSANet). Since the shifted window Transformer methods limits the horizon of self-attention computation, we develop a spatial sparse multihead attention (SSMA) module, which is capable of expanding the receptive field and efficiently extracting local features while reducing the computation. Meanwhile, in order to better capture non-local features, we introduce a channel dimensional self-attention mechanism and design a channel sparse multihead attention (CSMA) module. These two modules are integrated into a dual sparse attention block (DSAB), enabling the model to utilise local and non-local information more efficiently. In addition, we incorporate the DSAB into a residual learning framework to form a residual sparse transformer group (RSTG) and use the RSTG as the basis to construct an end-to-end deep CNN model, DSSANet. Experiments demonstrate that our proposed DSSANet has superior performance while reducing network parameters and computation.