A CNN-Transformer hybrid U-shaped network
Pengzhi Wang, Yu Ma · 2024
Currently, convolutional neural network-based and Transformer-based U-shaped structures are applied to various medical image segmentation tasks. The former can efficiently learn local information of images while requiring much more image-specific inductive biases inherent to convolution operation. The latter can effectively capture long range dependency at different feature scales using self-attention, whereas it typically encounters the challenges. To solve this problem, a hybrid U-shaped network is proposed that combines the advantages of CNN and Transformer modes. Specifically, the network uses a bi-layer routing attention module as the core block, introduces a channel-spatial attention module using convolution operations to form a skip connection, and combines it into a U-shaped hierarchical encoder and decoder structure. It reduces the computational complexity while capturing global semantic information and minimizing the loss of local spatial information. Extensive experiments on two public fundus image datasets demonstrate that our proposed approach surpasses other state-of-the-art methods.