Towards improved breast cancer detection on digital mammograms using local self-attention-based transformer
Han M. Chen, Anne Louise Martel · 2024
Deep-learning-based models have been proposed as an automated second reader for mammograms that might help reduce radiologists’ workload and improve screening accuracy. However, the inherent traits of mammograms, characterized by significantly higher resolutions and smaller regions of interest (ROIs) in comparison to natural images, impose constraints on the adaptability of deep neural networks that are well-suited for natural image analysis to the domain of mammogram analysis. In this work, we propose a novel neural network to effectively detect breast cancer on screening mammograms and address the above issues. First, we use a local self-attention-based Swin Transformer as the backbone to select the most informative patch regions from the whole mammogram. We then utilize a second CNN based network to further extract the fine-grained features of the selected patches. Finally, we employ a fusion module that aggregates global and local information to make a prediction. The final loss function is the combination of the prediction from both the transformer and CNN modules. With local self-attention and a hierarchical structure, our backbone can effectively model the relationships between ROIs (e.g., masses or micro-calcifications) of different sizes and their surrounding tissues. Thus introduces meaningful contextual information for robust feature extraction. The experimental results show that our model can achieve state-of-the-art performance, in terms of classification AUC of 0.856 on a public mammogram dataset.