Leveraging CNN-Transformer Hybrid Architecture for Accurate Breast Tumor Segmentation in Mammograms
Mudassar Ali, Muhammad Zeeshan Tahir, Iqra Mariam, Khadija Nawaz, Tariq Mahmood, Samia Allaoua Chelloug · 2025
Breast cancer detection through mammograms is very crucial for early diagnosis, and tumor segmentation is an important step in treatment planning. This paper proposes a novel approach by combining CNNs with transformers, which we call the Hybrid CNN + Transformer model, for the task of breast tumor segmentation. It leverages the strengths of CNN s in local feature extraction to capture fine details, such as tumor boundaries, and combines them with transformers that model long-range dependencies and contextual relationships across images for the exact delineation of tumor regions in complex tissue structures. Our methodology will be tested on the high-quality mammog-raphy dataset annotated with both benign and malignant lesions, the so-called INbreast dataset. Herein, the CNN backbone-first for instance, ResNet or EfficientNet-extracts the input mammo-gram hierarchically; features are fed into a transformer layer in order to capture global dependencies, after which the final segmentation mask is generated by using a U-Net style decoder for a pixel-wise tumor classification. The segmentation accuracies of our models demonstrated the significant improvement achieved in tumor segmentation, using Hybrid CNN + Transformer when compared with tra-ditional CNN-based models on challenging tumor boundaries and complex tissue patterns. An accurate, robust, completely automatic method for tumor segmentation will thus enable improved diagnostic feasibility on clinical grounds. Furthermore, the combination of learning at both local and global features improved generalizability to previously unseen mammogram images and therefore strengthened clinical applicability.