Enhanced Breast Cancer Detection in Ultrasound Images Using LoRA-Fine-Tuned Florence-2 Model
Aarushi Agrawal, Harleen Kaur Bagga, Richa Khandelwal · 2025
Breast cancer remains a significant cause of mortality among women globally, where early and precise diagnosis is crucial for improving patient outcomes and lowering mortality rates. Although traditional diagnostic techniques like mammography and biopsy are effective, they have limitations regarding invasiveness, accessibility, and cost. Ultrasound imaging has emerged as a valuable non-invasive method for breast cancer detection, providing greater accessibility and minimizing risks. However, accurate interpretation of ultrasound images is often challenging due to issues like low contrast and noise. In this study, we introduce an innovative deep learning-based approach for breast cancer detection using ultrasound imaging by fine-tuning the Florence-2 base model with Low-Rank Adaptation (LoRA). This method leverages transfer learning, enabling the model to learn efficiently from pre-trained features while minimizing the need for additional computational resources during fine-tuning. LoRA enhances the fine-tuning process by introducing low-rank parameter updates, which improves the model's generalization capabilities without leading to overfitting. Our approach classifies breast ultrasound images into three categories: normal, benign, and malignant, thereby streamlining the diagnostic workflow. The model achieved a mean Average Precision (mAP) of 0.53 at an Intersection over Union (IoU) threshold of 0.5, demonstrating its robustness in differentiating various breast tissue types. This level of accuracy indicates the potential of the model to be utilized in clinical settings, offering a cost-effective and scalable solution for early breast cancer diagnosis. Future work will focus on further optimizing the model and validating it on larger, more diverse datasets to enhance its clinical applicability.