Vision Transformers with Efficient LoRA Finetuning for Breast Cancer Classification
Tawfik Ezat Mousa, Ramzi Zouari, Mouna Baklouti · 2024
The diagnosis of breast cancer remains a very interesting area of research because of its high mortality rate among women worldwide. During the last decade, several studies have focused on the development of early detection systems for breast cancer based on new advances in deep learning models. In this context, we presented a new system of breast cancer detection using Vision Transformers (ViT) architecture. Due to the large number of trainable parameters in ViT, we have incorporated the Low-Rank Adaptation (LoRA) mechanism as an efficient fine-tuning method to significantly reduce the training calculation costs while maintaining the model performance. This reduction in the number of trainable parameters results in a significant reduction in memory consumption and training time. The experiments were conducted on two different histopathological breast images databases, and demonstrate the effectiveness of the proposed methodology in terms of accuracy, memory consumption and training process.