Enhancing Cancer Detection with Fine-Tuned Large Language Models: A Comparative Study on Low-Rank Adaptation
Amine Bechar, Youssef Elmir, Yassine Himeur, Rafik Medjoudj, Abbes Amira · 2024
Large Language Models (LLMs) have been utilized extensively for cancer detection and diagnosis, benefiting from the vast textual data available in the medical field. However, these models often lack specific training on cancer-related data, which can limit their effectiveness in specialized medical contexts. Traditional methods typically deploy LLMs directly for diagnosis without incorporating domain-specific expertise, potentially compromising outcome reliability. This paper presents a comparative study focusing on the application of Low-Rank Adaptation (LoRA) to fine-tune LLMs for cancer-related tasks. LoRA modifies the self-attention and feed-forward layers of transformer architectures with low-rank matrices, allowing for specialized adaptation with fewer parameters. A general LLM was fine-tuned using LoRA on a dataset derived from four annotated books on breast cancer. The performance of this LoRA-enhanced model was compared against several baseline LLMs fine-tuned through traditional methods. It was found that the LoRA-fine-tuned Biomstral-7B demonstrated the best training loss of 0.91025 and validation loss of 0.912722 scores, indicating enhanced integration of domain-specific knowledge. The potential of adaptive fine-tuning techniques like LoRA in specialized applications is highlighted, suggesting further exploration into their effectiveness across various complex domains requiring expert knowledge. Further research is encouraged to assess such approaches’ broader applicability and impact in diverse AI applications.