Enhancing Large Language Models’ Utility for Medical Question-Answering: A Patient Health Question Summarization Approach
Nour Eddine Zekaoui, Siham Yousfi, Mounia Mikram, Maryem Rhanoui · 2023
Large language models (LLMs) offer tremendous potential for answering diverse questions and providing valuable insights. However, to maximize their utility, it is essential to formulate effective prompts or questions. This challenge is particularly pronounced when searching for health-related information inundates the web with questions from consumers about their health. In fact, patients with limited technical vocabulary and inadequate phrasing may struggle to submit questions that accurately convey their concerns, resulting in a significantly higher occurrence of false positives in answer generation. To address this pressing issue, we propose a solution for generating concise and understandable medical questions by summarizing consumer health questions (CHQs). Moreover, our method leverages state-of-the-art language models, has shown effective results. Specifically, we fine-tune a set of Transformer-based models, including Flan-T5, on low resources, using three different medical question summarization datasets. Furthermore, through ablation studies, we demonstrate how generative configuration choices and instruction fine-tuning can significantly impact the final results. Our best English model achieves scores of 54.32%, 38.08%, and 51.98% in terms of ROUGE-1, ROUGE-2, and ROUGE-L, respectively, outperforming exiting methods by 5.82 ROUGE-1 points and providing a potential means to craft good prompts.