Conversational Agent for Medical Question-Answering Using RAG and LLM
La Ode Muhammad Yudhy Prayitno, Annisa Nurfadilah, Septiyani Bayu Saudi, Widya Dwi Tsunami, Adha Mashur Sajiah · Journal of Artificial Intelligence and Engineering Applications (JAIEA) · 2025
This study analyzes the application of the RAG concept alongside an LLM in the context of PubMed QA data to augment question-answering capabilities in the medical context. For answering questions relevant to private healthcare institutions, the Mistral 7B model was utilized. To limit hallucinations, an embedding model was used for document indexing, ensuring that the LLM answers based on the provided context information. The analysis was conducted using five embedding models, two of which are specialized medical models, PubMedBERT-base and BioLORD-2023, as well as three general models, GIST-large-Embedding-v0, blade-embed-kd, and all-MiniLM-L6-v2. As the results showed, general models performed better than domain specific models, especially GIST-large-Embedding-v0 and b1ade-embed-kd, which underscores the dominance of general-purpose training datasets in terms of fundamental semantic retrieval, even in medical domains. The outcome of this research study demonstrates that applying RAG and LLM locally can safeguard privacy while still responding to medical queries with appropriate precision, thus establishing a foundation for a dependable medical question-answering system.