Toward Inclusive Healthcare: An LLM-Based Multimodal Chatbot for Preliminary Diagnosis
Ishita Agarwal, V. Sakthivel, P Prakash · IEEE Access · 2025
This paper presents the design and development of a multimodal medical chatbot that leverages Gemini-2.0-Flash Model alongside a novel Retrieval-Augmented Generation (RAG) architecture to support preliminary medical diagnosis and recommendations. The system integrates textual prompt analysis and medical image interpretation, aiming to improve healthcare accessibility, particularly for underserved populations. Focused on data-rich medical conditions, the chatbot generates reliable diagnostic insights based on natural language inputs and/or medical images, requiring minimal user expertise. The proposed RAG-based architecture incorporates a curated medical knowledge base and structured retrieval mechanisms, significantly reducing hallucinations and enhancing response credibility compared to direct Large Language Model (LLM) querying. By demonstrating the efficacy of multimodal reasoning in conjunction with structured retrieval, this work paves the way for more accessible, accurate, and scalable AI-driven health support systems.