A Retrieval-Augmented Dialogue Framework for Multimodal Medical Consultation
Xinsheng S. Zhang, Yi Zhang · 2024
The medical consultation system based on artificial intelligence technology aims to discover users' illnesses and provide accurate treatment recommendations through interactive dialogue consultation, in order to timely and effectively solve users' problems. We propose a retrieval-augmented large language model framework for multimodal medical consultation. This framework combines generative large language model and information retrieval technique to enhance the responsiveness of language model by extracting relevant information from large-scale medical databases. Specifically, our framework can not only provide intelligent dialogue based on language models, but also integrate the latest medical knowledge to ensure the professionalism and reliability of consulting content by introducing a retrieval-augmented mechanism. The experimental results show that the framework is significantly superior to traditional methods in terms of accuracy and response time, demonstrating its enormous potential in practical medical applications.