MedThink: A Rationale-Guided Framework for Explaining Medical Visual Question Answering
Xiaotang Gai, Chenyi Zhou, Jiaxiang Liu, Yang Feng, Jian Hua Wu, Zuozhu Liu · 2025
Medical Visual Question Answering (Med-VQA), which offers language responses to image-based medical inquiries, represents a challenging task and significant advancement in healthcare.It assists medical experts to swiftly interpret medical images, thereby enabling faster and more accurate diagnoses.However, the model interpretability and transparency of existing Med-VQA solutions are often limited, posing challenges in understanding their decision-making processes.To address this issue, we devise a semi-automated annotation process to streamline data preparation and build new benchmark Med-VQA datasets R-RAD, R-SLAKE and R-Path.These datasets provide intermediate medical decisionmaking rationales generated by multimodal large language models and human annotations for question-answering pairs in existing Med-VQA datasets, i.e., VQA-RAD, SLAKE and PathVQA.Moreover, we design a novel framework, MedThink, which finetunes lightweight pretrained generative models by incorporating medical decision-making rationales.Med-Think includes three distinct strategies to generate decision outcomes and corresponding rationales, clearly showcasing the medical decision-making process during reasoning.Our comprehensive experiments show that our method achieves an accuracy of 83.5% on R-RAD, 86.3% on R-SLAKE and 87.2% on R-Path.These results significantly exceed those of existing state-of-the-art models with comparable parameters.