Graph-Based Grounding in a Conversational Clinical Decision Support System
Samuel Barham, Michael T. Kelbaugh, Arun Reddy, Devin Ramsden, Chase Chandler, Adam Tobin-Williams, Ben Shalom Elhadad, Georgia R. Cooper, Kari Alexander · 2024
We introduce a novel method of grounding LLM conversational systems in technical domains, and demonstrate the technique’s use in a prototype clinical decision support system (CDSS). Our CDSS is designed to talk to warfighters with little to no formal medical training and walk them through properly sequenced diagnoses and mitigations. Because LLMs have been found to suffer from hallucination, we equip the LLM controlling the dialogue with access to a graph-navigation mechanism that allows it to maintain an index, or position, in a directed acyclic graph (DAG) that represents correct clinical sequencing. We evaluate our system on several scenarios drawn from emergency battlefield medical care, and show that a graph-grounding-enhanced ChatGPT 3.5 outperforms a GPT-4 baseline.