Can we consult large language models for antibiotic prophylaxis?
Rana TURUNÇ OĞUZMAN · International Dental Journal · 2024
The aim of this study is to determine the reliability of large language models (LLMs) on antibiotic prophylaxis and thereby understand whether the loss of sessions due to medical consultation and financial loss for the patient can be avoided. ChatGPT and Gemini LLMs were asked 170 questions each regarding the necessity of antibiotic prophylaxis for various dental procedures in patients with different cardiovascular problems, and 4 questions each about what the antibiotic regimen should be, according to the 2021 American Heart Association guidelines for the prevention of infective endocarditis. The correct answer rates of the LLMs were calculated, and the difference between the two LLMs was evaluated using the chi-square test (p<0.05). It was found that Gemini (91%) provided significantly more accurate answers than ChatGPT (61%) regarding whether prophylaxis should be administered for various dental procedures in patients with different cardiovascular problems. Regarding how antibiotic regimen should be in necessary cases, ChatGPT (75%) provided significantly more accurate answers than Gemini (0%). Although LLMs are now quite advanced, they still have error margins, and referring to them for vital issues like infective endocarditis is not yet the right approach. Moreover, LLMs do not take responsibility in this regard and direct the dentist to medical consultation.