Enhancing Chinese Medical Diagnostic Chatbot through Supervised Fine-Tuning of Large Language Models
Baha Ihnaini, Yawen Huang, Lianglin Li, Jiayi Wei, Shengyi Qi · 2024
This paper aims to fine-tune and evaluate the large language model on the Chinese medical dialogue datasets, establish a highly accurate Chinese diagnosis model, solve the problems of low accuracy and poor reliability of Chinese medical diagnosis chatbots when processing complex Chinese input, and improve the ability of Chinese medical diagnosis chatbots to deal with different complex situations. To provide patients with better and more convenient health consultation and treatment programs. The project involves training the model on a huge medical dataset to learn basic medical diagnosis knowledge, and then fine-tuning the model on a Chinese medical dataset to enhance the model's understanding of complex Chinese input, prevent overfitting, improve diagnostic accuracy, and thus optimize the chatbot's performance. In this project, ROUGE-1, ROUGE-2, ROUGE-L and BLEU-4 evaluation methods were used to evaluate the accuracy of LLaMA3-8B-Chinese-Chat, ChineseLLaMA2-7B-Chat, Gemma-7B-Chat and Qwen 1.5-7B-Chat. The results show that Qwen1.5-7B-Chat performs well on several indexes and has high medical conversation processing capability. This project aims to provide early health consultation for patients by improving the performance of Chinese-language medical diagnosis chatbots and providing better treatment opportunities for patients in medically underdeveloped areas.