Large language model based framework for knowledgebase coverage and correctness using chatbot and human feedback
Unmesh Shukla, Sanjeev Singh, Amit Pundir, Geetika Jain Saxena · 2023
There is a recent surge in the use of large language models (LLMs) to query documents pertaining to a restricted domain. This study formulated a comprehensive framework that harnesses the potential of these models and expertise of multiple human experts to enhance the coverage and correctness of information in a domain-restricted knowledgebase. This framework enables multiple domain-specific experts within a larger organization to improve a knowledgebase by querying it using a chatbot. The open-source LLMs analyzed for the proposed framework utilized a maximum of 32 GB GPU memory. They were fine-tuned using a precision reduction strategy dictated by QLoRA. To the best of our knowledge, this is the first study that applies LLMs for knowledgebase coverage.