Development and Evaluation of a Support Bot for College Activities and Placements Using Fine-Tuned Quantized Large Language Models
Vikas J. Nandeshwar, Sarvadnya Bhatlawande, Safalya Satpute, Shivanand Satao, Atharv Saste, Harsh Sathe, Saumya Gaikwad · 2024
Advancements in Natural Language Processing (NLP) and Large Language Models (LLMs) have significantly enhanced the ability of machines to understand and generate human language, enabling their application in diverse domains. This paper presents a detailed approach to the development of a support bot tailored to respond to inquiries about college activities and job placements, leveraging a fine-tuned and quantized FLAN-T5 base model. The research focuses on refining and optimizing this large language model (LLM) using advanced techniques such as LoRa to handle specific and detailed information pertinent to this field. The fine-tuned FLAN-T5 model is evaluated on performance metrics including BLEU score, ROUGE scores, METEOR score, and BERTScore. These metrics offer a thorough evaluation of the model's performance, assessing its predictive accuracy, response validity, alignment with reference texts, and semantic relevance. The results indicate that the fine-tuned FLAN-T5 model effectively generates precise and contextually relevant responses to user inquiries. This study provides a detailed methodology for customizing large language models to specialized informational contexts and delivers insights into enhancing automated support systems for targeted application areas.