LLM-Based Chatbot for the Indonesia University IT Service Desk: Integrating DeepSeek-v3 API and a RAG Approach
Rahmat Pratami, Muhammad Lutfi Ruhallah, Alfian Akbar Gozali · 2025
The growing need for streamlined IT service management in higher education has highlighted the limitations of traditional helpdesk models, which depend heavily on manual processes and human operators. These legacy systems often suffer from slow response times, accumulating service requests, and staff fatigue. To overcome these challenges, this study introduces an intelligent, LLM-powered chatbot that integrates the DeepSeek-v3 API with a Retrieval-Augmented Generation (RAG) strategy tailored to an Indonesian university’s IT service desk environment. Built upon a modular architecture, the proposed solution automates document ingestion with multi-format preprocessing, dynamic chunking, and embedding generation, which are stored in Supabase for efficient top-k contextual retrieval. When a user submits an inquiry, pertinent knowledge segments are retrieved in real time and synthesized into coherent, contextually appropriate responses by the DeepSeek-v3 language model. The system was evaluated through a comprehensive experimental framework, including functional and hardware benchmarking, performance testing, and a standardized User Experience Questionnaire (UEQ). Functional tests confirmed stable, 24/7 operation with over 98% retrieval accuracy, while performance benchmarking demonstrated an average response time of 2.1 ± 0.2 seconds—a 35% improvement over a legacy helpdesk baseline. User studies with 26 IT staff and students yielded a high UEQ score of 4.5 ± 0.3 (out of 5), reflecting strong satisfaction across pragmatic (efficiency, dependability) and hedonic (stimulation, novelty) dimensions. In addition, nine prompting strategies were compared via statistical analysis to optimize response quality. These findings demonstrate the viability and scalability of integrating LLMs with institutional IT infrastructures and offer practical guidance for deploying RAG-based chatbots in resource-constrained academic settings. By detailing implementation best practices and comparative evaluations, this work advances AI-driven automation in academic IT support and lays the groundwork for future enhancements in multilingual and low-resource environments.