A Modular Retrieval-Augmented Conversational AI Chatbot System with Integrated Recommender Engine Using Local LLMs
Narendra Reddy, Manas Ranjan Patra, Brojo Kishore Mishra · Cureus Journal of Computer Science. · 2025
Recent advancements in large language models (LLMs) and natural language processing (NLP) have enabled more capable conversational AI systems. However, challenges such as hallucinations, limited context retention, and poor personalization persist. This study presents a modular, privacy-preserving conversational AI framework that operates entirely on local infrastructure to address these limitations. The proposed system integrates a Retrieval-Augmented Generation (RAG) architecture using LangChain, with semantic search powered by FAISS vector indexing. Text preprocessing is performed via NLP pipelines, and document embeddings are generated using locally hosted models through the Ollama runtime. At query time, relevant content is retrieved from user-provided documents and web inputs to ground responses in factual context. A hybrid recommender system further enhances personalization by suggesting related content based on semantic similarity and interaction history. The framework includes both a Flask-based API and a command-line interface, offering deployment flexibility. Evaluation demonstrates the system’s ability to generate accurate, contextually grounded responses with low computational overhead. By avoiding reliance on cloud services, the framework ensures data privacy without compromising performance. This research contributes a scalable blueprint for secure, responsive conversational AI. Future work will focus on multi-document handling, real-time interaction, and deeper personalization to improve adaptability and user engagement.