RAGNER: Improving Performance of LLMs using RAG and Specialized Domain Specific Entity Recognition
Rumit Pingleshwar Gore, Karukriti Kaushik Ghosh, Chiranjib Sur · 2025
Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding natural language. However, challenges still persist in their ability to generate contextually relevant responses and accurately comprehend nuanced information. This work is based on an idea that explores integrating two powerful techniques, Retrieval Augmented Generation (RAG) architecture and Specialized Named Entity Recognition (sNER), to enhance the performance of LLMs to retrieve critical information. The RAG architecture enables language models to retrieve relevant passages from a knowledge source before generating responses, thereby improving context awareness and coherence. Additionally, incorporating specialized NER into the model facilitates better identification and understanding of named entities within the text, leading to more accurate and finer-grained responses. These entities are organization-specific and were unexplored or uncaptured in previous works, mainly because of the limitations of the LLMs to adapt and comprehend specific domain knowledge. Experimental results have significantly improved quantitative and qualitative metrics and qualitative assessments of contextual relevance and entity understanding. The findings of this study contribute to advancing the capabilities of LLMs in understanding and generating language, with implications for a wide range of applications, including information retrieval systems.