Keyword vs Semantic Search for Retrieval-Augmented Generation: A Survey
Theo Chihaia, Radu‐Ioan Ciobanu · 2025
The retrieval-augmented generation (RAG) paradigm combines a large language model (LLM) with a content retrieval engine to simplify retrieval and presentation in a knowledge base. Based on the user query, the content retrieval engine fetches the most relevant content and feeds it into a specially engineered prompt, with guidelines on how to summarize and present the retrieved information. This prompt is processed by the LLM to provide an answer tailored to the use case. Keyword-based search and semantic search with embeddings are the core search methods employed by the retrieval engine. Keyword-based search is computationally light, explainable, and works well when the query contains the same keywords found within the content. Semantic search uses machine learning to compute embeddings, making it more computationally costly. Its main advantage is coverage for queries which do not exactly contain the keywords, but are semantically similar. A hybrid weighted approach might improve retrieval results. A good retrieval system is a key component in RAG, as more relevant content improves usability and user experience. Thus, in this work, we discuss the strong and weak point of the approaches, as well as the practical considerations of using them, highlighting open questions and proposing a hybrid search paradigm as further research.