Enhancing Retrieval-Augmented Generation for Text Completion Through Query Selection
Idan Pogrebinsky, David Carmel, Oren Kurland · 2025
In a Retrieval-Augmented Generation (RAG) system designed for the fundamental task of text completion, a query is derived from the prompt of a large language model (LLM) and is issued to an external resource. The retrieved content is then incorporated into the prompt to enhance text completion. Since prompts can be considerably long, it is common practice to use a short suffix of the prompt as the query. We empirically show, using a suite of oracle experiments, that this approach is often suboptimal with respect to other choices of a query from the prompt. This finding gives rise to a novel research challenge: identifying the optimal query for RAG in this setting. As an initial study, we propose a few query selection methods, some of which yield statistically significant improvements over using the prompt's suffix as a query.