Optimizing Test-Time Query Representations for Dense Retrieval
Mujeen Sung, Jungsoo Park, Jaewoo Kang, Danqi Chen, Jinhyuk Lee · 2023
Recent developments of dense retrieval rely on quality representations of queries and contexts from pre-trained query and context encoders.In this paper, we introduce TOUR (Test-Time Optimization of Query Representations), which further optimizes instance-level query representations guided by signals from testtime retrieval results.We leverage a crossencoder re-ranker to provide fine-grained pseudo labels over retrieval results and iteratively optimize query representations with gradient descent.Our theoretical analysis reveals that TOUR can be viewed as a generalization of the classical Rocchio algorithm for pseudo relevance feedback, and we present two variants that leverage pseudo-labels as hard binary or soft continuous labels.We first apply TOUR on phrase retrieval with our proposed phrase re-ranker, and also evaluate its effectiveness on passage retrieval with an off-the-shelf reranker.TOUR greatly improves end-to-end open-domain question answering accuracy, as well as passage retrieval performance.TOUR also consistently improves direct re-ranking by up to 2.0% while running 1.3-2.4×faster with an efficient implementation. 1