Query Encoder Distillation via Embedding Alignment is a Strong Baseline Method to Boost Dense Retriever Online Efficiency
Yuxuan Wang, Lyu Hong · 2023
The information retrieval community has made significant progress in improving the efficiency of Dual Encoder (DE) dense passage retrieval systems, making them suitable for latencysensitive settings.However, many proposed procedures are often too complex or resourceintensive, which makes it difficult for practitioners to adopt them or identify sources of empirical gains.Therefore, in this work, we propose a trivially simple recipe to serve as a baseline method for boosting the efficiency of DE retrievers leveraging an asymmetric architecture.Our results demonstrate that even a 2-layer, BERT-based query encoder can still retain 92.5% of the full DE performance on the BEIR benchmark via unsupervised distillation and proper student initialization.We hope that our findings will encourage the community to re-evaluate the trade-offs between method complexity and performance improvements.