ReQA: An Evaluation for End-to-End Answer Retrieval Models
Amin Ahmad, Noah Constant, Yinfei Yang, Daniel Cer · 2019
Popular QA benchmarks like SQuAD have driven progress on the task of identifying answer spans within a specific passage, with models now surpassing human performance.However, retrieving relevant answers from a huge corpus of documents is still a challenging problem, and places different requirements on the model architecture.There is growing interest in developing scalable answer retrieval models trained end-to-end, bypassing the typical document retrieval step.In this paper, we introduce Retrieval Question-Answering (ReQA), a benchmark for evaluating large-scale sentence-level answer retrieval models.We establish baselines using both neural encoding models as well as classical information retrieval techniques.We release our evaluation code to encourage further work on this challenging task.