AmbigQA: Answering Ambiguous Open-domain Questions
Sewon Min, Julian Michael, Hannaneh Hajishirzi, Luke Zettlemoyer · 2020
Ambiguity is inherent to open-domain question answering; especially when exploring new topics, it can be difficult to ask questions that have a single, unambiguous answer.In this paper, we introduce AMBIGQA, a new open-domain question answering task which involves finding every plausible answer, and then rewriting the question for each one to resolve the ambiguity.To study this task, we construct AMBIGNQ, a dataset covering 14,042 questions from NQ-OPEN, an existing opendomain QA benchmark.We find that over half of the questions in NQ-OPEN are ambiguous, with diverse sources of ambiguity such as event and entity references.We also present strong baseline models for AMBIGQA which we show benefit from weakly supervised learning that incorporates NQ-OPEN, strongly suggesting our new task and data will support significant future research effort.Our data and baselines are available at https://nlp.cs. washington.edu/ambigqa.Type Example Event references (39%) What season does meredith and derek get married in grey's anatomy?Q: In what season do Meredith and Derek get informally married in Grey's Anatomy? / A: Season 5 Q: In what season do Meredith and Derek get legally married in Grey's Anatomy? / A: Season 7 Properties (27%) How many episode in seven deadly sins season 2? Q: How many episodes were there in seven deadly sins season 2, not including the OVA episode?/ A: 25 Q: How many episodes were there in seven deadly sins season 2, including the OVA episode?/ A: 26 Entity references (23%) How many sacks does clay matthews have in his career?Q: How many sacks does Clay Matthews Jr. have in his career?/ A: 69.5 Q: How many sacks does Clay Matthews III have in his career?/ A: 91.5 Answer types (16%) Who sings the song what a beautiful name it is?Q: Which group sings the song what a beautiful name it is?/ A: Hillsong Live Q: Who is the lead singer of the song what a beautiful name it is?/ A: Brooke Ligertwood