Few-shot In-context Learning on Knowledge Base Question Answering
Tianle Li, Xueguang Ma, Alex Zhuang, 裕二 池谷, Yu Su, Wenhu Chen · 2023
Question answering over knowledge bases is considered a difficult problem due to the challenge of generalizing to a wide variety of possible natural language questions.Additionally, the heterogeneity of knowledge base schema items between different knowledge bases often necessitates specialized training for different knowledge base question-answering (KBQA) datasets.To handle questions over diverse KBQA datasets with a unified trainingfree framework, we propose KB-BINDER, which for the first time enables few-shot incontext learning over KBQA tasks.Firstly, KB-BINDER leverages large language models like Codex to generate logical forms as the draft for a specific question by imitating a few demonstrations.Secondly, KB-BINDER grounds on the knowledge base to bind the generated draft to an executable one with BM25 score matching.The experimental results on four public heterogeneous KBQA datasets show that KB-BINDER can achieve a strong performance with only a few in-context demonstrations.Especially on GraphQA and 3-hop MetaQA, KB-BINDER can even outperform the state-of-the-art trained models.On GrailQA and WebQSP, our model is also on par with other fully-trained models.We believe KB-BINDER can serve as an important baseline for future research.Our code is available at