Adapt and Decompose: Efficient Generalization of Text-to-SQL via Domain Adapted Least-To-Most Prompting
Aseem Arora, Shabbirhussain Bhaisaheb, Harshit Nigam, Manasi S. Patwardhan, Lovekesh Vig, Gautam M. Shroff · 2023
Cross-domain and cross-compositional generalization of Text-to-SQL semantic parsing is a challenging task.Existing Large Language Model (LLM) based solutions rely on inference-time retrieval of few-shot exemplars from the training set to synthesize a run-time prompt for each Natural Language (NL) test query.In contrast, we devise an algorithm which performs offline sampling of a minimal set-of few-shots from the training data, with complete coverage of SQL clauses, operators and functions, and maximal domain coverage within the allowed token length.This allows for synthesis of a fixed Generic Prompt (GP), with a diverse set-of exemplars common across NL test queries, avoiding expensive test time exemplar retrieval.We further auto-adapt the GP to the target database domain (DA-GP), to better handle cross-domain generalization; followed by a decomposed Least-To-Most-Prompting (LTMP-DA-GP) to handle cross-compositional generalization.The synthesis of LTMP-DA-GP is an offline task, to be performed one-time per new database with minimal human intervention.Our approach demonstrates superior performance on the KaggleDBQA dataset, designed to evaluate generalizability for the Textto-SQL task.We further showcase consistent performance improvement of LTMP-DA-GP over GP, across LLMs and databases of Kag-gleDBQA, highlighting the efficacy and model agnostic benefits of our prompt based adapt and decompose approach.Domain Adapted Exemplar 1: NL: Find the latitudes of nuclear power plants with capacity more than 50.Decompositions: Find the latitudes of nuclear power plants; with capacity more than 50.Intermediate Representations (NAT-SQL): select distinct nuclear_power_plants.Latitude; select where