Instruction Tuning Text-to-SQL with Large Language Models in the Power Grid Domain
Sun Gang, Ran Shen, Liangfeng Jin, Y.H. Wang, Shiyu Xu, Jinpeng Chen, Weihao Jiang · 2023
This paper explores the large language models to address the Text-to-SQL task in real-world scenarios in the electricity domain. To tackle the lack of training data and corresponding databases for vertical domain real-world scenarios, the paper devised specific prompts to leverage ChatGPT for data generation, achieving significant improvements in annotation efficiency through automated data generation. Furthermore, to apply the powerful semantic parsing and generation capabilities of large language models to Text-to-SQL, the paper utilized a large language model for instruction tuning for SQL generation. This model has undergone secondary pre-training with electrical knowledge, tailoring it to the specific SQL generation task. On the power grid test set, the paper’s matching accuracy reached 65.7%, and the execution accuracy reached 80.9%. Additionally, the paper conducted further tests on various general large language models for zero-shot learning and single-sample prompt-based Text-to-SQL. The results indicate that while simple single-table queries can be achieved, meeting the requirements for complex queries remains challenging.