Enhancing Text-to-SQL Conversion in Turkish: An Analysis of LLMs with Schema Context
Ferhat Demirkıran, Ali Kemal Coşkun, Yavuz Kömeçoğlu, Başak Buluz Kömeçoğlu, Ramazan Güven · 2024
The task of converting natural language text to SQL queries has gained significant attention, particularly with the advancement of large language models (LLMs). However, research focused on text-to-SQL systems in non-English languages, such as Turkish, remains limited. This paper presents a comparative study evaluating the performance of LLMs, including GPT-3.5 Turbo, T5, and SQLCoder, on the TUR2SQL dataset-a cross-domain Turkish text-to-SQL dataset. Our experiments highlight the critical role of schema context in enhancing the accuracy of SQL generation, particularly for the T5 model, which showed significant improvements in logical-form and execution accuracy when fine-tuned with schema context. This study provides valuable insights into the challenges and potential of applying LLMs to Turkish text-to-SQL tasks, underscoring the importance of model fine-tuning and schema linking for accurate S Q L query generation in low-resource languages.