Evaluating Large Language Models in Interaction with Open Government Data
Ioannis Maslaris, Areti Karamanou, Evangelos Kalampokis, Konstantinos A. Tarabanis · 2024
Large Language Models (LLMs) exhibit great abilities in understanding and generating natural language.Open Government Data (OGD) are datasets that while are available to the public, their linked structure makes it difficult to access.Large language models can significantly enhance access to linked open government data by enabling users to interact with OGD portals using natural language.This study examines the use of LLMs to interact with open government linked data effectively and efficiently.Based on the QB vocabulary, we develop a framework that formulates the task.A set of 20 questions is developed to assess the capabilities of LLMs to execute OGD-related tasks.We propose a simple system in which LLMs interact semi-automatically with OGD.Our findings indicate that smaller and quantized versions of popular LLMs are capable of effectively managing these tasks, with Llama-3.1-8B-Instructbnb-4bitidentified as the most effective model.This paper aims to promote further interest in systems that enhance Open Government Data portals and improve public access to open data.