Investigating Open Source LLMs to Retrofit Competency Questions in Ontology Engineering
Reham Alharbi, Valentina A. M. Tamma, Floriana Grasso, Terry R. Payne · Proceedings of the AAAI Symposium Series · 2024
Competency Questions (CQs) are essential in ontology engineering; they express an ontology's functional requirements as natural language questions, offer crucial insights into an ontology's scope and are pivotal for various tasks, e.g. ontology reuse, testing, requirement specification, and pattern definition. Despite their importance, the practice of publishing CQs alongside ontological artefacts is not commonly adopted. We propose an approach based on Generative AI, specifically Large Language Models (LLMs) for retrofitting CQs from existing ontologies and we investigate how open LLMs (i.e. Llama-2-70b, Mistral 7B and Flan-T5-xl) perform in generating CQs for existing ontologies. We compare these results with our previous efforts using closed-source LLMs and we reflect on the results.