A framework for automated SPARQL conformance evalution across major graph databases
FreiDok plus (Universitätsbibliothek Freiburg) · 2026
SPARQL is the standard query language for graph databases, the systems that store data as interconnected facts rather than in tables. A formal specification defines how every query should behave but real engines do not always follow it in the same way and checking how faithfully an engine conforms is tedious: each system is started, queried and returns its answers differently. This thesis presents a framework that automates the task in an engine-agnostic way: it runs the official SPARQL 1.1 conformance test suite of 631 tests against any engine through a small, system-specific adapter, while the shared logic that drives the tests, compares the answers and reports the outcome is written once. Applied to seven widely used engines—QLever, Apache Jena Fuseki, Virtuoso, GraphDB, Blazegraph, Oxigraph and MillenniumDB—it produced roughly 4,400 test outcomes, with strict pass rates from about 55% to 85%. The aim is to provide reusable, reproducible infrastructure that makes ongoing SPARQL conformance testing across diverse graph databases practical.