Using OpenStreetMap Data to Create Benchmarks for Description Logic Reasoners.
Thomas Eiter, Patrik Schneider, Mantas Šimkus, Guohui Xiao · 2014
Description Logics (DLs) are a well-established and popular family of decidable logics for knowledge representation and reasoning [6]. Several systems have been developed to reason over DL knowledge bases (KBs), which usually consist of a TBox and an ABox. A TBox describes the domain in terms of concepts and roles, while an ABox stores information about known instances of concepts and their participation in roles. Naturally, classical reasoning tasks like TBox satisfiability and subsumption under a TBox have received most attention and many reasoners have been devoted to them, e.g. FaCT++ [24], HermiT [22], or ELK [14]. These and other mature systems geared towards TBox reasoning have been rigorously tested and compared (e.g., JustBench [7]) using several real-life ontologies like GALEN and SNOMED-CT. A different category are reasoners for ontology-based query answering (OQA), which plays a major role in ontology-based data access (OBDA) [19]. They are designed to answer queries over DL KBs in the presence of large data instances (see e.g. Ontop [20], Pellet [21], Stardog [4] and OWL-BGP [16]). TBoxes in this setting are usually expressed in low complexity DLs, and are relatively small in size compared to the size of instance data. which makes OQA different from classical TBox reasoners. Several works have considered testing OQA systems [12, 17, 18, 23, 13]. Despite this, it has been acknowledged that judging the performance of OQA reasoners is difficult due to the lack of publicly available benchmarks consisting of large amounts of real-life instance data. In this paper we consider publicly available geographic datasets as a source of test data for OQA systems. In similar spirit, recently some benchmarks have been proposed to test implementations of geospatial extensions of SPARQL [15, 10]. The latter benchmarks are geared towards testing spatial reasoning capabilities, and cannot be easily adapted for testing OQA systems. The main goal of this paper is to describe how benchmarks for OQA systems can be created from OpenStreetMap [2] (OSM) geospatial data by employing a rule-based data transformation framework. The OSM project aims to collaboratively create an open map of the world. It has proven hugely successful and is constantly updated and extended. OSM data describes maps in terms of (possibly tagged) points, ways (geometries), and more complex aggregate objects called relations. We believe the following features make OSM a good source to obtain instance data for OQA reasoners: