A Distributional Approach to Evaluating Ontology Learning Methods Using a Gold Standard
Elias Zavitsanos, Γεώργιος Παλιούρας · 2008
Abstract. This paper presents a method for the evaluation of learned ontologies against gold standards. The proposed method transforms the ontology concepts to a vector space representation to avoid the common string matching of concepts at the lexical layer. We propose a set of evaluation measures that exploit the concepts’ representations and calculate the similarity of the two hierarchies. Experiments show that these measures scale gradually in the closed interval of [0, 1] as learned ontologies deviate increasingly from the gold standard. The proposed method is tested using the Genia and the Lonely Planet gold standard ontologies. RNA domain with DNA domain, since they only differ in one letter. Obviously, this case would lead to matching completely different concepts, which have completely different instances (and thus meaning). Furthermore, comparing concepts lexicalized with very 1