Datasets and gate evaluation framework for benchmarking Wikipedia-based NER systems
Milan Dojchinovski, Tomáš Klie · 2013
Abstract. We present a wikifier evaluation framework consisting of software support and two datasets (News and Tweets), which were derived from datasets previously published at WEKEX 2011 and MSM Challenge 2013. Entities recognized in the original datasets were enriched with new annotations – a link to Wikipedia and the most specific type from the DBpedia Ontology. The annotations were created by two annotators and a judge. The datasets are supplemented by plugins for their import to the GATE NLP framework and a DBpedia Ontology-aware plugin for aligning annotations created by a wikifier with the ground truth.