Annotators' Certainty and Disagreements in Coreference and Bridging Annotation in Prague Dependency Treebank
Anna Nedoluzhko, Jiří Mírovský · 2013
In this paper, we present the results of the parallel Czech coreference and bridging annotation in the Prague Dependency Treebank 2.0. The annotation is carried out on dependency trees (on the tectogrammatical layer). We describe the inter-annotator agreement measurement, classify and analyse the most common types of annotators ’ disagreement. On two selected long texts, we asked the annotators to mark the degree of certainty they have in some most problematic points; we compare the results to the inter-annotator agreement measurement. 1