Inter-annotator Agreement for a German Newspaper Corpus
Thorsten Brants · 2000
This paper presents the results of an investigation on inter-annotator agreement for the NEGRA corpus, consisting of German newspaper texts.The corpus is syntactically annotated with part-of-speech and structural information.Agreement for part-of-speech is 98.6%, the labeled F-score for structures is 92.4%.The two annotations are used to create a common final version by discussing differences and by several iterations of cleaning.Initial and final versions are compared.We identify categories causing large numbers of differences and categories that are handled inconsistently.