Inter-annotator Agreement for ERE annotation
Seth Kulick, Ann Bies, Justin L. Mott · 2014
This paper describes a system for inter-annotator agreement analysis of ERE an-notation, focusing on entity mentions and how the higher-order annotations such as EVENTS are dependent on those entity mentions. The goal of this approach is to provide both (1) quantitative scores for the various levels of annotation, and (2) infor-mation about the types of annotation in-consistencies that might exist. While pri-marily designed for inter-annotator agree-ment, it can also be considered a system for evaluation of ERE annotation. 1