Evaluation for Partial Event Coreference
Jun Araki, Eduard H. Hovy, Teruko Mitamura · 2014
This paper proposes an evaluation scheme to measure the performance of a system that detects hierarchical event structure for event coreference resolution.We show that each system output is represented as a forest of unordered trees, and introduce the notion of conceptual event hierarchy to simplify the evaluation process.We enumerate the desiderata for a similarity metric to measure the system performance.We examine three metrics along with the desiderata, and show that metrics extended from MUC and BLANC are more adequate than a metric based on Simple Tree Matching.