Detecting Missing Annotation Disagreement using Eye Gaze Information
Koh Mitsuda, Ryu Iida, Takenobu Tokunaga · 2013
This paper discusses the detection of miss-ing annotation disagreements (MADs), in which an annotator misses annotating an annotation instance while her counterpart correctly annotates it. We employ anno-tator eye gaze as a clue for detecting this type of disagreement together with lin-guistic information. More precisely, we extract highly frequent gaze patterns from the pre-extracted gaze sequences related to the annotation target, and then use the gaze patterns as features for detecting the MADs. Through the empirical evaluation using the data set collected in our previ-ous study, we investigated the effective-ness of each type of information. The re-sults showed that both eye gaze and lin-guistic information contributed to improv-ing performance of our MAD detection model compared with the baseline model. Furthermore, our additional investigation revealed that some specific gaze patterns could be a good indicator for detecting the MADs. 1