Annotation and Evaluation of Utterance Intention Tag for Interview Dialogue Corpus
Manabu Sasayama, Kazuyuki Matsumoto · 2021
In this paper, we proposed the utterance intention tags for an interview dialogue corpus. We constructed an interview dialogue corpus with the tags we designed. Three or five annotators annotated the tags to the interview dialogue corpus (the total number of 49999 utterances) consisting of 30 dialogues. We conducted an evaluation experiment using Fleiss's kappa value to evaluate the reliability of the proposed tags. When three annotators annotated 18 different tags to the corpus, we obtained the kappa value of 0.55.