Integrated taxonomy of errors in chat-oriented dialogue systems
Ryuichiro Higashinaka, Masahiro Araki, Hiroshi Tsukahara, Masahiro Mizukami · 2021
This paper proposes a taxonomy of errors in chat-oriented dialogue systems.Previously, two taxonomies were proposed; one is theorydriven and the other data-driven.The former suffers from the fact that dialogue theories for human conversation are often not appropriate for categorizing errors made by chat-oriented dialogue systems.The latter has limitations in that it can only cope with errors of systems for which we have data.This paper integrates these two taxonomies to create a comprehensive taxonomy of errors in chat-oriented dialogue systems.We found that, with our integrated taxonomy, errors can be reliably annotated with a higher Fleiss' kappa compared with the previously proposed taxonomies.