A Holistic Evaluation Methodology for Multi-Party Spoken Conversational Agents
Nancie Gunson, Angus Addlesee, Daniel Hernández García, Marta Romeo, Christian Dondrup, Oliver Lemon · 2024
While research in multi-party spoken conversation with intelligent embodied agents has made significant progress in sub-tasks like speaker identification and non-verbal cues, there’s a gap in fully autonomous applications users can directly interact with. This lack translates to the absence of a standard methodology for evaluating multi-party conversational speech agents that considers both task-based system performance and user experience.