Talking with robots about objects
Julia Peltason, Nina Riether, Britta Wrede, Ingo Lütkebohle · 2012
We present the design process, realization and evaluation of a robot system for nteractive object learning. The system-oriented evaluation, in particular, addresses an open problem for the evaluation of systems, where overall user satisfaction depends not only on the performance of the parts, but also on their combination, and on user behavior. Based on the PARADISE method known from spoken dialog systems, we have defined and applied internal and external metrics for fine-grained and largely automatable identification of such relationships. Through evaluation with n=28 subjects, indicator functions explaining up to 55% of variation in several satisfaction metrics were found. Furthermore, we demonstrate that the system's interaction style reduces the need for instruction and successfully recovers partial failures.