PROMISE - A Procedure for Multimodal Interactive System Evaluation
Nicole Beringer, Ute Kartal, Katerina Louka, Florian Schiel · 2002
This paper describes a general framework for evaluating and comparing the performance of multimodal dialogue systems: PROMISE (Procedure for Multimodal Interactive System Evaluation). PROMISE is a possible extention to multimodality of the PARADISE framework ([1, 2] used for the evaluation of spoken dialogue systems), where we aimed to solve the problems of scoring multimodal inputs and outputs, weighting the different recognition modalities and of how to deal with not directed (non-directed) task definitions and the resulting, potentially uncompleted tasks by the users.