Combining semantic and temporal constraints for multimodal integration in conversation systems
Joyce Yue Chai, Pengyu Hong, Michelle X. Zhou · 2003
In a multimodal conversation, user referring patterns could be complex, involving multiple referring expressions from speech utterances and multiple gestures. To resolve those references, multimodal integration based on semantic constraints is insufficient. In this paper, we describe a graph-based probabilistic approach that simultaneously combines both semantic and temporal constraints to achieve a high performance.