Plan recognition using multimodal integration
Выборнова Ольга Николаевна, Konstantinos Gaitanis, Macq Benoit · eScholarship (California Digital Library) · 2006
We describe an ongoing research aimed at development of an anthropomorphic multimodal interface (meaning an interface for which the goal is not to precisely achieve a measurable task, like in instrumental interfaces, but more to provide a natural interaction) for recognition of a general intention, as well as of particular goals and plan fragments. To recognize the communicative and overall practical intentions and goals of a person we use information arriving from natural language and gestures (actions). The result of the multimodal integration is a joint semantic meaning representation of the multimodal behavior and communicative intentions of the user. We use semantic frames (with slots to be filled, situational modeling) defining for every type of intention (goal) what can be said, with which gestures, actions emotions (using a multimodal