Multimodal semantic fusion based on context
Danli Wang · Jisuanji gongcheng yu sheji · 2007
Multimodal interaction has become an important direction in the field of human computer interaction,and how to fuse mul-timodal inputs in a semantic level is the key problem.A method that combines context with the traditional semantic fusion is presented,and it is applied to a storytelling prototype for children.Children could create stories with restricted natural language and pen strokes.It concludes that the multimodal interaction combined with pen and voice could make the communication between children and computer more convenient and natural,and the context-based semantic fusion method could correct input errors to some extent.