Multimodal Systems: Aspects of Events Fusion and a Taxonomy
Arno P. J. Gourdol, Laurence Nigay, Daniel Salber, Joëlle Coutaz · 1992
Thispaperdiscussesmultimodalsystems.Wehavedefinedataxonomyofmultimodalinteractionsandbuilttwoexamplemultimodalsystems. Weconsiderseverallevelswherethefusioncantakeplaceinthesoftwarearchitectureofthose systems.Weemphasizetheneedforacommonabstractrepresentationtorepresentinputdataandcommands, andtocombinethemsuccessfully.Thisisstillan area of exploration. Keyword Codes: H.1.2; D.2.2; H.5.1 Keywords:InformationSystems,User/MachineSystems;Software,SoftwareEngineering, ToolsandTechniques;InformationSystems,InformationInterfacesand Presentation, Multimedia Information Systems 1. INTRODUCTION Communicationbetweenthecomputerandtheuserhasbeenshowntobesignificantlyenhancedwhendifferentinputmodesaresimultaneouslyemployed [Hauptmann89]. There is ahighpotentialforsystemscombininginputmodes, but our knowledge for designing and building them is still primitive. Thispaperwillpresentadefinitionandclassificationofsuchmultimodalsystems. Wewillthendiscusssomepossiblesoftwarearchitecturesforsuchsystems...