Empirical study of mobile multimodal interaction in natural situation
Guillaume Calvet-Inglada, PAVARD, Bernard · OpenGrey (Institut de l'Information Scientifique et Technique) · 2008
Ubiquity of mobile interactive devices either in professional, cultural or domestic domain constitutes a new research perspective to both individual and cooperative usage of new communication technologies. Mobile technologies rise, among others, new questioning about mobile usability specific issues, adapted methodologies and field data collection, identification of relevant contextual factors for mobile systems design, superimposition of interaction spaces. We believe that mobility is a new interaction paradigm carrying potential contextual variations, preventing traditional direct manipulation from being the most appropriate interaction style without serious adaptation because it currently requires a high level of visual attention . In this PhD, we first present a review of current knowledge concerning Context Aware Computing (CAC) as well as mobile and multimodal interaction. We then present an empirical study of a mobile multimodal application in the context of a situated collaborative quiz-answering task. Based on quantitative and qualitative results, our research first describes global and individual usage of the available modalities (tactile and vocal). In a second time, we study the influence of some identified internal and external contextual factors on the modalities selection,. For instance, we examinate the impact of factors such as noise context, social context or history of the interaction on the modalities usage. A third part of our results highlights the relation between users choice of a modality depending on the attentional resources required by the mobile interaction. All along the quantitative results, an extensive analysis of qualitative data such as user‘s verbalisation during self confrontation sessions, or activity chronicles examination, allowed us to argue and explain some modality choices and helped us identify some new factors taken into account by the users. Finally, we use these results to propose some models for mobile multimodal applications, as well as some criteria and recommendations for their design.