Informally prototyping multimodal, multidevice user interfaces
Anoop Kumar Sinha, James A. Landay · 2003
Increasingly, it is important to look at the end-user's tool of the future as not a solitary PC, but as a diverse set of devices, ranging from laptops to PDA's to tablet computers. Some of these devices do not have keyboard and mouse, and thus multimodal interaction techniques, such as pen input and speech input, will be required to interface with them. Interaction designers are beginning to be faced with the challenge of creating interfaces that target this style of interface. Our study into their interface design practice uncovered the lack of processes and tools to help them. This dissertation covers the motivation, design, and development of CrossWeaver, a tool for helping these designers prototype multimodal, multidevice user interfaces. This tool embodies the informal prototyping paradigm, leaving design representations in an informal, sketched form and creating a working prototype from these sketches. Informal prototypes created with CrossWeaver can run across multiple standalone devices simultaneously, processing multimodal input from each one. CrossWeaver captures all of the user interaction when running a test of a prototype. This input log can quickly be viewed for the details of the users' multimodal interaction, and it can be replayed across all participating devices, giving the designer information to help him or her iterate the interface design. Our evaluation of CrossWeaver with professional designers has shown that we have created an effective tool for early creative design of multimodal, multidevice user interfaces. CrossWeaver dovetails with existing design processes and can assist in a number of current design challenges.