Single application model, multiple synchronized views
Rafah A. Hosn, Stephane Herman Maes, T. V. Raman · 2001
User interface is a mean to an end —its primary goal is to capture user intent and communicate the results of the re-quested computation. On today’s devices, user interaction can be achieved through a multiplicity of interaction modal-ities including speech and visual interfaces. As we evolve toward an increasingly connected world where we access and interact with applications throughmul-tiple devices, it becomes crucial that the various access paths to the underlying content be synchronized. This synchro-nization ensures that the user interacts with the same un-derlying content independent of the interaction modality — despite the difference in presentation that eachmodalitymight impose. It also ensures that the effect of user interaction in any given modality is reflected consistently across all avail-able modalities. We describe an application framework that enables tightly synchronized multimodal user interaction. This framework derives its power from representing the application model in a modality-independent manner, and by traversing this model to produce the various synchronizedmultimodal views. As the user interaction proceeds, we maintain our current position in the model and update the application data as de-termined by user intent, then reflect these updates in the var-ious views being presented. We conclude the paper by outlining an example that demonstrates this tightly synchronized multimodal interac-tion, and describe some of the future challenges in building such multimodal frameworks. 1.