Living Hand to Mouth: Psychological Theories about Speech and Gesture in Interactive Dialogue Systems
Justine Cassell, Matthew D. Stone · 1999
In this paper we discuss the application of aspects of a psy-chological theory about the relationship between speech and gesture to the implementation of interactive dialogue systems. We first lay out some uncontroversial facts about the interac-tion of speech and gesture in conversation and describe some psychological theories put forth to explain those data, settling on one theory as being the most interesting for interactive di-alogue systems. We then lay out our implementation of an interactive dialogue system that is informed by the theory— concentrating on two particular claims of the theory: that ges-ture and speech reflect a common conceptual source; and that the content and form of gesture is tuned to the communicative context and the actor’s communicative intentions. We com-pare our work to some other similar interactive systems, and conclude with some thoughts about how both implementation and theory can benefit from this kind of close partnership. Epigraph Pantomime without discourse will leave you nearly tranquil, discourse without gestures will wring tears from you. Essay on the Origin of Languages, Jean-Jacques Rousseau