Language in Multimedia: Interaction of Spoken and Written Information
Nader T. Tavassoli · Journal of Consumer Research · 1998
Recent findings on the brain's anatomy suggest that there are modality-specific resources for the processing of language. This article develops a theoretical model on the behavioral consequences of modality-specific resources for spoken and written information. The model predicts antagonistic effects of reduced interference versus reduced integration of verbal information presented across the two modalities, as compared to within a single modality. Three experiments compare unimodal to multimedia messages containing either a mix of modalities or information presented bimodally. The results provide qualified support for the model in the realm of item-specific versus relational memory and elaboration. Compared to unimodal presentations, alternating the presentation modality decreases relational processing but can increase item-specific processing. Furthermore, a simultaneous bimodal presentation format increases recognition memory but does not appear to affect relational processing or item-specific elaboration.