Text-image relations and empirical methods
John A. Bateman · 2014
This chapter sketches some of the empirical methods that have been applied to investigate comprehension and the reception process with textual and visual materials. All of these approaches are also being employed to explore the text-image situation, examining for example just when readers look at what on a page in their attempts to extract meaning, or what range of relationships can be expected in different kinds of documents, and so on. Psycholinguistic accounts of discourse comprehension explore how successive utterances or sentences can be related to one another during reception by finding appropriate discourse relations. Text-image functional relationships such as decoration, representation, organisation, interpretation and transformation, for example, are each shown to interact in different ways with comprehension. Corpora are also ideal for testing whether a theory is making accurate predictions. Carrying out corpus-based research on multimodality is, then, highly desirable at this time but does raise some challenges of its own.