Automated intention recognition of grouped bar charts in multimodal documents
M. Sandra Carberry, Richard J. Burns · 2013
Information graphics (bar charts, line graphs, grouped bar charts, etc.) often appear in popular media such as newspapers and magazines. In most cases, the information graphic is intended to convey a high-level message. This message plays a role in understanding the document but is seldom repeated in the document's text, headlines, or captions. This thesis presents a methodology and implemented system for recognizing the intended message of a grouped bar chart. This thesis identifies the types of high-level messages that are overwhelmingly conveyed by grouped bar charts in popular media, as well as the communicative signals that signal them. It also presents a linguistic classifier that predicts the most linguistic salient entity from multiple entities that are mentioned in text, as well as a cognitive model that estimates the relative perceptual effort for an individual to recognize a high-level message given a graph. A Bayesian network captures the probabilistic relationship between the high-level intended message of a graphic and its communicative signals. The automatic identification of the intended message of a grouped bar chart contributes to the following areas: 1) extending screen readers to information graphics for assisting sight-impaired individuals in accessing electronic documents, 2) considering information graphics alongside article text in the summarization of multimodal documents, and 3) indexing and retrieving information graphics with their intended messages in digital libraries.