Data-driven indelible planning of discourse generation using salience
E. Jeffrey Conklin · 1983
Natural language generation can be divided into two stages: deep generation, in which content and style are specified, and realization, in which this specification is converted into text. This thesis rejects the goal-driven planning of previous work, using instead the salience of objects in a database to provide data-driven selection--a computationally much less expensive approach. This research studies the generation of descriptions of natural suburban scenes. The phenomenon of salience was explored through a series of psychological experiments--in some, subjects provided subjective ratings of the relative importance of the items in photographs; in others, subjects wrote short descriptions of the same pictures. The rating data provided the basis for the beginnings of a theory of visual salience as a perceptual phenomenon, while analysis of the combination of rating and textual data showed how salience orders the mention of objects in descriptions. The LISP program GENARO plans paragraph descriptions of scenes. Its input representation, though hand-built, is designed to simulate the perceptual output of a computer vision system, and includes an annotation of empirically-based salience values. GENARO uses rules (each of which knows about some specific or stylistic effect) to build rhetorical specifications (which are then realized by McDonald's MUMBLE program). Its processing is data-driven, i.e. a current-item is selected from a salience-ordered list of perceptual objects, and it and its salience are the primary determiners for the actions of the rules. This planning process is myopic--it has no explicit goal. Also, all of GENARO's actions are indelible--the control structure provides no look-ahead or backup. The fact that this localized planning is able to devise quite natural-sounding paragraphs demonstrates that a data-driven approach to deep generation is viable, and that salience can be a powerful heuristic in guiding natural language generation.