Semantic understanding and commonsense reasoning in an adaptive photo agent
Xinyu Hugo Liu · 2002
In a story telling authoring task, an author often wants to set up meaningful connections between different media, such as between a text and photographs. To facilitate this task, it is helpful to have a software agent dynamically adapt the presentation of a media database to the user's authoring activities, and look for opportunities for annotation and retrieval. Expecting the user to manually annotate photos with keywords greatly burdens the user. Furthermore, even when photos are properly annotated, their retrieval is often very brittle because semantic connections between annotations and the story text that are "obvious " to people (e.g. between “bride ” and “wedding”) may easily be missed by the computer. ARIA (Annotation and Retrieval Integration Agent) is a software agent that acts as an assistant to a user writing e-mail or Web pages. As the user types a story, it does continuous retrieval and ranking on a photo database. It can use descriptions in the story