A Hybrid Approach to Identifying Objects from Verbal Descriptions
Gudrun Socher, Gernot A. Fink, Franz Kümmert, Gerhard Sagerer · 1996
In this paper we present a hybrid approach for the identification of objects in a scene from spoken verbal descriptions. An integrated knowledge base realized by a semantic network is used to build conceptual descriptions of a scene observed by cameras as well as of an utterance possibly referring to one or more objects in that scene. The object identification problem is solved using Bayesian networks. The matrices for the nodes and links of the Bayesian networks are estimated from the results of psycholinguistic experiments with naive users. 1. Introduction Man-machine-interaction in real environments is one of the greatest challenges in a number of scientific fields related to computer vision, speech understanding, and robotics. At the University of Bielefeld the joint research project "Situated Artificial Communicators" has been established with the goal to develop an integrated system where visual, linguistic, senso-motoric, and cognitive abilities interact. The system plays the r...