A NEURAL NETWORK FOR SPATIAL RELATIONS: CONNECTING VISUAL SCENES TO LINGUISTIC DESCRIPTIONS
Lars Kopp · 1994
This paper presents a system based on neural networks that can analyse spatial relations in a visual scene and connect them to appropriate linguistic descriptions. The system learns spatial concepts like "right of" and "above" by viewing a visual scene containing a number of objects and simultaneously receiving a text string describing the scene. The spatial relations between the objects in the scene are analysed with the aid of saccadic shifts of the focus attention. The system thus learns to correlate linguistic expressions for spatial relation with different kinds of saccades. After being trained, the system can correctly describe previously viewed scenes