SCENE MODELING AND IMAGE MINING WITH A VISUAL GRAMMAR
Selim Aksoy, Carsten Tusk, Krzysztof Koperski, Giovanni Marchisio · WORLD SCIENTIFIC eBooks · 2003
Automatic content extraction, classi¯cation and content-based retrieval are highly desired goals in intelligent remote sensing databases. Pixel level processing has been the common choice for both academic and commercial systems. We extend the modeling of remotely sensed imagery to three levels: Pixel level, region level and scene level. Pixel level features are generated using unsupervised clustering of spectral values, texture features and ancillary data like digital elevation models. Region level features include shape information and statistics of pixel level feature values. Scene level features include statistics and spatial relationships of regions. This chapter describes our work on developing a probabilistic visual grammar to reduce the gap between low-level features and high-level user semantics, and to support complex query scenarios that consist of many regions with di®erent feature characteristics. The visual grammar includes automatic identi¯cation of region prototypes and modeling of their spatial relationships. The system learns the prototype regions in an image collection using unsupervised clustering. Spa-tial relationships are represented by fuzzy membership functions. The system automatically selects signi¯cant relationships from training data and builds vi-sual grammar models which can also be updated using user relevance feedback. A Bayesian framework is used to automatically classify scenes based on these mod-els. We demonstrate our system with query scenarios that cannot be expressed by traditional region or scene level approaches but where the visual grammar provides accurate classi¯cations and e®ective retrieval. 1.