EXTENSIONS OF ICA AS MODELS OF NATURAL IMAGES AND VISUAL PROCESSING
Patrik O. Hoyer, Jarmo Hurri · 2003
Using statistical models one can estimate features from natural images, such as images that we see in everyday life. Such models can also be used in computional vi-sual neuroscience by relating the estimated features to the response properties of neurons in the brain. A sem-inal model for natural images was linear sparse coding which, in fact, turned out to be equivalent to ICA. In these linear generative models, the columns of the mixing matrix give the basis vectors (features) that are adapted to the statistical structure of natural images. Estimated features resemble wavelets or Gabor func-tions, and provide a very good description of the prop-erties of simple cells in the primary visual cortex. We have introduced extensions of ICA that are based on modelling dependencies of the "independent " compo-nents estimated by basic ICA. The dependencies of the components are used to dene either a grouping or a topographic order between the components. With nat-ural image data, these models lead to emergence of further properties of visual neurons: the topographic organization and complex cell receptive elds. We have also modelled the temporal structure of natural image sequences using models inspired by blind source sep-aration methods. All these models can be combined in a unifying framework that we call bubble coding. Finally, we have developed a multivariate autoregres-sive model of the dependencies, which lead us to the concept of \\double-blind " source separation. 1.