Modelling the Statistics of Natural Images with Topographic Product of Student-t Models
Simon Osindero · 2004
Keywords: Energy-based model; natural scene statistics; overcomplete representations; contrastive divergence. We present an energy-based model that uses a product of generalised Student-t dis-tributions to capture the statistical structure in datasets. This model is inspired by and particularly applicable to “natural ” datasets such as images. We begin by providing the mathematical framework, where we discuss complete as well as undercomplete 1 and overcomplete models and provide algorithms for training these models from data. Using patches of natural scenes we demonstrate that our approach represents a viable alternative to “independent components analysis ” as an interpretive model of biolog-ical visual systems. Although the two approaches are similar in flavor there are also important differences, particularly when the representations are overcomplete. We study the topographic organization of Gabor-like receptive fields that are learned by our model, both on mono- as well as stereo-inputs. Finally, we discuss the relation of our new approach to previous work — in particular Gaussian Scale Mixture models and variants of independent components analysis. 1