A general framework for component estimation
Jason A. Palmer, Kenneth Kreutz-Delgado · 2003
Component estimation arises in Independent Component Analysis (ICA), Blind Source Separation (BSS), wavelet analysis and signal denoising [1], image reconstruction [2, 3], Factor Analysis [4], and sparse coding [5, 6]. In theoretical and algorithmic developments, an important distinction is commonly made between sub- and super-gaussian densities, super-gaussian densities being characterized as having high kurtosis, or having a sharp peak and heavy tails. In this paper we present a generalized convexity framework similar to a classical concept of E.F.Beckenbach [7], which we refer to as relative convexity. Based on a partial ordering induced by relative convexity, we derive a new measure of function curvature and a new criterion for super-gaussianity that is both simpler and of wider application than the kurtosis criterion. The relative convexity framework also provides an inequality that can be used to derive stable and eective descent algorithms for estimation of the parameters in the Bayesian linear model when sub- or super-gaussian priors are used. Apparently almost all common symmetric densities are comparable in this ordering to Gaussian, and thus are either sub- or super-gaussian, despite the fact that the measure is instantaneous, in contrast to momentbased measures. We present several algorithms for component estimation that are shown to be descent algorithms based on the relative convexity inequality arising from the assumption of super-gaussian priors. We also show an interesting relationship between the curvature of a convex or concave function and the curvature of its Fenchel-Legendre conjugate, which results in an elegant duality relationship between estimation with sub- and super-gaussian densities.