Negentropy and Kurtosis as Projection Pursuit Indices Provide Generalised ICA Algorithms
Mark Girolami · 1996
We develop a generalised form of the independent component analysis (ICA) algorithm introduced by Bell and Sejnowski [1], Amari et al [2] and lately by Pearlmutter and Parra [3] and also MacKay [4]. Motivated by information theoretic indices for exploratory projection pursuit (EPP) we show that maximisation by natural gradient ascent of the divergence of a multivariate distribution from normality, using the negentropy as a distance measure, yields a generalised ICA. We introduce a form of nonlinearity which has an inherently simple form and exhibits the Bussgang property [30] within the algorithm. We show that this is sufficient to perform ICA on data which has latent variables exhibiting either unimodal or bimodal probability density functions (PDF) or both. Kurtosis has been used as a moment based projection pursuit index and as a contrast for ICA [5, 6, 7]. We introduce a simple adaptive nonlinearity which is formed by on-line estimation of the latent variable kurtosis and demonstra...