Ensemble Learning and Linear Response Theory for ICA
Pedro A.d.F.R. Højen-Sørensen, Ole Winther, Lars Kai Hansen · 2000
We propose a general framework for performing independent component analysis (ICA) which relies on ensemble learning and linear response theory known from statistical physics. We apply it to both discrete and continuous sources. For the continuous source the underdetermined (overcomplete) case is studied. The naive mean-field approach fails in this case whereas linear response theory--which gives an improved estimate of covariances--is very efficient. The examples given are for sources without temporal correlations. However, this derivation can easily be extended to treat temporal correlations. Finally, the framework offers a simple way of generating new ICA algorithms without needing to define the prior distribution of the sources explicitly. 1 Introduction Reconstruction of statistically independent source signals from linear mixtures is an active research field. For historical background and early references see e.g. [1]. The source separation problem has a Bayesian formulation, se...