Unsupervised learning for blind source separation: an information-theoretic approach
Dragan D. Obradovic, Gustavo Deco · 2002
This paper provides a detailed and rigorous analysis of the two commonly used methods for redundancy reduction: linear independent component analysis (ICA) and information maximization (InfoMax). The paper shows analytically that ICA based on the Kullback-Leibler information as a mutual information measure and InfoMax lead to the same solution if the parameterization of the output nonlinear functions in the latter method is sufficiently rich. Furthermore, this work discusses the alternative redundancy measures not based on the Kullback-Leibler information distance and nonlinear ICA. The practical issues of applying ICA and InfoMax are also discussed.