Blind source separation: are information maximization and redundancy minimization different?
Dragan D. Obradovic, Gustavo Deco · 2002
This paper provides a detailed and rigorous analysis of the two commonly used methods for blind source separation: linear independent component analysis (ICA) and information maximization (InfoMax). The paper shows analytically that ICA based on Kullback-Leibler information as a mutual information measure and InfoMax lead to the same solution if the parameterization of the output nonlinear functions in the latter method is sufficiently rich. Furthermore, this work discusses the alternative redundancy measures not based on the Kullback-Leibler information distance and nonlinear ICA. The practical issues of applying ICA and InfoMax are also discussed.