Randomized independent component analysis
Matan Sela, Ron Kimmel · 2016
Independent component analysis (ICA) is a method for recovering statistically independent signals from observations of unknown linear combinations of sources. Some of the most accurate ICA decomposition methods require optimizing different approximations of the Mutual Information, a measure of statistical independence between random variables. Two such approximations are the Kernel Generalized Variance or the Kernel Canonical Correlation which has been shown to reach the highest performance of ICA methods. However, the computational effort necessary just for computing them is cubic in the sample size. Hence, optimizing them becomes even more computationally demanding, in terms of both space and time. Alternatively, we propose a couple of alternative novel statistical independence measures based on randomized features. The computational complexity for calculating the proposed alternatives is linear in the sample size and provide a controllable approximation of their kernel-based deterministic versions. We also demonstrate that optimizing over the proposed statistical properties yields a comparable separation error at an order of magnitude faster.