Scaling Laws and Local Minima in Hebbian ICA
Magnus Rattray, Gleb Basalyga · The MIT Press eBooks · 2002
We study the dynamics of a Hebbian ICA algorithm extracting a single non-Gaussian component from a high-dimensional Gaussian background. For both on-line and batch learning we find that a surprisingly large number of examples are required to avoid trapping in a sub-optimal state close to the initial conditions. To extract a skewed signal ¢¡¤£¦¥¨ § at £ least examples are required for-dimensional data and ¢¡©£��� § examples are required to extract a symmetrical signal with non-zero kurtosis. 1