A Non-linear Information Maximisation Algorithm that Performs Blind Separation
Anthony J. Bell, Terrence J. Sejnowski · 1994
A new learning algorithm is derived which performs online stochas-tic gradient ascent in the mutual information between outputs and inputs of a network. In the absence of a priori knowledge about the 'signal ' and 'noise ' components of the input, propagation of information depends on calibrating network non-linearities to the detailed higher-order moments of the input density functions. By incidentally minimising mutual information between outputs, as well as maximising their individual entropies, the network 'fac-torises ' the input into independent components. As an example application, we have achieved near-perfect separation of ten digi-tally mixed speech signals. Our simulations lead us to believe that our network performs better at blind separation than the Herault-J utten network, reflecting the fact that it is derived rigorously from the mutual information objective. 468 Anthony J. Bell, Terrence J. Sejnowski 1