Multiplicative Newton-like algorithm and independent component analysis
Toshinao Akuzawa · 2000
A new algorithm is constructed. We introduce a multiplicative updating method, which is natural from the structure of group manifolds and has numerous merits for the rigorous treatment of the dynamics. Cost functions invariant under the componentwise scaling are chosen. We assume that the dynamics takes place in a coset space introduced by identifying points which can be transformed to each other by this componentwise scaling. Thus, redundant degrees of freedom are eliminated. A point can still move toward any direction in this coset and there is no need to prewhiten the data. Individual steps are determined by Newton-like conditions, for which it is not necessary to tune the learning rate. An explicit and exact solution to these conditions is obtained in a form which can readily be used in practice. The second order convergence is also shown.