/spl alpha/-EM algorithm and /spl alpha/-ICA learning based upon extended logarithmic information measures
Y. Mtsuyama, T. Nimoto, N. Katsumata, Yudai Suzuki, S. Furukawa · 2000
The /spl alpha/-logarithm extends the logarithm as the special case of /spl alpha/=-1. Usage of /spl alpha/-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the /spl alpha/-EM algorithm (/spl alpha/-expectation-maximization algorithm) which is derived from the /spl alpha/-log-likelihood ratio. The other is the /spl alpha/-ICA (/spl alpha/-independent component analysis) which is formulated as minimizing the /spl alpha/-mutual information. In the derivation of both algorithms, the /spl alpha/-divergence plays the main role. For the /spl alpha/-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the /spl alpha/-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop /spl alpha/-EM and sample-based /spl alpha/-ICA. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the /spl alpha/-logarithm is a general property beyond individual problems.