Fast Iterative Kernel PCA
Nicol Norbert Schraudolph, Simon Günter, S. V. N. Vishwanathan · The MIT Press eBooks · 2007
We introduce two methods to improve convergence of the Kernel Hebbian Algo-rithm (KHA) for iterative kernel PCA. KHA has a scalar gain parameter which is either held constant or decreased as 1/t, leading to slow convergence. Our KHA/et algorithm accelerates KHA by incorporating the reciprocal of the current estimated eigenvalues as a gain vector. We then derive and apply Stochastic Meta-Descent (SMD) to KHA/et; this further speeds convergence by performing gain adaptation in RKHS. Experimental results for kernel PCA and spectral clustering of USPS digits as well as motion capture and image de-noising problems confirm that our methods converge substantially faster than conventional KHA. 1