A Dimension-Independent Generalization Bound for Kernel Supervised Principal Component Analysis
Hassan Ashtiani, Ali Ghodsi · Neural Information Processing Systems · 2015
Kernel supervised principal component analysis (KSPCA) is a computationally ecient supervised feature extraction method that can learn non-linear transformations. We start the study of the statistical properties of KSPCA, providing the rst bound on its sample complexity. This bound is dimension-independent, which justies the good performance of KSPCA on high-dimensional data. Another observation is that in the kernelized version, the number of parameters of KSPCA grows linearly with the sample size. While this potentially increases the risk of over-tting, KSPCA works well in practice. In this work, we justify this compelling characteristic of KSPCA by providing a guarantee indicating that KSPCA generalizes well even when the number of parameters is large, as long as they have small norms.