Faster Principal Component Regression via Optimal Polynomial Approximation to sgn(x).
Zeyuan Allen Zhu, Yuanzhi Li · arXiv (Cornell University) · 2016
We solve principle component regression (PCR) by providing an efficient algorithm to project any vector onto the subspace formed by the top principle components of a matrix. Our algorithm does not require any explicit construction of the top principle components, and therefore is suitable for large-scale PCR instances. Specifically, to project onto the subspace formed by principle components with eigenvalues above a threshold $\lambda$ and with a multiplicative accuracy $(1\pm \gamma) \lambda$, our algorithm requires $\tilde{O}(\gamma^{-1})$ black-box calls of ridge regression. In contrast, previous result requires $\tilde{O}(\gamma^{-2})$ such calls. We obtain this result by designing a degree-optimal polynomial approximation of the sign function.