A statistical interpretation of Krylov subspaces decomposition
Pierre Druilhet, Alain Mom · 2006
Consider the standard linear model Y = Xβ + ε. where β the p−vectors of unknown parameters. To stabilize the OLS estimator when explanatory variables are highly correlated, PLS uses a Gram-Schmidt decomposition of the Krylov subspaces Kq = span(s, S s, S 2 s,..., S q−1 s) generated by S = X ′ X and s = X ′ Y. We show how this decomposition can be obtained from an algorithm that iteratively maximizes the directional signal-to-noise ratio (SNR) applied to the least squares estimator under orthogonality constraints. The SNR on the direction given by x ∈ Rp is defined by SNR = |x ′ βols | σ √ x ′ S−1x, and is related with optimal shrinkage factors that realize optimal trade-off between bias and variance.