On-Line Singular Value Decomposition of Stochastic Process Covariances
Tomas Landelius, Hans E. Knutsson, Magnus Borga · 1995
This paper presents novel algorithms for finding the singular value decomposition (SVD) of a general covariance matrix by stochastic approximation. General in the sense that also non-square, between sets, covariance matrices are dealt with. For one of the algorithms, convergence is shown using results from stochastic approximation theory. Proofs of this sort, establishing both the point of equilibrium and its domain of attraction, have been reported very rarely for stochastic, iterative feature extraction algorithms. 1 Introduction The ability to perform dimensionality reduction is crucial to systems exposed to high dimensional data. One way of approaching this problem is to project the data on the direction of maximal data variation, the largest principal component. There are also a number of applications in signal processing where the largest eigenvalue and the corresponding eigenvalue of input data correlation or covariance matrices play an important role. One such example is foun...