Gram-Schmidt versus Bauer-Rutishauser in alternating least-squares algorithms for three-mode principal component analysis
Pieter M. Kroonenberg, ten Josephus Berge, Piet W. Brouwer, Henk A. L. Kiers · University of Groningen research database (University of Groningen / Centre for Information Technology) · 1989
Introduction In nonlinear data analysis, alternating least squares algorithms are frequently employed. The basic principle underlying these algorithms is that for a specific loss function the parameters can be divided into blocks, which can be estimated separately with least squares conditionally upon the other blocks. By estimating each block in turn, the overall loss function can be shown to converge in a monotone fashion to at least a local optimum. A large number of applications of alternating least squares algorithms have been reviewed by Young (1981). Many alternating least squares algorithms initially used the singular value decomposition (SVD) in one of their steps. However, as shown by Gifi (1981, p. 180), the SVD can sometimes be replaced by (modified) Gram-Schmidt orthogonalization. The reason for using Gram-Schmidt (GS) orthogonalization rather than a singular value decomposition is that one GS-step is cheaper than one SVD step (see Schwarz, Rutishauser, & Stiefel, 1968, p. 186).