The Least Squares Solution
James E. H. Davidson · 2018
The rank of a matrix is the dimension of its column space. In other words, every element of the column space can be formed as a linear combination of just this number of linearly independent columns. The correspondence between full rank and nonsingularity is important both because it connects the concept of rank with the problem of solving equations systems, and because it suggests an easily implemented method for testing rank. It may be helpful to think of a positive definite matrix as the generalization of the concept of a positive number in scalar algebra. The fundamental theorem of least squares has been proved by a direct examination of the sum of squares criterion. However, many textbooks appeal to differential calculus to find the minimum of the sum of squares, and this solution deserves to be reviewed for comparison.