Matrix calculus: the essentials
Jan R. Magnus, Jan R. Magnus · Wiley series in probability and statistics · 2019
This chapter summarizes the theory and the practical applications of matrix calculus. It serves as an introduction for (advanced) undergraduates or Master's and PhD students in economics, statistics, mathematics, and engineering, who want to know how to apply matrix calculus without going into all the theoretical details. The chapter begins by introducing the concept of a differential, which lies at the heart of matrix calculus. Next, it discusses vector calculus and optimization, with and without constraints. The chapter also emphasizes the importance of a correct definition and notation for the derivative, presents the ‘first identification theorem', which links the first differential with the first derivative, and applies these results to least squares. Then, it extends the theory from vector calculus to matrix calculus and obtains the differentials of the determinant and inverse. Finally, the chapter introduces the vec operator and the Kronecker product, and discusses symmetry (commutation and duplication matrices).