Fast (Structured) Newton Computations
Thomas F. Coleman, W. L. Xu · SIAM Journal on Scientific Computing · 2008
Many vector-valued functions, representing expensive computations, are also structured computations. In this case the calculation of the Newton step can be greatly accelerated by exploiting this structure. It is often not necessary, nor economic, to form the true Jacobian in the process of computing the Newton step; instead, a more cost-effective auxiliary Jacobian matrix is used. This auxiliary matrix can be sparse even when the true Jacobian matrix is dense; consequently, sparse matrix technology can be used, to great speed advantage, both in forming the auxiliary matrix and in solving the auxiliary linear system.