Appendix C: A 15‐Minute Tutorial on Nonlinear Optimization
DeWayne R. Derryberry · 2014
It is unlikely that the reader will ever write a program that solves a nonlinear optimization problem, but functions in R (and many other languages) exist for solving such problems and such functions, in R or otherwise, have many special issues.In particular, when a function fails to report a useful answer, the user needs to know if the problem is a bug (poorly written code), or a failure of the called function to converge to a reliable answer.The purpose of this section is to understand the difficulties in solving a nonlinear optimization problem, what error messages are likely to occur, and how to address these errors. C.2 NEWTON'S METHOD FOR ONE-DIMENSIONAL NONLINEAR OPTIMIZATIONConsider the following problem: min e x + x 2 over all possible values of x.The obvious option is to take the derivative and set it equal to zero, f ′ (x) = e x + 2x = 0, but this does not have a closed form solution.When an optimization problem produces a derivative or set of first derivatives that cannot be solved as a system of k linear equations in k unknowns, it is a nonlinear optimization problem.Such problems usually require iterative solutions.A method for solving such problems that is often learned in first-year calculus is Newton's method.The method uses the last guess to produce a next guess and Basic Data Analysis for Time Series with R, First Edition.DeWayne R. Derryberry.