Iterative dynamic programming : From curiosity to a practical optimization procedure
Rein Luus · Control and Intelligent Systems · 1998
The development of iterative dynamic programming into a practical optimization procedure for solving optimal control problems has taken numerous steps. In a systematic way, changes have been made to improve the convergence properties and to improve the computational efficiency. These changes have enabled dynamic programming to be used in an iterative manner to solve complex optimal control problems. Although, mathematically, convergence of iterative dynamic programming has not been proven, there is evidence that for many problems it works better than methods based on Pontryagin's maximum principle. The ideas underlying the development of changes to dynamic programming, and future developments are discussed in this review paper and illustrated by two examples.