Backpropagation Through Time and Derivative Adaptive CriticsA Common Framework for Comparison Portions of this chapter were previously published in [4, 7,9, 1214,23].
2009
This chapter compares and contrasts derivative adaptive critics (DAC) such as dual heuristic programming (DHP), which was first introduced in Chapter 1 and also discussed in Chapter 3 with back-propagation through time (BPTT). A common framework is built and it is shown that both are techniques for determining the derivatives for training parameters in recurrent neural networks. This chapter goes into sufficient mathematical detail that the reader can understand the theoretical relationship between the two techniques. The author presents a hybrid technique that combines elements of both BPTT and DAC and provides detailed pseudocode. Computational issues and classes of challenging problems are discussed.