Heuristic dynamic programming using echo state network as online trainable adaptive critic
Petia Koprinkova‐Hristova, Mohamed Oubbati, Günther Palm · International Journal of Adaptive Control and Signal Processing · 2012
SUMMARY The present paper proposes an implementation of a relatively new recurrent neural network architecture—the echo state network (ESN)–within the frame of heuristic dynamic programming. The ESN is trained online to estimate the utility function and to adapt the control policy of an embodied agent. With the advantage of an easy training algorithm, the ESN architecture offers a simple way to calculate the derivatives required for adapting the controller. Experimental results are provided to validate the proposed learning approach. Copyright © 2012 John Wiley & Sons, Ltd.