Model-Based Reinforcement Learning for Infinite-Horizon Approximate Optimal Tracking
Rushikesh Kamalapurkar, Lindsey Andrews, Patrick Walters, Warren E. Dixon · IEEE Transactions on Neural Networks and Learning Systems · 2016
This brief paper provides an approximate online adaptive solution to the infinite-horizon optimal tracking problem for control-affine continuous-time nonlinear systems with unknown drift dynamics. To relax the persistence of excitation condition, model-based reinforcement learning is implemented using a concurrent-learning-based system identifier to simulate experience by evaluating the Bellman error over unexplored areas of the state space. Tracking of the desired trajectory and convergence of the developed policy to a neighborhood of the optimal policy are established via Lyapunov-based stability analysis. Simulation results demonstrate the effectiveness of the developed technique.