Online Learning-based Optimal Control of Nonlinear Systems with Finite-Time Convergence Guarantees
Nick-Marios T. Kokolakis, Kyriakos G. Vamvoudakis · 2022 American Control Conference (ACC) · 2022
This paper develops a critic-only reinforcement learning-based algorithm for learning the solution to the Hamilton-Jacobi-Bellman equation in finite time. In particular, a non-Lipschitz experience replay-based learning law utilizing recorded and current data is introduced for updating the critic weights to learn the value function. The non-Lipschitz property of the dynamics gives rise to finite-time convergence and stability, while the experience replay-based approach eliminates the need to satisfy the persistence of excitation condition if the recorded data is sufficiently rich. Simulation results demonstrate the efficacy of the proposed approach.