Computational Performance of State-Value Function Approximators Based on RLS-HDP Estimators for Online DLQR Control System Design
Ernesto Franklin Marçal Ferreira, João Viana da Fonseca Neto, Patrícia Helena Moraes Rêgo · 2016
The Adaptive/Approximate Dynamic Programming (ADP) is an online design approach proposed to make possible the implementations in real time of optimal controllers based on Hamilton-Jacobi-Bellman (HJB) equation solution. In this paper, ADP schemes are presented in a Heuristic Dynamic Programming (HDP) framework, where Policy Iteration (PI) strategies in conjunction with Recursive Least Squares (RLS) methods are oriented to solve online the Riccati-type HJB equation associated with the Discrete Linear Quadratic Regulator (DLQR) problem. However, these schemes have a reasonable numerical complexity and, furthermore, numerical instability may be caused due to the covariance matrix illconditioning of the RLS approach. Thus, in order to improve numerical stability, as well as to reduce the computational effort spent on approximating the DLQR cost function, UDUT factorization and orthogonal decomposition methods, such as QR decomposition, are incorporated into the standard PI-HDP framework. The performance of the standard PI-HDP method and its variants are compared in terms of numerical stability and computational cost. It is shown that such variants lead to significant computational performance improvements when compared to the standard PI-HDP method.