Near-optimal Control of Switched Systems with Continuous-time Dynamics using Approximate Dynamic Programming
Tohid Sardarmehni · SMU Scholar (Southern Methodist University) · 2018
Optimal control is a control method which provides inputs that minimize a performance index subject to state or input constraints [58]. The existing solutions for finding the exact optimal control solution such as Pontryagin’s minimum principle and dynamic programming suffer from curse of dimensionality in high order dynamical systems. One remedy for this problem is finding near optimal solution instead of the exact optimal solution to avoid curse of dimensionality [31]. A method for finding the approximate optimal solution is through Approximate Dynamic Programming (ADP) methods which are discussed in the subsequent chapters. In this dissertation, optimal switching in switched systems with autonomous subsystems is studied. In order to derive the optimal solution, ADP method is used. Two iterative schemes, namely policy iteration and value iteration, from ADP methods are studied. For policy iteration, continuous-time dynamics is considered and two different methods for solving the underlying Lyapunov equation as gradient descent and recursive least squares are studied. Also, a new method is introduced which tries to reduce the computational burden in policy iteration. For all the policy iteration based solutions, convergence of the iterations to the optimal solution and stability of the system during the training is studied. Additionally, three methods for implementing the policy iteration based solutions as offline training, online training, and concurrent training methods are discussed in details. For value iteration, the problem of deriving the optimal switching policy in anti-lock brake system of ground vehicles is investigated. For this purpose, a typical hydraulic brake system is used which can increase, decrease or hold the hydraulic braking pressure. The control goal is switching such that a variable in the model, called slip ratio, is regulated at its optimal value which leads to minimum stopping distance. For this problem, discrete time dynamics is used.