Asymptotic properties of Markov decision processes
Roger W. Brockett · 2009
In our earlier paper we derived the optimal control policy for a time varying, finite horizon, continuous time Markov processes subject to a quadratic penalty on the amount of the rate adjustment. Some aspects of the time invariant, infinite horizon problem were resolved but questions remained. In this paper we consider a more general class of performance measures and constraints on the controls. After deriving the appropriate Hamilton-Jacobi equation, we discuss asymptotic properties. The central question here relates to the possibility of that non constant (e.g., periodic) steady state policies may have better average performance than the best constant policy.