Asymptotic Properties of a Finite State Continuous Time Markov Decision Process
Gerd Rodé · SIAM Journal on Control and Optimization · 1982
We consider a continuous time Markov decision process with a finite state space. There is a specified terminal reward, but the reward or cost rate is always zero. The maximum expected final gain can then be obtained by means of the exponential of a certain sublinear operator on $R^n$ This representation allows us to describe the asymptotic properties of the reward vector. We prove that the expected reward always tends to a limit as the time parameter $t \to \infty $. If we assume that it is allowed to stop the process in any state, then we can construct an almost optimal stationary control. Finally, we characterize the case where the asymptotic gain is independent of the initial state.