Exponential Convergence in Undiscounted Continuous-Time Markov Decision Chains
W.H.M. Zijm · Mathematics of Operations Research · 1987
In this paper, we analyze the asymptotic behaviour of the value function v(t) of an undiscounted continuous-time Markov decision chain. Both the state space and the action space are assumed to be finite. A new proof of the convergence of v(t) − tg is presented (where g denotes the maximal expected average reward over an infinite time horizon). Moreover, it is shown that this convergence is exponential.