STRONG AVERAGE OPTIMALITY FOR CONTROLLED NONHOMOGENEOUS MARKOV CHAINS*
Xianping Guo, Peng Jia Shi, Weiping Zhu · Stochastic Analysis and Applications · 2001
In this paper, the problem of three kinds of average criteria (totally so-called strong average criterion) for controlled nonhomogeneous Markov chains with possibly unbounded rewards and denumerable state space is studied. We proposed a new set of conditions under which the existence of both a solution to the optimality equations and the strong average ∈-optimal Markov policies is derived by martingale theory. Furthermore, an algorithm for computing strong average ∈(>0)-optimal Markov policies is presented.