Denumerable State Markovian Decision Processes-Average Cost Criterion
Cyrus Derman · The Annals of Mathematical Statistics · 1966
Abstract : Markovian decision processes with a countable number of states and average cost criterion are considered. Counterexamples are presented to show that optimal control rules need not exist or if they do exist they may not be of a deterministic stationary Markovian character. Conditions are presented under which optimal rules do exist and are stationary deterministic. Further conditions are presented under which a policy improvement procedure converges to an optimal rule. (Author)