Finite state markovian decision processes with weak and strong interactions
Qing Zhang · Stochastics and stochastics reports · 1996
This paper is concerned with hierarchical control of a class of finite state continuous time Markovian decision processes having both weak and strong interactions. The state of the process can be divided into several groups such that transitions among the states within each group occur much more frequently than the transitions among the states belonging to different groups. This gives rise to a limiting control problem which is simpler to solve. The limiting problem is obtained by replacing the states in each group by the corresponding average distribution. A nearly optimal solution for the original problem is constructed by using an optimal solution to the limiting problem. Both discounted cost and long-run average cost criteria are considered. Error bounds of constructed controls are obtained for problems with both cost criteria. Finally, computational methods are discussed.