Some computational experiments with a special generalized Markov programming model
P.J. Weeda · Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands · 1974
The principles of generalized Markov programming were developed by DE LEVE [3] to solve continuous time Markov decision problems under the long run average return criterion.In this report the special generalized Markov decision model is investigated that arises if the natural process 1 is given by a finite state semi Markov process and interventions are restricted to the points in time just after a state transition in the natural process.The iteration method for this model induced by the general iteration scheme of DE LEVE is given.Four variants on the iteration method are developed which all have the pleasant property in this special model of convergence within a finite number of steps to an optimal strategy.The results of computational experience with these variants are presented.The problems solved include randomly generated problems as well as three numerical versions of a preformulat.edproblem from the field of production control.The numerical results are compared with those obtained by applying existing policy iteration methods to these problems.