Contracting Markov Decision Processes.
Peter R. Freeman, J.A.E.E. van Nunen · Journal of the Royal Statistical Society Series A (General) · 1978
INTROVUCTIONIn the last three decades much attention has been given to Markov decision processes.Markov decision processes w!;)re first introduced by Bellman [ 2] in 1957, and constitute a special class of dynamic programming problems.In 1960 How?-rd [ 35] published his book "Dynamic programming and Markov processes".This publication gave an important impulse to the investigation of Markov decision processes. 'We will first give an outline of the decision processes to be investigated.Consider a system with a countable state space s.The system can be controlled at discrete points in time t = 0,1,2, ••• , by a decision maker.,IfDEFINITION 2.3.6.The metr>ie dµ on V µ,b,p is defined by LEMMA 2.3.2.A set V b with the metric d is a complete metric space.µ, ,p µ unless explicitely mentioned we fixed µ, b, and p for the remaining part of this monograph.Referring to these fixed µ, b and p we will omit the subscripts µ, b, p.2.4.Some ~ on bounding 6unction.6In this section we give some properties of a bounding function µ' with respect to the corresponding spaces W , and V , • µ µ