Contraction mappings underlying undiscounted Markov decision problems—II
Paul J. Schweitzer · Journal of Mathematical Analysis and Applications · 1988
Let Q denote the one-step value-iteration operator in undiscounted Markov decision problems. This paper gives necessary and sufficient conditions for Qj to be a contraction in span-norm, for sufficiently large J. It is shown that the phenomenon of noncontraction for every J is associated with the occurrence of unbounded vectors. Since all computations in practice use bounded vectors, in practice QJ will be a contraction for sufficiently large J.