Controlled Markov set-chains with discounting
Masami Kurano, Jinjie Song, Masanori Hosaka, Youqiang Huang · Journal of Applied Probability · 1998
In the framework of discounted Markov decision processes, we consider the case that the transition probability varies in some given domain at each time and its variation is unknown or unobservable. To this end we introduce a new model, named controlled Markov set-chains, based on Markov set-chains, and discuss its optimization under some partial order. Also, a numerical example is given to explain the theoretical results and the computation.