Chapter 7: Applications to Markov Decision Processes
Society for Industrial and Applied Mathematics eBooks · 2013
7.1 Markov Decision Processes: Concepts and Introduction Whereas Markov chains (MCs) form a good description of some discrete event stochastic processes, they are not automatically equipped with a capability to model situations where there may be a “decision-maker” or a “controller” who—by a judicious choice of actions—can influence the trajectory of the process. Hence, in this chapter, we consider discrete time Markov decision processes (MDPs) with finite state and action spaces and study the dependence of optimal policies/controls of these decision processes on certain important parameters. In this context our usual ∊-perturbation is seen as simply an instance of a more generic parametric dependence.