CONTROLLED MARKOV CHAINS WITH RISK-SENSITIVE CRITERIA: SOME (COUNTER) EXAMPLES Agust in Brau-Roj as2
Departamento de Estadistica, Agraria Antonio Narro, Emmanuel Fernhndez-Gaucherand · 1998
This paper is concerned with risk-sensitive versions of the standard average cost and discounted cost criteria for controlled Markov chains. Risk-sensitivity is modelled by means of the family of exponential disutility functions. First, we study the risk-sensitive average cost corresponding to a fixed stationary deterministic policy, for finite state space models. We examine some (counter) examples which illustrate how the behavior of the risk-sensitive model departs from that of the risknull one in the non-irreducible case. Finally, we present an example of a controlled Markov chain with infinite state space which, as opposed to Jaquette’s result for finite models, does not have ultimately stationay optimal policies with respect to the risk-sensitive (exponential) discounted cost. The controlled Markov chain in that example satisfies a simultaneous Doeblin condition, an assumption that enables the vanishing discount approach in the risk-null models. Thus, the example provides more evidence to think that obtaining the mentioned approach for the risk-sensitive models might be even harder than what Jaquette’s result had already indicated.