Controlled two-dimensional Markov chains between two absorbing barriers
Mario Lefebvre · The Journal of Difference Equations and Applications · 2023
Let (Xn,Yn) be a two-dimensional discrete-time controlled Markov chain. The problem of optimally controlling the chain until the sum Xn+Yn is equal to either of two constants is considered. The aim is to leave the continuation region as soon as possible. The optimal control is found in two particular cases by solving the partial difference equation satisfied by the value function.