A Markovian reset problem
PETER J. KOLESAR · IEEE Transactions on Automatic Control · 1970
The optimal control of a system whose transitions through time are described by a finite state stationary Markov chain is studied. It is assumed that there are two boundary states which are to be avoided and a single calibration state to which the system is contolled by resetting. The determination of an optimal reset policy is formulated as an ordinary linear programming problem. It is shown that under certain symmetry and regularity conditions the optimal control rule has a simple and intuitive structure, while examples are given to show that relaxation of these conditions leads to the need for more complex control.