Optimal Infinite-Horizon Undiscounted Control of Finite Probabilistic Systems
Loren K. Platzman · SIAM Journal on Control and Optimization · 1980
A finite-input, finite-state, finite-output stochastic control problem with imperfect state observation and classical information pattern is shown to be meaningful as the horizon increases without bound and the discount rate approaches unity. The plant model, a finite probabilistic system, includes the Markov decision and partially-observed Markov decision problems as special cases. Under conditions resembling controllability and observability in linear systems we show that: an optimal strategy exists, it may be realized by a stationary policy on the state estimate, its performance does not depend on the initial state distribution, and convergence rates for its finite-horizon and discounted performances are readily established.