On partially observed stochastic shortest path problems
S.D. Patek · Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228) · 2002
We analyze a class of partially observed stochastic shortest path problems. These are terminating Markov decision process with imperfect state information that evolve on an infinite time horizon and have a total cost criterion. For wellposedness, we make reasonable stochastic shortest path type assumptions: (1) the existence of a policy that guarantees termination with probability one; and (2) the property that any policy that fails to guarantee termination has infinite expected cost from some initial state. We also assume that termination is perfectly recognized. We establish the existence of a stationary optimal policy along with the existence of a unique bounded solution to Bellman's equation. We also reveal the convergence properties of value and policy iteration. For the case where policies exist that do not guarantee termination, the dynamic programming operator fails to be a contraction mapping with respect to any norm, somewhat complicating the analysis.