Regularization of bellman equations for infinite-horizon probabilistic properties
Ilya Tkachev, Alessandro Abate · 2012
This work studies Bellman integral equations arising in infinite-horizon probabilistic verification problems for discrete time homogeneous Markov processes over general state spaces. The problems of interest are expressed via specifications such as probabilistic reachability, invariance, reach-avoid and mean exit time. The contribution shows that the uniqueness of the solutions of the corresponding Bellman equations depends on the presence of absorbing sets within the state space. Furthermore, the work puts forward methods to modify the integral equations to obtain unique solutions for them, techniques to compute such solutions with explicit bounds on the approximation error, and conditions to characterize the possible presence of absorbing sets over the state space.