Approximating reachable belief points in POMDPs
Kyle Hollins Wray, Shlomo Zilberstein · 2017
We propose an algorithm called σ-approximation that compresses the non-zero values of beliefs for partially observable Markov decision processes (POMDPs) in order to improve performance and reduce memory usage. Specifically, we approximate individual belief vectors with a fixed bound on the number of non-zero values they may contain. We prove the correctness and a strong error bound when the σ-approximation is used with the point-based value iteration (PBVI) family algorithms. An analysis compares the algorithm on six larger domains, varying the number of non-zero values for the σ-approximation. Results clearly demonstrate that when the algorithm used with PBVI (σ-PBVI), we can achieve over an order of magnitude improvement. We ground our claims with a full robotic implementation for simultaneous navigation and localization using POMDPs with σ-PBVI.