Provable-Correct Partitioning Approach for Continuous-Observation POMDPs With Special Observation Distributions
Wei Zheng, Hai Lin · IEEE Control Systems Letters · 2022
The partially observable Markov decision process with continuous observations has emerged as a popular model for system modeling and sequential decision-making for many real-world problems. The main challenge induced by continuous observations is its high computational complexity in the planning process because it is impossible to enumerate all observations in a continuous space. In this letter, we propose a static observation space partitioning approach to solve a continuous-observation POMDP approximately. Although observation space partitioning approaches have been investigated in the literature, a formal analysis of the partitioning effect on the system performance is still missing. We aim to fill this gap by providing a formal analysis of the approximation error. For this, the belief update function is shown to be Lipschitz continuous for the observation when the observation function satisfies certain properties. With this property, we formally prove that the approximation error of each value iteration is bounded. Meanwhile, we show that the proposed approach can be integrated into the heuristic search value iteration algorithm with performance guarantees. Finally, the advantage of using the static partitioning approach rather than the Monte Carlo sampling approach is validated by experimental results.