Solving Nonlinear Continuous State-Action-Observation POMDPs for Mechanical Systems with Gaussian Noise
Marc Peter Deisenroth, Jan Peters · TUbilio (Technical University of Darmstadt) · 2012
In this paper, we introduce a novel model-based approach to solving the important subclass of partially observable Markov decision processes (POMDPs) with Gaussian noise in contin-uous states, actions, and observations. This kind of POMDP frequently appears in robotics and many other real-world control problems. However, except for the linear quadratic Gaus-sian case, no efficient ways of computing optimal controllers are known. We propose a novel method for efficiently approximating optimal solutions of nonlinear stochastic continuous state-action-observation POMDPs in high dimensions. We use Gaussian processes (GPs) to model both the latent transition dynamics and the measurement mapping. By explicit marginalization over the GP posteriors our method is robust to model errors and can be used for principled belief space inference, policy learning, and policy execution.