Sample Efficient Reinforcement Learning with Gaussian Processes
Robert C. Grande, Thomas J. Walsh, Jonathan P. How · 2014
This paper derives sample complexity results for using Gaussian Processes (GPs) in both model-based and model-free reinforcement learning (RL). We show that GPs are KWIK learnable, proving for the first time that a model-based RL approach using GPs, GP-Rmax, is sample effi-cient (PAC-MDP). However, we then show that previous approaches to model-free RL using GPs take an exponential number of steps to find an optimal policy, and are therefore not sample ef-ficient. The third and main contribution is the introduction of a model-free RL algorithm us-ing GPs, DGPQ, which is sample efficient and, in contrast to model-based algorithms, capable of acting in real time, as demonstrated on a five-dimensional aircraft simulator.