Value Function Approximation on Non-Linear Manifolds for Robot Motor Control
Masashi Sugiyama, Hirotaka Hachiya, Christopher Towell, Sethu Vijayakumar · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · 2007
The least squares approach works efficiently in value function approximation, given appropriate basis functions. Because of its smoothness, the Gaussian kernel is a popular and useful choice as a basis function. However, it does not allow for discontinuity which typically arises in real-world reinforcement learning tasks. In this paper, we propose a new basis function based on geodesic Gaussian kernels, which exploits the non-linear manifold structure induced by the Markov decision processes. The usefulness of the proposed method is successfully demonstrated in a simulated robot arm control and Khepera robot navigation.