Designing the Reward System: Computational and Biological Principles
Kenji Doya · 2007
Summary form only given. In the standard framework of optimal control and reinforcement learning, a "cost" or "reward" function is given a priori, and an intelligent agent is supposed to optimize it. In real life, control engineers and machine learning researchers are often faced with the problem of how to design a good objective function to let the agent attain an intended goal efficiently and reliably. Such meta-level tuning of adaptive processes is the major challenge in bringing intelligent agents to real-world applications. While development of theoretical frameworks for 'meta-learning' of adaptive agents is an urgent engineering problem, it is an important biological question to ask what kind of meta-learning mechanisms our brain implements to enable robust and flexible control and learning. We are putting together computational, neurobiological, and robotic approaches to attack these dual problems of computational theory and biological implementation of meta-learning