Neural Computations Supporting Cognition: Rumelhart Prize Symposium in Honor of Peter Dayan
Kenji Doya, John P. O’Doherty, Alexandre Pouget, Peter L. Bossaerts, Nathaniel D. Daw, Yael Niv · eScholarship (California Digital Library) · 2012
Neural Computations Supporting Cognition: Rumelhart Prize Symposium in Honor of Peter Dayan Participants Kenji Doya ([email protected]) John O’Doherty ([email protected]) Neural Computation Unit, Okinawa Institute of Science and Technology, 1919-1 Tancha, Onna Okinawa 904-0495 Japan Division of the Humanities and Social Sciences, California Institute of Technology, MC 228-77 Pasadena, CA 91125 USA Alexandre Pouget ([email protected]) Peter Bossaerts ([email protected]) Departement de neuroscience fondementale, Universite de Geneve, 1 rue Michel-Servet CH-1211 Geneva 4, Switzerland Division of the Humanities and Social Sciences, California Institute of Technology, MC 228-77 Pasadena, CA 91125 USA Organizors Nathaniel Daw ([email protected]) Center for Neural Science New York University New York, NY, 10003 Yael Niv ([email protected]) Princeton Neuroscience Institute and Psychology Department Princeton University, Princeton, NJ, 08544 Keywords: neural computation; reinforcement learning; inference; uncertainty and reward. After more than a decade from the discovery, however, there still remain questions to be answered, such as what striatal neuron firing represents, how and where an action is selected, and how negative reinforcement is realized. Here we review Peter Dayan's seminal contributions and recent developments. Motivation Principles of sound statistical inference underpin prominent accounts for a variety of cognitive phenomena, including perception, learning, and decision-making. Linking these building blocks of cognition to the biological substrate that supports them, recent work has investigated how the brain implements probabilistic inference and learning under uncertainty. The interplay between the psychological and biological levels of analysis has shed light on the structure of cognition and computation at both levels. This symposium builds on Peter Dayan’s seminal contributions to linking psychological, neural and computational phenomena. In particular, speakers will discuss recent work growing out of two areas where Dayan made early and fundamental contributions: the brain’s mechanisms for reinforcement learning, and neural representations supporting probabilistic inference under uncertainty. Fractionating model-based reinforcement- learning its component neural processes Author: John P. O’Doherty Abstract: It has recently been proposed that action- selection in the mammalian brain depends on at least two distinct mechanisms: a model-free reinforcement learning (RL) mechanism in which actions are selected on the basis of cached values acquired through trial and error, and a model-based RL system in which actions are chosen using values computed on-line by means of a rich cognitive model of the decision problem and knowledge of the current incentive value of goals. While much is now known about the putative neural substrates of the model-free RL system and its concomitant temporal difference prediction error, much less is known about how model-based RL is implemented at the neural level. In this talk I will review recent evidence from a series of functional neuroimaging studies in humans supporting the presence of neural signals within a wide expanse of cortex that are relevant to model- based RL. These include, a state-action based prediction error signal within a fronto-parietal network that could mediate learning of the cognitive model, a goal-value signal encoding the value of putative goal-outcomes within the Reinforcement learning and the basal ganglia Authors: Kenji Doya and Makoto Ito Abstract: The discovery of the parallel between the firing of dopamine neurons and the temporal difference error signal of the reinforcement theory in the 1990s brought a breakthrough in understanding the function of the basal ganglia. Previously the most enigmatic part of the brain is now considered as the center for linking perception, action,