The Journal of Neurophysiology Vol. 80 No. 2 August 1998, pp. 964-977
Copyright ©1998 The American Physiological Society

Modifications of Reward Expectation-Related Neuronal Activity During Learning in Primate Striatum

Léon Tremblay, Jeffrey R. Hollerman, and Wolfram Schultz

Institute of Physiology, University of Fribourg, CH-1700 Fribourg, Switzerland

Tremblay, Léon, Jeffrey R. Hollerman, and Wolfram Schultz. Modifications of reward expectation-related neuronal activity during learning in primate striatum. J. Neurophysiol. 80: 964-977, 1998. This study investigated neuronal activity in the anterior striatum while monkeys repeatedly learned to associate new instruction stimuli with known behavioral reactions and reinforcers. In a delayed go-nogo task with several trial types, an initial picture instructed the animal to execute or withhold a reaching movement and to expect a liquid reward or not. During learning, new instruction pictures were presented, and animals guessed and performed one of the trial types according to a trial-and-error strategy. Learning of a large number of pictures resulted in a learning set in which learning took place in a few trials and correct performance exceeded 80% in the first 60-90 trials. About 200 task-related striatal neurons studied in both familiar and learning conditions showed three forms of changes during learning. Activations related to the preparation and execution of behavioral reactions and the expectation of reward were maintained in many neurons but occurred in inappropriate trial types when behavioral errors were made. The activations became appropriate for individual trial types when the animals' behavior adapted to the new task contingencies. In particular, reward expectation-related activations occurred initially in both rewarded and unrewarded movement trials and became subsequently restricted to rewarded trials. These changes occurred in parallel with the visible adaptation of reward expectations by the animals. The second learning change consisted in decreases of task-related activations that were either restricted to the initial trials of new learning problems or persisted during the subsequent consolidation phase. They probably reflected reductions in the expectation and preparation of upcoming task events, including reward. The third learning change consisted in transient or sustained increases of activations. These might reflect the increased attention accompanying learning and serve to induce synaptic changes underlying the behavioral adaptations. Both decreases and increases often induced changes in the trial selective occurrence of activations. In conclusion, neurons in anterior striatum showed changes related to adaptations or reductions of expectations in new task situations and displayed activations that might serve to induce structural changes during learning.

This article has been cited by other articles:

				B. Draganski, F. Kherif, S. Kloppel, P. A. Cook, D. C. Alexander, G. J. M. Parker, R. Deichmann, J. Ashburner, and R. S. J. Frackowiak Evidence for Segregated and Integrative Connectivity Patterns in the Human Basal Ganglia J. Neurosci., July 9, 2008; 28(28): 7143 - 7152. [Abstract] [Full Text] [PDF]

				C. S. Lansink, P. M. Goltstein, J. V. Lankelma, R. N. J. M. A. Joosten, B. L. McNaughton, and C. M. A. Pennartz Preferential Reactivation of Motivationally Relevant Information in the Ventral Striatum J. Neurosci., June 18, 2008; 28(25): 6372 - 6382. [Abstract] [Full Text] [PDF]

				B. Lau and P. W. Glimcher Action and Outcome Encoding in the Primate Caudate Nucleus J. Neurosci., December 26, 2007; 27(52): 14502 - 14514. [Abstract] [Full Text] [PDF]

				L. H. Corbit and P. H. Janak Inactivation of the Lateral But Not Medial Dorsal Striatum Eliminates the Excitatory Impact of Pavlovian Stimuli on Instrumental Responding J. Neurosci., December 19, 2007; 27(51): 13977 - 13981. [Abstract] [Full Text] [PDF]

				B. W. Balleine, M. R. Delgado, and O. Hikosaka The Role of the Dorsal Striatum in Reward and Decision-Making J. Neurosci., August 1, 2007; 27(31): 8161 - 8165. [Abstract] [Full Text] [PDF]

				F. Colpaert, W. Koek, M. Kleven, and J. Besnard Induction by Antipsychotics of "Win-Shift" in the Drug Discrimination Paradigm J. Pharmacol. Exp. Ther., July 1, 2007; 322(1): 288 - 298. [Abstract] [Full Text] [PDF]

				P. W. German and H. L. Fields Rat Nucleus Accumbens Neurons Persistently Encode Locations Associated With Morphine Reward J Neurophysiol, March 1, 2007; 97(3): 2094 - 2106. [Abstract] [Full Text] [PDF]

				M. A. Teagarden and G. V. Rebec Subthalamic and Striatal Neurons Concurrently Process Motor, Limbic, and Associative Information in Rats Performing an Operant Task J Neurophysiol, March 1, 2007; 97(3): 2042 - 2058. [Abstract] [Full Text] [PDF]

				M. R. Roesch, T. A. Stalnaker, and G. Schoenbaum Associative Encoding in Anterior Piriform Cortex versus Orbitofrontal Cortex during Odor Discrimination and Reversal Learning Cereb Cortex, March 1, 2007; 17(3): 643 - 652. [Abstract] [Full Text] [PDF]

				C. A. Seger and C. M. Cincotta Dynamics of Frontal, Striatal, and Hippocampal Systems during Rule Learning Cereb Cortex, November 1, 2006; 16(11): 1546 - 1555. [Abstract] [Full Text] [PDF]

				C. Amiez, J.P. Joseph, and E. Procyk Reward Encoding in the Monkey Anterior Cingulate Cortex Cereb Cortex, July 1, 2006; 16(7): 1040 - 1055. [Abstract] [Full Text] [PDF]

				O. Hikosaka, K. Nakamura, and H. Nakahara Basal Ganglia Orient Eyes to Reward J Neurophysiol, February 1, 2006; 95(2): 567 - 584. [Abstract] [Full Text] [PDF]

				M. Haruno and M. Kawato Different Neural Correlates of Reward Expectation and Reward Expectation Error in the Putamen and Caudate Nucleus During Stimulus-Action-Reward Association Learning J Neurophysiol, February 1, 2006; 95(2): 948 - 959. [Abstract] [Full Text] [PDF]

				K. Watanabe and O. Hikosaka Immediate Changes in Anticipatory Activity of Caudate Neurons Associated With Reversal of Position-Reward Contingency J Neurophysiol, September 1, 2005; 94(3): 1879 - 1887. [Abstract] [Full Text] [PDF]

				M. Campos, B. Breznen, K. Bernheim, and R. A. Andersen Supplementary Motor Area Encodes Reward Expectancy in Eye-Movement Tasks J Neurophysiol, August 1, 2005; 94(2): 1325 - 1335. [Abstract] [Full Text] [PDF]

				M. R. Roesch and C. R. Olson Neuronal Activity Dependent on Anticipated and Elapsed Delay in Macaque Prefrontal Cortex, Frontal and Supplementary Eye Fields, and Premotor Cortex J Neurophysiol, August 1, 2005; 94(2): 1469 - 1497. [Abstract] [Full Text] [PDF]

				D. I. G. Wilson and E. M. Bowman Rat Nucleus Accumbens Neurons Predominantly Respond to the Outcome-Related Properties of Conditioned Stimuli Rather Than Their Behavioral-Switching Properties J Neurophysiol, July 1, 2005; 94(1): 49 - 61. [Abstract] [Full Text] [PDF]

				W. A. Suzuki and E. N. Brown Behavioral and Neurophysiological Analyses of Dynamic Learning Processes Behav Cogn Neurosci Rev, June 1, 2005; 4(2): 67 - 95. [Abstract] [PDF]

				C. A. Seger and C. M. Cincotta The Roles of the Caudate Nucleus in Human Classification Learning J. Neurosci., March 16, 2005; 25(11): 2941 - 2951. [Abstract] [Full Text] [PDF]

				F. Worgotter and B. Porr Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms Neural Comput., February 1, 2005; 17(2): 245 - 319. [Abstract] [Full Text] [PDF]

				D. Arkadir, G. Morris, E. Vaadia, and H. Bergman Independent Coding of Movement Direction and Reward Prediction by Single Pallidal Neurons J. Neurosci., November 10, 2004; 24(45): 10047 - 10056. [Abstract] [Full Text] [PDF]

				N. Schmitzer-Torbert and A. D. Redish Neuronal Activity in the Rodent Dorsal Striatum in Sequential Navigation: Separation of Spatial and Reward Responses on the Multiple T Task J Neurophysiol, May 1, 2004; 91(5): 2259 - 2272. [Abstract] [Full Text] [PDF]

				H. Yamada, N. Matsumoto, and M. Kimura Tonically Active Neurons in the Primate Caudate Nucleus and Putamen Differentially Encode Instructed Motivational Outcomes of Action J. Neurosci., April 7, 2004; 24(14): 3500 - 3510. [Abstract] [Full Text] [PDF]

				S. M. Nicola, I. A. Yun, K. T. Wakabayashi, and H. L. Fields Cue-Evoked Firing of Nucleus Accumbens Neurons Encodes Motivational Significance During a Discriminative Stimulus Task J Neurophysiol, April 1, 2004; 91(4): 1840 - 1865. [Abstract] [Full Text] [PDF]

				A. J. Tindell, K. C. Berridge, and J. W. Aldridge Ventral Pallidal Representation of Pavlovian Cues and Reward: Population and Rate Codes J. Neurosci., February 4, 2004; 24(5): 1058 - 1069. [Abstract] [Full Text] [PDF]

				G. Schoenbaum and B. Setlow Lesions of Nucleus Accumbens Disrupt Learning about Aversive Outcomes J. Neurosci., October 29, 2003; 23(30): 9833 - 9841. [Abstract] [Full Text] [PDF]

				M. R. Roesch and C. R. Olson Impact of Expected Reward on Neuronal Activity in Prefrontal Cortex, Frontal and Supplementary Eye Fields and Premotor Cortex J Neurophysiol, September 1, 2003; 90(3): 1766 - 1789. [Abstract] [Full Text] [PDF]

				M. Pessiglione, D. Guehl, Y. Agid, E. C. Hirsch, J. Feger, and L. Tremblay Impairment of context-adapted movement selection in a primate model of presymptomatic Parkinson's disease Brain, June 1, 2003; 126(6): 1392 - 1408. [Abstract] [Full Text] [PDF]

				I. Toni, J. Rowe, K. E. Stephan, and R. E. Passingham Changes of Cortico-striatal Effective Connectivity during Visuomotor Learning Cereb Cortex, October 1, 2002; 12(10): 1040 - 1047. [Abstract] [Full Text] [PDF]

				J. N. D. Kerr and D. Plenz Dendritic Calcium Encodes Striatal Neuron Output during Up-States J. Neurosci., March 1, 2002; 22(5): 1499 - 1512. [Abstract] [Full Text] [PDF]

				A. Messinger, L. R. Squire, S. M. Zola, and T. D. Albright Neuronal representations of stimulus associations develop in the temporal lobe during learning PNAS, September 19, 2001; (2001) 211431098. [Abstract] [Full Text] [PDF]

				O. K. Hassani, H. C. Cromwell, and W. Schultz Influence of Expectation of Different Rewards on Behavior-Related Neuronal Activity in the Striatum J Neurophysiol, June 1, 2001; 85(6): 2477 - 2489. [Abstract] [Full Text] [PDF]

				R. E. Suri and W. Schultz Temporal Difference Model Reproduces Anticipatory Neural Activity Neural Comput., April 1, 2001; 13(4): 841 - 862. [Abstract] [Full Text]

				J. T. Williams, M. J. Christie, and O. Manzoni Cellular and Synaptic Adaptations Mediating Opioid Dependence Physiol Rev, January 1, 2001; 81(1): 299 - 343. [Abstract] [Full Text] [PDF]

				M. M. Solis, M. S. Brainard, N. A. Hessler, and A. J. Doupe Song selectivity and sensorimotor signals in vocal learning and production PNAS, October 24, 2000; 97(22): 11836 - 11842. [Abstract] [Full Text] [PDF]

				R. M. Carelli, S. G. Ijames, and A. J. Crumling Evidence That Separate Neural Circuits in the Nucleus Accumbens Encode Cocaine Versus "Natural" (Water and Food) Reward J. Neurosci., June 1, 2000; 20(11): 4255 - 4266. [Abstract] [Full Text] [PDF]

				L. Tremblay and W. Schultz Modifications of Reward Expectation-Related Neuronal Activity During Learning in Primate Orbitofrontal Cortex J Neurophysiol, April 1, 2000; 83(4): 1877 - 1885. [Abstract] [Full Text] [PDF]

				W. Schultz, L. Tremblay, and J. R. Hollerman Reward Processing in Primate Orbitofrontal Cortex and Basal Ganglia Cereb Cortex, March 1, 2000; 10(3): 272 - 283. [Abstract] [Full Text] [PDF]

				M. S. Jog, Y. Kubota, C. I. Connolly, V. Hillegaart, and A. M. Graybiel Building Neural Representations of Habits Science, November 26, 1999; 286(5445): 1745 - 1749. [Abstract] [Full Text]

				J. R. Hollerman, L. Tremblay, and W. Schultz Influence of Reward Expectation on Behavior-Related Neuronal Activity in Primate Striatum J Neurophysiol, August 1, 1998; 80(2): 947 - 963. [Abstract] [Full Text] [PDF]

				A. Messinger, L. R. Squire, S. M. Zola, and T. D. Albright Neuronal representations of stimulus associations develop in the temporal lobe during learning PNAS, October 9, 2001; 98(21): 12239 - 12244. [Abstract] [Full Text] [PDF]