|
|
||||||||
The Journal of Neurophysiology Vol. 80 No. 1 July 1998,
pp. 1-27
Copyright ©1998 The American Physiological Society
INVITED REVIEW
Institute of Physiology and Program in Neuroscience, University of Fribourg, CH-1700 Fribourg, Switzerland
Schultz, Wolfram. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80: 1-27, 1998. The effects of lesions, receptor blocking, electrical self-stimulation, and drugs of abuse suggest that midbrain dopamine systems are involved in processing reward information and learning approach behavior. Most dopamine neurons show phasic activations after primary liquid and food rewards and conditioned, reward-predicting visual and auditory stimuli. They show biphasic, activation-depression responses after stimuli that resemble reward-predicting stimuli or are novel or particularly salient. However, only few phasic activations follow aversive stimuli. Thus dopamine neurons label environmental stimuli with appetitive value, predict and detect rewards and signal alerting and motivating events. By failing to discriminate between different rewards, dopamine neurons appear to emit an alerting message about the surprising presence or absence of rewards. All responses to rewards and reward-predicting stimuli depend on event predictability. Dopamine neurons are activated by rewarding events that are better than predicted, remain uninfluenced by events that are as good as predicted, and are depressed by events that are worse than predicted. By signaling rewards according to a prediction error, dopamine responses have the formal characteristics of a teaching signal postulated by reinforcement learning theories. Dopamine responses transfer during learning from primary rewards to reward-predicting stimuli. This may contribute to neuronal mechanisms underlying the retrograde action of rewards, one of the main puzzles in reinforcement learning. The impulse response releases a short pulse of dopamine onto many dendrites, thus broadcasting a rather global reinforcement signal to postsynaptic neurons. This signal may improve approach behavior by providing advance reward information before the behavior occurs, and may contribute to learning by modifying synaptic transmission. The dopamine reward signal is supplemented by activity in neurons in striatum, frontal cortex, and amygdala, which process specific reward information but do not emit a global reward prediction error signal. A cooperation between the different reward signals may assure the use of specific rewards for selectively reinforcing behaviors. Among the other projection systems, noradrenaline neurons predominantly serve attentional mechanisms and nucleus basalis neurons code rewards heterogeneously. Cerebellar climbing fibers signal errors in motor performance or errors in the prediction of aversive events to cerebellar Purkinje cells. Most deficits following dopamine-depleting lesions are not easily explained by a defective reward signal but may reflect the absence of a general enabling function of tonic levels of extracellular dopamine. Thus dopamine systems may have two functions, the phasic transmission of reward information and the tonic enabling of postsynaptic neurons.
This article has been cited by other articles:
![]() |
A. Ishikawa, F. Ambroggi, S. M. Nicola, and H. L. Fields Dorsomedial Prefrontal Cortex Contribution to Behavioral and Nucleus Accumbens Neuronal Responses to Incentive Cues J. Neurosci., May 7, 2008; 28(19): 5088 - 5098. [Abstract] [Full Text] [PDF] |
||||
![]() |
V. VORHOLD The Neuronal Substrate of Risky Choice: An Insight into the Contributions of Neuroimaging to the Understanding of Theories on Decision Making under Risk Ann. N.Y. Acad. Sci., April 1, 2008; 1128(1): 41 - 52. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. A. Trafton and E. V. Gifford Behavioral Reactivity and Addiction: The Adaptation of Behavioral Response to Reward Opportunities J Neuropsychiatry Clin Neurosci, February 1, 2008; 20(1): 23 - 35. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. Lau and P. W. Glimcher Action and Outcome Encoding in the Primate Caudate Nucleus J. Neurosci., December 26, 2007; 27(52): 14502 - 14514. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. A. Klein, J. Neumann, M. Reuter, J. Hennig, D. Y. von Cramon, and M. Ullsperger Genetically Determined Differences in Learning from Errors Science, December 7, 2007; 318(5856): 1642 - 1645. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. T. Moyer, J. A. Wolf, and L. H. Finkel Effects of Dopaminergic Modulation on the Integrative Properties of the Ventral Striatal Medium Spiny Neuron J Neurophysiol, December 1, 2007; 98(6): 3731 - 3748. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. A. Farries and A. L. Fairhall Reinforcement Learning With Modulated Spike Timing Dependent Synaptic Plasticity J Neurophysiol, December 1, 2007; 98(6): 3648 - 3665. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. B. Kim, N. Huh, H. Lee, E. H. Baeg, D. Lee, and M. W. Jung Encoding of Action History in the Rat Ventral Striatum J Neurophysiol, December 1, 2007; 98(6): 3548 - 3556. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. P. O'DOHERTY Lights, Camembert, Action! The Role of Human Orbitofrontal Cortex in Encoding Stimuli, Rewards, and Choices Ann. N.Y. Acad. Sci., December 1, 2007; 1121(1): 254 - 272. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Bunzeck, H. Schutze, S. Stallforth, J. Kaufmann, S. Duzel, H.-J. Heinze, and E. Duzel Mesolimbic Novelty Processing in Older Adults Cereb Cortex, December 1, 2007; 17(12): 2940 - 2948. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Brovelli, N. Laksiri, B. Nazarian, M. Meunier, and D. Boussaoud Understanding the Neural Computations of Arbitrary Visuomotor Learning through fMRI and Associative Learning Theory Cereb Cortex, November 27, 2007; (2007) bhm198v2. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. J. Frank, J. Samanta, A. A. Moustafa, and S. J. Sherman Hold Your Horses: Impulsivity, Deep Brain Stimulation, and Medication in Parkinsonism Science, November 23, 2007; 318(5854): 1309 - 1312. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Gruning Elman Backpropagation as Reinforcement for Simple Recurrent Networks Neural Comput., November 1, 2007; 19(11): 3108 - 3131. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Okada, J. Rybak, G. Manz, and R. Menzel Learning-Related Plasticity in PE1 and Other Mushroom Body-Extrinsic Neurons in the Honeybee Brain J. Neurosci., October 24, 2007; 27(43): 11736 - 11747. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. J. Lodge and A. A. Grace Aberrant Hippocampal Activity Underlies the Dopamine Dysregulation in an Animal Model of Schizophrenia J. Neurosci., October 17, 2007; 27(42): 11424 - 11430. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. Porr and F. Worgotter Learning with "Relevance": Using a Third Factor to Stabilize Hebbian Learning Neural Comput., October 1, 2007; 19(10): 2694 - 2719. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. M. Izhikevich Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling Cereb Cortex, October 1, 2007; 17(10): 2443 - 2452. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Sabatinelli, M. M. Bradley, P. J. Lang, V. D. Costa, and F. Versace Pleasure Rather Than Salience Activates Human Nucleus Accumbens and Medial Prefrontal Cortex J Neurophysiol, September 1, 2007; 98(3): 1374 - 1379. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. M. Bayer, B. Lau, and P. W. Glimcher Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate J Neurophysiol, September 1, 2007; 98(3): 1428 - 1439. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. A. Kliem, N. T. Maidment, L. C. Ackerson, S. Chen, Y. Smith, and T. Wichmann Activation of Nigral and Pallidal Dopamine D1-Like Receptors Modulates Basal Ganglia Outflow in Monkeys J Neurophysiol, September 1, 2007; 98(3): 1489 - 1500. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. W. Balleine, M. R. Delgado, and O. Hikosaka The Role of the Dorsal Striatum in Reward and Decision-Making J. Neurosci., August 1, 2007; 27(31): 8161 - 8165. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Seo and D. Lee Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game J. Neurosci., August 1, 2007; 27(31): 8366 - 8377. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Michelet, B. Bioulac, D. Guehl, L. Escola, and P. Burbaud Impact of Commitment on Performance Evaluation in the Rostral Cingulate Motor Area J. Neurosci., July 11, 2007; 27(28): 7482 - 7489. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Mazzoni, A. Hristova, and J. W. Krakauer Why Don't We Move Faster? Parkinson's Disease, Movement Vigor, and Implicit Motivation J. Neurosci., July 4, 2007; 27(27): 7105 - 7116. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. L. Cowen and B. L. McNaughton Selective Delay Activity in the Medial Prefrontal Cortex of the Rat: Contribution of Sensorimotor Information and Contingency J Neurophysiol, July 1, 2007; 98(1): 303 - 316. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. G. Roseberry, T. Painter, G. P. Mark, and J. T. Williams Decreased Vesicular Somatodendritic Dopamine Stores in Leptin-Deficient Mice J. Neurosci., June 27, 2007; 27(26): 7021 - 7027. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Geisler, C. Derst, R. W. Veh, and D. S. Zahm Glutamatergic Afferents of the Ventral Tegmental Area in the Rat J. Neurosci., May 23, 2007; 27(21): 5730 - 5743. [Abstract] [Full Text] [PDF] |
||||
![]() |
P.R. Corlett, G.D. Honey, and P.C. Fletcher From prediction error to psychosis: ketamine as a pharmacological model of delusions J Psychopharmacol, May 1, 2007; 21(3): 238 - 252. [Abstract] [PDF] |
||||
![]() |
J. P. O'DOHERTY, A. HAMPTON, and H. KIM Model-Based fMRI and Its Application to Reward Learning and Decision Making Ann. N.Y. Acad. Sci., May 1, 2007; 1104(1): 35 - 53. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. KOBAYASHI and K.-I. OKADA Reward Prediction Error Computation in the Pedunculopontine Tegmental Nucleus Neurons Ann. N.Y. Acad. Sci., May 1, 2007; 1104(1): 310 - 323. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. LEE and H. SEO Mechanisms of Reinforcement Learning and Decision Making in the Primate Dorsolateral Prefrontal Cortex Ann. N.Y. Acad. Sci., May 1, 2007; 1104(1): 108 - 122. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. SCHWEIGHOFER, S. C. TANAKA, and K. DOYA Serotonin and the Evaluation of Future Rewards: Theory, Experiments, and Possible Neural Mechanisms Ann. N.Y. Acad. Sci., May 1, 2007; 1104(1): 289 - 300. [Abstract] [Full Text] [PDF] |
||||
![]() |
O. HIKOSAKA Basal Ganglia Mechanisms of Reward-Oriented Eye Movement Ann. N.Y. Acad. Sci., May 1, 2007; 1104(1): 229 - 249. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. C. HORVITZ, W. Y. CHOI, C. MORVAN, Y. EYNY, and P. D. BALSAM A "Good Parent" Function of Dopamine: Transient Modulation of Learning and Performance during Early Stages of Training Ann. N.Y. Acad. Sci., May 1, 2007; 1104(1): 270 - 288. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. NIV Cost, Benefit, Tonic, Phasic: What Do Response Rates Tell Us about Dopamine and Motivation? Ann. N.Y. Acad. Sci., May 1, 2007; 1104(1): 357 - 376. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. M. COSTA Plastic Corticostriatal Circuits for Action Learning: What's Dopamine Got to Do with It? Ann. N.Y. Acad. Sci., May 1, 2007; 1104(1): 172 - 191. [Abstract] [Full Text] [PDF] |
||||
![]() |
X. Liu, D. K. Powell, H. Wang, B. T. Gold, C. R. Corbly, and J. E. Joseph Functional Dissociation in Frontal and Striatal Areas for Processing of Positive and Negative Reward Information J. Neurosci., April 25, 2007; 27(17): 4587 - 4597. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. Cui, B. E. Bernier, M. T. Harnett, and H. Morikawa Differential Regulation of Action Potential- and Metabotropic Glutamate Receptor-Induced Ca2+ Signals by Inositol 1,4,5-Trisphosphate in Dopaminergic Neurons J. Neurosci., April 25, 2007; 27(17): 4776 - 4785. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. D. Smillie, L. I. Dalgleish, and C. J. Jackson Distinguishing Between Learning and Motivation in Behavioral Tests of the Reinforcement Sensitivity Theory of Personality Pers Soc Psychol Bull, April 1, 2007; 33(4): 476 - 489. [Abstract] [PDF] |
||||
![]() |
M. Witkowski An Action-Selection Calculus Adaptive Behavior, March 1, 2007; 15(1): 73 - 97. [Abstract] [PDF] |
||||
![]() |
M. X Cohen Individual differences and the neural representations of reward expectation and reward prediction error Soc Cogn Affect Neurosci, March 1, 2007; 2(1): 20 - 30. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. W. German and H. L. Fields Rat Nucleus Accumbens Neurons Persistently Encode Locations Associated With Morphine Reward J Neurophysiol, March 1, 2007; 97(3): 2094 - 2106. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. W. Dunlop and C. B. Nemeroff The Role of Dopamine in the Pathophysiology of Depression Arch Gen Psychiatry, March 1, 2007; 64(3): 327 - 337. [Full Text] [PDF] |
||||
![]() |
S. B. Floresco and M. T. Tse Dopaminergic Regulation of Inhibitory and Excitatory Transmission in the Basolateral Amygdala-Prefrontal Cortical Pathway J. Neurosci., February 21, 2007; 27(8): 2045 - 2057. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. M. Cincotta and C. A. Seger Dissociation between Striatal Regions while Learning to Categorize via Feedback and via Observation. J. Cogn. Neurosci., February 1, 2007; 19(2): 249 - 265. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. W. Kalivas and N. D. Volkow The Neural Basis of Addiction: A Pathology of Motivation and Choice Focus, January 1, 2007; 5(2): 208 - 219. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. E. Hyman Addiction: A Disease of Learning and Memory Focus, January 1, 2007; 5(2): 220 - 228. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. B. Margolis, H. Lock, G. O. Hjelmstad, and H. L. Fields The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J. Physiol., December 15, 2006; 577(3): 907 - 924. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. D. Humphries, R. D. Stewart, and K. N. Gurney A Physiologically Plausible Model of Action Selection and Oscillatory Activity in the Basal Ganglia J. Neurosci., December 13, 2006; 26(50): 12921 - 12942. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Pecina, K. S. Smith, and K. C. Berridge Hedonic Hot Spots in the Brain Neuroscientist, December 1, 2006; 12(6): 500 - 511. [Abstract] [PDF] |
||||
![]() |
L. D. Smillie, A. D. Pickering, and C. J. Jackson The New Reinforcement Sensitivity Theory: Implications for Personality Measurement Personality and Social Psychology Review, November 1, 2006; 10(4): 320 - 335. [Abstract] [PDF] |
||||
![]() |
S. D. Gale and D. J. Perkel Physiological Properties of Zebra Finch Ventral Tegmental Area and Substantia Nigra Pars Compacta Neurons J Neurophysiol, November 1, 2006; 96(5): 2295 - 2306. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. C. Canavier and R. S. Landry An Increase in AMPA and a Decrease in SK Conductance Increase Burst Firing by Different Mechanisms in a Model of a Dopamine Neuron In Vivo J Neurophysiol, November 1, 2006; 96(5): 2549 - 2563. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. L. Remijnse, M. M. A. Nielen, A. J. L. M. van Balkom, D. C. Cath, P. van Oppen, H. B. M. Uylings, and D. J. Veltman Reduced orbitofrontal-striatal activity on a reversal learning task in obsessive-compulsive disorder. Arch Gen Psychiatry, November 1, 2006; 63(11): 1225 - 1236. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. Loewenstein and H. S. Seung Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity PNAS, October 10, 2006; 103(41): 15224 - 15229. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Rodriguez, I. Morales, I. Gomez, S. Gonzalez, T. Gonzalez-Hernandez, and J. L. Gonzalez-Mora Heterogeneous Dopamine Neurochemistry in the Striatum: The Fountain-Drain Matrix J. Pharmacol. Exp. Ther., October 1, 2006; 319(1): 31 - 43. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. L. Ames and C. McBride Translating genetics, cognitive science, and other basic science research findings into applications for prevention. Eval Health Prof, September 1, 2006; 29(3): 277 - 301. [Abstract] [PDF] |
||||
![]() |
B. Schilstrom, R. Yaka, E. Argilli, N. Suvarna, J. Schumann, B. T. Chen, M. Carman, V. Singh, W. S. Mailliard, D. Ron, et al. Cocaine Enhances NMDA Receptor-Mediated Currents in Ventral Tegmental Area Cells via Dopamine D5 Receptor-Dependent Redistribution of NMDA Receptors. J. Neurosci., August 15, 2006; 26(33): 8549 - 8558. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Kempf and A. Meyer-Lindenberg Imaging Genetics and Psychiatry Focus, August 1, 2006; 4(3): 327. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. D. Daw, A. C. Courville, and D. S. Touretzky Representation and Timing in Theories of the Dopamine System Neural Comput., July 1, 2006; 18(7): 1637 - 1677. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. I. Velligan, R. S. Kern, and J. M. Gold Cognitive Rehabilitation for Schizophrenia and the Putative Role of Motivation and Expectancies Schizophr Bull, July 1, 2006; 32(3): 474 - 485. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Ding and O. Hikosaka Comparison of reward modulation in the frontal eye field and caudate of the macaque. J. Ne |