| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Center for the Neural Basis of Cognition, Mellon Institute, Pittsburgh, Pennsylvania 15213-2683
Submitted 10 January 2003; accepted in final form 21 May 2003
 |
ABSTRACT |
|---|
|
|
|
INTRODUCTION |
|---|
|
; Kalivas and Nakamura 1999; Mogenson et al. 1980; Ono et al. 2000). The limbic system is crucial for the initial stage of evaluating rewards. This view is supported by the finding that lesions of the amygdala (Aggleton and Passingham 1981; Baylis and Gaffan 1991; Jones and Mishkin 1972; Málková et al. 1997; Murray et al. 1996) and ventromedial-orbitofrontal cortex (Baylis and Gaffan 1991; Butter and Snyder 1972; Butter et al. 1969 1970; Gaffan and Murray 1990) interfere with reward evaluation. It is also supported by microelectrode recording studies demonstrating the presence of neurons influenced by the value of an experienced or expected reward in the amygdala (Nakamura et al. 1992; Nishijo et al. 1988a,b; Sanghera et al. 1979), the ventromedial-orbitofrontal cortex (Critchley and Rolls 1996; Hikosaka and Watanabe 2000; Rolls and Baylis 1994; Rolls et al. 1996; Thorpe et al. 1983; Tremblay and Schultz 1999, 2000a,b), the midbrain dopamine nuclei (Ljungberg et al. 1991, 1992; Mirenowicz and Schultz 1994; Romo and Schultz 1990; Schultz et al. 1993a), the lateral hypothalamus (Burton et al. 1976; Fukuda et al. 1986 1987; Mora et al. 1976; Rolls et al. 1976, 1979, 1986), and the ventral striatum (Apicella et al. 1991; Schultz et al. 1992).
The transition from evaluating an anticipated reward to initiating action in pursuit of it is thought to depend on structures interposed functionally between the limbic system and motor cortex. The dorsal striatum and the dorsolateral prefrontal cortex are frequently thought of in connection with this mediating role. They differ from limbic structures in that neuronal activity signals both the nature of action the monkey is planning to perform and the value of the expected reward (Hassani et al. 2001; Hikosaka and Watanabe 2000; Hikosaka et al. 1989; Hollerman et al. 1998; Kawagoe et al. 1998; Kobayashi et al. 2002; Lauwereyns et al. 2002a,b; Leon and Shadlen 1999; Takikawa et al. 2002a; Tremblay et al. 1998; Watanabe 1990, 1992, 1996; Watanabe et al. 2002). Thus it makes sense to think of them as at the watershed between limbic evaluative functions and motor output. However, the exact nature of the transitional role played by these areas is not clear. In particular, the functional significance of their reward-related neuronal activity remains uncertain.
On the one hand, it is widely assumed that reward-related activity in the dorsal striatum and dorsolateral prefrontal cortex corresponds to an internal representation of the reinforcement value of the expected reward (Hassani et al. 2001; Hikosaka and Watanabe 2000; Schultz 2000; Watanabe 1998). For example, Hassani et al. (2001) speculate that reward-dependent neuronal signals in the dorsal striatum contribute to "the representation of goals before and during the execution of actions," while Watanabe (1998) proposes that dorsolateral prefrontal neurons with reward-related activity have as their function "monitoring the goal (in this case, reward)" for which the monkey is working. This view is reasonable 1) because goal-directed behavior requires that the value of the outcome associated with an action be explicitly represented at the time when the action is being planned (Dickinson and Balleine 1994) and 2) because there must exist somewhere in the brain neurons whose activity underlies emotional states experienced subjectively (Papez 1937).
On the other hand, reward-related activity in the dorsal striatum and dorsolateral prefrontal cortex might arise from motivational modulation of control signals for motor preparation and motor output. It is a central feature of motivation, considered as a psychological construct, that "the motivational state serves to prime, facilitate, or potentiate a response mechanism that leads to the appetitive or consummatory behavior" (Stellar and Stellar 1985; p. 73). A commonly cited example is the tendency of rats to run faster down an alleyway in pursuit of a more valued reward (Stellar 1982). In a monkey waiting during a delay period to perform an operant response for a known reward, there are at least 4 ways in which the size of the reward might exert motivational control over neuronal activity related to motor planning and motor output. 1) Neurons representing the plan for the instructed response might fire more strongly when a more valued reward is at stake. 2) Neuronal activity sensitive to arousal or responsible for generalized motor readiness might be enhanced on trials involving a more valued reward. 3) Neuronal activity governing overt behaviors that automatically accompany response planning, such as increases of axial tonus, might be enhanced when a more valued reward is at stake. 4) Neurons involved in preparing ingestive movements might be more active before delivery of a more valued reward. These are not mere speculative possibilities. The fact that speed and accuracy are enhanced when a more valued reward is expected (Hassani et al. 2001; Hollerman et al. 1998; Kawagoe et al. 1998; Kobayashi et al. 2002; Lauwereyns et al. 2002a,b; Leon and Shadlen 1999; Takikawa et al. 2002b; Tremblay et al. 1998; Watanabe 1990; Watanabe et al. 2001) indicates, in accordance with scenarios 1 and 2, that the representation of the planned action is enhanced, or generalized readiness to respond is greater, during the delay period leading up to the response. The occurrence, in some contexts, of anticipatory licking (Hassani et al. 2001) indicates, in accordance with scenario 4, that ingestive movements tend to be programmed during the delay period. The distinction between neuronal activity representing the value of an expected reward and neuronal activity reflecting motivational modulation of motor planning and performance has been acknowledged in principle by previous authors. However, little consideration has been given to the question of how to distinguish between them in practice.
As a step toward resolving this issue, we have extended the analysis of reward-related activity beyond the prefrontal cortex (PFC) into adjacent premotor areas involved directly in oculomotor and skeletomotor control. Lesions and inactivation of the frontal eye field (FEF) (Dias and Segraves 1999; Sommer and Tehovnik 1997), supplementary eye field (SEF) (Sommer and Tehovnik 1999), premotor cortex (PM) (Kurata and Hoffman 1994), and supplementary motor area (SMA) (Brinkman 1984) result in impairments of motor control but do not interfere with the evaluation of rewards or with motivation. Thus reward-related activity, if observed in them, could reasonably be ascribed to changes in motor outflow or motor readiness rather than to the internal representation of rewards as goals. The results indicate not only that activity modulated by reward is present in premotor areas but that it is strikingly more prominent in them than in the PFC. This finding raises an important issue with reference to areas other than premotor cortex in which the value of an expected reward influences neuronal activity. Reward-dependent activity in these areas 1) might reflect the motivational modulation of motor signals as it presumably does in the premotor cortex, 2) might represent the value of the expected reward as a basis for emotional experience and goal selection, or 3) might do both. To distinguish among these possibilities will require the use of nonstandard behavioral paradigms in which reward value varies independently of motivation.
|
|
METHODS |
|---|
|
Four adult male rhesus monkeys were used (Macaca mulatta; laboratory designations N, P, A, and F). Experimental procedures were approved by the Carnegie Mellon University Animal Care and Use Committee and were in compliance with the guidelines set forth in the United States Public Health Service Guide for the Care and Use of Laboratory Animals.
Preparatory surgery
At the outset of the training period, each monkey underwent sterile surgery under general anesthesia maintained with isofluorane inhalation. The top of the skull was exposed, bone screws were inserted around the perimeter of the exposed area, a continuous cap of rapidly hardening acrylic was laid down so as to cover the skull and embed the heads of the screws, a head-restraint bar was embedded in the cap, and scleral search coils were implanted on the eyes, with the leads directed subcutaneously to plugs on the acrylic cap (Robinson 1963). After initial training, recording chambers were implanted into the acrylic. At each selected site, a 2-cm-diameter disk of acrylic and skull was removed. A cylindrical recording chamber was cemented into the hole with its base just above the exposed dural membrane. The medial chambers placed over SEF and SMAr were centered on the midline of the brain approximately 21 mm anterior to the Horsley–Clarke interaural plane. The lateral chambers placed over PFC, FEF, and PM were centered approximately at anterior 23 mm and lateral 23 mm.
Memory-guided saccade task
The aim of this task was to allow characterizing the spatial selectivity of each neuron. The monkeys performed memory-guided saccades (MGS) to 6 targets forming a hexagonal array at an eccentricity of 10° (Fig. 1A). Each trial began with the monkey's fixating a central spot. At 500 ms after attainment of fixation, the 6 targets appeared. After an additional 300 ms a cue was flashed on one of the targets for 250 ms. After a random delay in the range of 500 to 1,000 ms, the fixation spot was extinguished, whereupon the monkey had to make a saccade directly to the previously cued target. Trials involving the 6 targets were interleaved in pseudo-random order. Testing continued until it was possible to identify the target eliciting maximal activity. Subsequent testing in the Variable Reward task involved this target and the one diametrically opposite with respect to the fixation point (Fig. 1A, 1 and 1' or 2 and 2' or 3 and 3').
|
Variable reward task
The monkeys performed an MGS task in which a cue presented early in the trial predicted a big (0.3 ml) or a small (0.1 ml) juice reward. Essential features of the task are summarized in Fig. 1. Each trial began with the onset of a central fixation spot (Fig. 1B). At a delay of 50 ms after attainment of fixation, the spot was transformed to a cue whose shape and color signified the magnitude of upcoming reward (Fig. 1C). After 400 ms 2 targets appeared (Fig. 1D) at diametrically opposed locations. A directional cue identical to the fixation cue except in size was then presented for 250 ms in superimposition on one of the targets (Fig. 1E). After a 1,500-ms delay period (Fig. 1F), the fixation spot was extinguished (Fig. 1G), whereupon the monkey was required to make a saccade directly to the previously cued target (Fig. 1H) and to maintain fixation on it for 300–450 ms after saccade completion until delivery of a reward of the predicted magnitude (Fig. 1I). Big and small rewards were unambiguously identifiable because the solenoid valve clicked 3 times (at intervals of 100 ms) in the former case and once in the latter. There were 4 possible conditions representing all possible combinations of reward (big or small) and direction (preferred or antipreferred). The conditions were interleaved in pseudo-random order. To prevent confounding reward selectivity with selectivity for the visual properties of the cues, the cue convention was reversed after each block of 40 successful trials. The collection of data from a given neuron commonly continued until 80 trials had been completed successfully.
Stimuli
The fixation spot was a 0.38° white square presented at the center of the screen. Targets were 0.38° white squares presented 10° from central fixation. The central reward cues, which spanned 0.96°, were an orange + and a green x. The pairing of the cues with large reward and small reward was reversed after each block of 40 successful trials. The directional cue shared all of the properties of the foveal reward cue with the exception that it spanned 1.32°. The background of the display had a luminance of 1.5 cd/m2 and CIE x and y chromaticity coefficients of 0.26 and 0.26, respectively. White stimuli had a luminance of 126.5 cd/m2 and CIE x and y chromaticity coefficients of 0.28 and 0.32, respectively. Orange stimuli had a luminance of 110.7 cd/m2 and CIE x and y chromaticity coefficients of 0.47 and 0.51, respectively. Green stimuli had a luminance of 116.5 cd/m2 and CIE x and y chromaticity coefficients of 0.25 and 0.66, respectively.
Single-neuron recording
At the beginning of each day's session, a varnish-coated tungsten microelectrode with an initial impedance of several megohms at 1 kHz (Frederick Haer, Bowdoinham, ME) was advanced vertically through the dura into the immediately underlying cortex. The electrode could be placed reproducibly at points forming a square grid with 1 mm spacing (Crist et al. 1988). The action potentials of a single neuron were isolated from the multineuronal trace by means of an on-line spike-sorting system using a template-matching algorithm (Signal Processing Systems, Prospect, Australia). The spike-sorting system, on detection of an action potential, generated a pulse whose time was stored with 1-ms resolution.
Electromyographic measurements
Adhesive surface electrodes were placed on the shaved skin overlying the right splenius capitus and masseter muscles. The voltage threshold was set as low as possible, subject to the constraint that the voltage did not cross threshold at rest. Muscle activity was stored as time-marked records of threshold crossings. From these, histograms were constructed off-line, representing the mean instantaneous threshold-crossing rate as a function of time during the trial under each of 4 trial conditions defined by size of reward (big or small) and direction of response (preferred = ipsiversive or antipreferred = contraversive as defined relative to the anatomical location of the muscle).
Experimental control and data collection
All aspects of the behavioral experiment, including presentation of stimuli, monitoring of eye movements, monitoring of neuronal activity, and delivery of reward, were under the control of a pentium-based computer running Cortex software provided by R. Desimone, Laboratory of Neuropsychology, National Institute of Mental Health. Eye position was monitored by means of a scleral search coil system (Riverbend Instruments, Birmingham, AL). The X and Y coordinates of eye position were stored with 4-ms resolution. Stimuli generated by an active matrix LCD projector were rear-projected onto a frontoparallel screen 25 cm from the monkey's eyes. Reward in the form of 0.1 or 0.3 ml of water or juice was delivered through a spigot under control of a solenoid valve.
Analysis of the dependency of behavior on predicted reward
We used paired t-tests to compare, across sessions, the session means of the following measures obtained on big-reward versus small-reward trials: reaction time, error rate, and fixation-break rate. Reaction time was defined as the delay from offset of the fixation spot to the moment when the eye left the central fixation window. Error rate was defined as the number of trials on which a saccade was directed to the wrong target expressed as a percentage of all trials on which a saccade was directed to either target. Fixation-break rate was defined as the percentage of all trials on which the eye left the central fixation window before offset of the fixation spot.
Analysis of the dependency of firing rate on task factors
We employed 2-factor ANOVAs to analyze the impact of reward size and response direction on the firing rate of each neuron. We independently analyzed data from 7 trial epochs: 1) from reward cue onset to directional cue onset (700 ms), 2) from onset to offset of the directional cue (250 ms), 3) 250 ms beginning with directional cue offset, 4) 250 ms before fixation spot offset, 5) 200 ms before saccade initiation, 6) from saccade onset to 100 ms after saccade completion, and 7) 100 ms before to 100 ms after initiation of reward delivery. In all tests, the criterion for statistical significance was taken as P 
0.05.
Assessing contribution of behavioral reaction time to reward-related activity
To determine whether neuronal activity continued to depend on reward size when the effects of behavioral reaction time were factored out, we performed a multivariate regression analysis, fitting three models:
where Y is the firing rate measured from onset of the reward cue to offset of the fixation spot and RT is the behavioral reaction time. The variable REWARD was set to 1 or 0 for trials with large or small rewards, respectively. To determine whether adding the variable REWARD produced a significant improvement in performance, we compared model 3 to model 1. To determine whether adding the variable RT produced a significant improvement, we compared model 3 to model 2. Significance was assessed with an F-test using
 |
0.05. Localization of recording sites
To characterize the location of the recording sites relative to gross anatomical landmarks, we projected the sites onto structural MR images. The images were collected by use of a Brükker 4.7 T magnet in which the anesthetized monkey was supported by an MR-compatible stereotaxic device. Fiducial marks made visible by means of a contrast agent included the centers of the ear bars and selected locations inside the recording chamber. Frontoparallel slices of 2 mm thickness spanning the entire brain were collected. In addition, slices of 2 mm thickness were collected parallel to the cortical surface underlying each lateral chamber. To determine the location of recording sites relative to functional divisions of cortex, we mapped out regions under each chamber from which motoric responses (eye, face, and limb movements) could be elicited at low threshold (40 µA) by electrical microstimulation (1.65-ms biphasic pulses delivered through the recording microelectrode at a frequency of 300 Hz in trains 200 ms long).
|
|
RESULTS |
|---|
|
After completion of training, during the period when neuronal data were being collected, all monkeys performed stably at a high level, selecting the correct target on >98% of all trials in which one target or the other was selected. Furthermore, all monkeys were sensitive to the cues indicating whether a large or small reward would be delivered, exhibiting greater engagement with the task on big-reward trials. This was evident in 3 behavioral measures computed for every neuronal data collection session. The error rate (percentage of trials when the incorrect target was selected relative to all trials when one target or the other was selected) was lower on big-reward (0.6%) than on small-reward (1.3%) trials (Fig. 2A). This trend was present in data from every monkey, achieving significance (2-tailed paired t-test, P < 0.05) in 3 out of 4 cases (Fig. 2D). Furthermore, it was highly significant in data collapsed across monkeys (P < 0.0001). The average behavioral reaction time was shorter on big-reward (232 ms) than on small-reward (239 ms) trials (Fig. 2B). This trend achieved significance (2-tailed paired t-test, P < 0.05) in every monkey (Fig. 2D). Furthermore, it was highly significant in data collapsed across monkeys (P < 0.0001). To analyze the impact of reward on the frequency of fixation breaks, we first asked how fixation breaks were distributed across time during the trial under both big- and small-reward conditions. The results, shown in Fig. 2C, indicate 1) that fixation breaks were more frequent under small- than under big-reward conditions and 2) that the tendency to break fixation declined over the course of the trial under both conditions. To determine whether the effect was significant, we compared the fixation-break frequencies (number of trials prematurely terminated by cessation of fixation expressed as a percentage of all trials) observed under small- and big-reward conditions in each monkey (Fig. 2D). The tendency for fixation breaks to occur more often under small-reward conditions was present and significant in every monkey (2-tailed paired t-test, P < 0.05) and was highly significant in data collapsed across monkeys (P < 0.001).
|
|
We recorded from superficial cortex underlying 3 chambers placed over the lateral frontal cortex of the right hemisphere (monkeys P, N, and F) and 3 chambers centered over midline frontal cortex (monkeys A, N, and F). To assign recording sites to specific areas, we noted the location of the chamber relative to gyri and sulci visible in MR images, using slices parallel to the cortical surface in the case of the lateral chambers and coronal slices in the case of midline chambers (Fig. 3A). We also mapped out the cortex under each chamber by means of microstimulation, noting whether movements of the eyes, face, or limbs were elicited at currents <40 µA (Fig. 3, B and C). We assigned recording sites to 6 areas according to the following criteria. Dorsolateral prefrontal cortex (PFC): a region in front of the arcuate sulcus and surrounding and within the principal sulcus in which microstimulation did not elicit movements. Frontal eye field (FEF): a region rostral to and in the anterior bank of the arcuate sulcus in which microstimulation elicited saccades and not movements of the face or limbs. Recording sites in the FEF were all within 4 mm of the cortical surface at locations where microstimulation at the corresponding depth elicited eye movements. Premotor cortex (PM): a region caudal to the arcuate sulcus in which microstimulation elicited face or limb movements and not saccades. Transitional cortex (FEF/PM): a region caudal to the pure saccade zone and rostral to the pure face/limb zone in which electrical stimulation elicited both saccades and movements of the face or limbs. Recording sites in FEF/PM were all within 4 mm of the cortical surface at locations where microstimulation at the corresponding depth elicited eye and face/limb movements. On the grounds of its location behind the arcuate sulcus, this cortex belongs to the premotor area. However, we have designated it as an independent zone with the possibility in mind that its distinct traits, as revealed by electrical stimulation, might be accompanied by some differential form of sensitivity to reward-predicting cues. The finding of an oculomotor representation in PM is not without precedent (Fujii et al. 1998, 2000). The supplementary eye field (SEF) is a region located rostral to the genu of the arcuate sulcus and extending 2–5 mm from the hemispheric midline, in which microstimulation elicited saccades. The rostral supplementary motor area (SMAr) is a region immediately caudal to the SEF in which microstimulation elicited movements of the face and limbs.
|
Selection of neurons for study
Neuronal activity was first monitored in the context of the MGS task with reward size fixed and with targets at 6 locations spaced at 60° intervals around fixation (Fig. 1A: locations 1, 1', 2, 2', 3 and 3'). Any neuron appearing to exhibit task-related activity in the MGS task was selected for study in the Variable Reward task (Fig. 1, B–I). Out of the 6 targets used in the MGS task 2 targets were selected for use in the Variable Reward task: the target associated with strongest neuronal activity and the target diametrically opposite this one. Thus the pair of targets employed in the Variable Reward task could be either 1 and 1', 2 and 2', or 3 and 3' (Fig. 1A). Neuronal activity was monitored while the monkey performed 20 successful trials under each of 4 conditions (2 directions x 2 reward levels) in the Variable Reward task.
Organization of single-neuron results
Neurons in many frontal areas exhibited reward-related activity. This activity commonly took the form of a main effect (with net firing rate higher or lower on big-reward trials) and less frequently took the form of an interaction effect (with the strength of the directional signal stronger or weaker on large-reward trials). Both forms of effect were present and significant in the neuron of Fig. 4. Its net firing rate was clearly higher when a large reward was expected (top row vs. bottom row). In addition, its directional signal, the difference in firing rate between trials requiring a leftward response (left column) and those requiring a rightward response (right column), was greater under the big-reward condition. For each cortical area, we analyze the nature and rate of incidence of such effects by proceeding through 3 steps. 1) Population histograms. The aim of this step is to indicate qualitatively how the magnitude of expected reward affected the population firing rate and the population directional signal. 2) Individual neurons by epoch. The aim of this step is to indicate whether effects evident at the level of the population were statistically demonstrable at the level of individual neurons. For each of 7 epochs spanning the duration of the trial, we indicate how many neurons showed significant increases or decreases in firing rate on big-reward as compared with small-reward trials, and how many showed significant increases or decreases in the strength of the directional signal. 3) Individual neurons across trial. To complement the statistically insensitive analysis by epoch, we also describe a more robust analysis based on the trial as a whole, indicating in how many neurons the firing rate and directional signal were significantly affected by the size of the expected reward. The details of each step of analysis and the conventions for presenting the results are laid out in the course of the next section, on the PFC, which will thus serve as a guide to later sections.
|
Prefrontal cortex (PFC)
POPULATION. We collected data from 201 neurons in the PFC of 2 monkeys (Table 1). As a basis for qualitative assessment of the effect of anticipated reward on the activity of these neurons, we constructed population curves representing firing rate as a function of time under the 4 trial conditions (Fig. 5A). In this display, thick and thin lines represent population activity on trials requiring responses in the preferred and antipreferred directions, respectively. Neuronal activity was strongly affected by response direction as indicated by the consistent elevation of thick above thin lines. Subtle effects of the size of the predicted reward are manifest as differences in firing rate between trials in which the response direction (indicated by line thickness) was the same but reward level (indicated by color) was different. To characterize the time course of reward-related activity, we computed independently the impact of reward on net firing rate independent of direction and on the directional signal. The effect on net firing rate was represented by an index corresponding to the average amount by which the firing rate increased under the big-reward condition. It was computed as (BP + BA – SP – SA)/2, where BP is the firing rate under the big-reward, preferred-direction condition, SA is the firing rate under the small-reward, antipreferred-direction condition, and so on. Over the course of the trial from onset of fixation to initiation of response, this index was more often positive than negative (Fig. 5B), indicating a subtle tendency for firing to be stronger when a big reward was expected. The effect on the directional signal was represented by an index that corresponded to the average amount by which big-reward caused the firing rate to increase on preferred-direction trials and to decrease on antipreferred-direction trials. It was computed as (BP – BA – SP + SA)/2. This index was consistently greater than zero during the period between presentation of the directional cue and execution of the saccade (Fig. 5D).
|
INDIVIDUAL NEURONS BY EPOCH. To determine whether effects present in the population were also observable at the level of individual neurons, we analyzed data from each neuron during 7 trial epochs (I–VII) defined in METHODS and depicted along the time line at the base of Fig. 5. For each epoch, we carried out an ANOVA with firing rate as the dependent variable and with reward size and response direction as factors. Counts of neurons exhibiting significant main effects of reward size on firing rate are shown in Fig. 5C, where blue (or red) symbols represent the percentage of cases in which firing was increased (or decreased) for big compared with small reward. The counts of significant effects were only slightly elevated above the 5% expected by chance in light of the criterion employed for judging statistical significance (P < 0.05). It is true that during preresponse epochs (I–V), the count of neurons firing more strongly for big reward (blue) was consistently higher than the count of neurons showing the opposite effect (red), but the difference in counts achieved significance only during epoch IV (
2 test, P < 0.05). Counts of neurons exhibiting a significant interaction between reward size and direction are shown in Fig. 5E, where blue (or red) symbols represent the percentage of cases in which the directional signal was stronger (or weaker) for big reward. Note that the counts indicated during epoch I must represent type 1 errors because it was only after this epoch that the directional instruction was delivered. With the epoch I counts as a basis for comparison, it is clear that reward x direction interaction effects were rare or absent in all epochs. There was an apparent slight elevation, during epochs III–V, in the number of neurons showing enhanced direction selectivity under big reward (blue) relative to those showing reduced direction selectivity (red). In no epoch, however, did this difference achieve significance. Thus analyses based on brief epochs in single neurons lack the sensitivity to detect effects of predicted reward evident in PFC population activity.
INDIVIDUAL NEURONS ACROSS TRIAL. To achieve a more robust statistical measure of the impact of predicted reward size on the neuronal firing rate, we carried out an ANOVA using, as the dependent variable, the mean firing rate across the entire period from onset of the reward cue to execution of the saccade, and employing as factors both reward size and response direction. This analysis revealed a significant main effect of predicted reward size in 15% of PFC neurons. The majority of these neurons fired more strongly under big-reward conditions (Table 1). This effect was not significantly different between monkeys and was significant in the data from the 2 monkeys combined (2 test, P < 0.01). To assess the impact of reward on the directional signal, we carried out an identical procedure with the exception that the measurement period began with onset of the direction cue. This analysis revealed a significant interaction between the size of the predicted reward and the direction of the response in 5% of PFC neurons. Among neurons exhibiting a significant interaction effect, those carrying stronger directional signals under the big-reward condition outnumbered those carrying a weaker signal (Table 1). This effect was not significantly different between monkeys and was significant in the data from the 2 monkeys combined (2 test, P < 0.05).
SUMMARY. Among PFC neurons, expectation of a large reward led to a subtle elevation in firing rate and a subtle enhancement of directional signals. These effects were evident both in population activity and in counts of neurons showing statistically significant effects over an epoch spanning the full extent of the trial.
Frontal eye field (FEF)
POPULATION. We collected data from 122 neurons in the FEF of 3 monkeys (Table 1). Curves representing the activity of this population as a function of time during the trial indicate that the level of anticipated reward exerted a moderate effect on neuronal activity (Fig. 6A). The net firing rate was elevated from shortly after presentation of the reward cue until onset of the saccade (Fig. 6B). The directional signal was elevated from presentation of the directional cue until onset of the saccade (Fig. 6D).
|
INDIVIDUAL NEURONS BY EPOCH. Neurons firing significantly more strongly under the big-reward condition (blue symbols) outnumbered neurons firing more weakly (red symbols) during epochs I–V (Fig. 6C). This trend was significant during epoch III (2 test, P < 0.05). In few neurons was there a significant interaction between reward size and response direction (Fig. 6E). In no epoch did the difference in counts between neurons showing stronger and weaker direction selectivity under the big reward condition (blue and red symbols in Fig. 6E) achieve significance.
INDIVIDUAL NEURONS ACROSS TRIAL. In 16% of FEF neurons, the net firing rate averaged across the entire trial was significantly dependent on reward size. Of these neurons, a majority fired more strongly under big-reward conditions (Table 1). This effect was not significantly different between monkeys and was significant in the data from the 3 monkeys combined (2 test, P < 0.0001). Only 2 neurons (2%) exhibited a significant interaction of reward size and directional signal. Both carried stronger direction signals under the big-reward condition (Table 1).
SUMMARY. The size of the anticipated reward exerted a moderate effect on neuronal activity in the FEF. Anticipation of a big reward led to enhanced firing, as revealed both by population measures and by counts of neurons showing significant effects of reward size. It led also to an enhancement of directional activity evident in population measures but too small to emerge from statistical tests on the activity of individual neurons.
Transitional cortex (FEF/PM)
POPULATION. We collected data from 46 neurons in the FEF/PM transition zone of two monkeys (Table 1). Curves representing the activity of this population as a function of time during the trial indicate that the level of anticipated reward exerted a strong effect on neuronal activity (Fig. 7A). The net firing rate was sharply elevated from shortly after presentation of the reward cue until completion of the saccade (Fig. 7B). The directional signal was moderately elevated from presentation of the directional cue until completion of the saccade (Fig. 7D).
|
INDIVIDUAL NEURONS BY EPOCH. Neurons firing significantly more strongly under the big-reward condition (blue symbols) dramatically outnumbered neurons firing more weakly (red symbols) during epochs I–VI (Fig. 7C). The trend was significant (by a 2 test) during epochs I (P < 0.05), III (P < 0.01), IV (P < 0.001), V (P < 0.01), and VI (P < 0.01). Neurons in which direction selectivity was significantly greater under the big-reward condition markedly outnumbered those in which it was weaker during epochs III–IV (Fig. 7E). The trend was significant (by a 2 test) in epoch III (P < 0.01) and epoch IV (P < 0.05).
INDIVIDUAL NEURONS ACROSS TRIAL. The firing rate averaged across the entire trial was significantly dependent on reward size in 43% of FEF/PM neurons. Of these, a majority fired more strongly under the big-reward condition (Table 1). This effect was not significantly different between monkeys and was significant in the data from the 2 monkeys combined (2 test, P < 0.0001). Five neurons (11% of the sample) exhibited a significant interaction of reward size and directional signal; 4 of these carried stronger direction signals under the big-reward condition (Table 1).
SUMMARY. Increasing the size of the anticipated reward markedly enhanced the mean firing rate and moderately enhanced the strength of the directional signal in the FEF/PM transition zone. The firing rate increase was evident both in measures of population activity and in counts of neurons showing statistically significant effects. The increase of direction selectivity was evident primarily in population measures.
Premotor cortex (PM)
POPULATION. We collected data from 83 neurons in the PM of 2 monkeys (Table 1). Curves representing the activity of this population as a function of time during the trial indicate that the level of anticipated reward exerted a strong effect on neuronal activity (Fig. 8A). The net firing rate was sharply elevated from shortly after presentation of the reward cue until completion of the saccade (Fig. 8B). The directional signal was moderately elevated from presentation of the directional cue until completion of the saccade (Fig. 8D).
|
INDIVIDUAL NEURONS BY EPOCH. Neurons firing significantly more strongly under the big-reward condition (blue symbols) dramatically outnumbered those firing more weakly (red symbols) during epochs I–V. The trend was significant (by a 2 test) during epochs I (P << 0.0001), II (P < 0.001), III (P << 0.0001), IV (P < 0.001), and V (P < 0.01). In contrast, few neurons exhibited an interaction between reward size and direction (Fig. 8E). In no epoch was there a significant difference between the counts of neurons exhibiting enhanced direction selectivity when a big reward was expected (blue symbols in Fig. 8E) and those showing the opposite effect (red symbols).
INDIVIDUAL NEURONS ACROSS TRIAL. The firing rate averaged across the entire trial was significantly dependent on reward size in 52% of sampled PM neurons. Of these, a large majority fired more strongly under big-reward conditions (Table 1). This effect was not significantly different between monkeys and was highly significant in the data from the 2 monkeys combined (2 test, P << 0.0001). Out of 6 neurons exhibiting a significant interaction of reward size and direction (Table 1), 3 showed stronger and 3 weaker directional signals under the big-reward conditions.
SUMMARY. Increasing the size of the anticipated reward strongly enhanced the mean firing rate of neurons in PM. This was evident in the population data and in counts of neurons showing significant effects. Expectation of a large reward led to only a weak enhancement of the strength of the directional signal.
Supplementary eye field (SEF)
POPULATION. We collected data from 164 neurons in the SEF of 2 monkeys (Table 1). Curves representing the activity of this population as a function of time during the trial indicate that the level of anticipated reward exerted only a very small effect on neuronal activity (Fig. 9A). The net firing rate tended to be lower under the big-reward conditions (Fig. 9B), whereas the directional signal tended to be stronger (Fig. 9D).
|
INDIVIDUAL NEURONS BY EPOCH. The number of neurons exhibiting significant effects of reward size on firing rate (Fig. 9C) was very small. In no epoch was there a significant tendency for neurons firing significantly more under the big-reward conditions to predominate over those firing less or vice versa. During epoch II, neurons in which direction selectivity was significantly stronger under the big-reward condition significantly outnumbered those in which it was weaker (2 test, P < 0.05).
INDIVIDUAL NEURONS ACROSS TRIAL. The firing rate averaged across the entire trial was significantly dependent on reward size in 11% of sampled neurons. Among these, 8 fired more under the big-reward condition and 10 fired less. Of two neurons (1% of the sample) exhibiting an interaction between reward and direction, one showed enhanced and the other reduced direction selectivity under big reward.
SUMMARY. Increasing the size of the anticipated reward exerted a very subtle effect on neuronal activity in the SEF. This took the form of a slight reduction of firing rate and a slight enhancement of the directional signal clearly evident only at the population level.
Rostral supplementary motor area (SMAr)
POPULATION. We collected data from 88 neurons in the SMAr of one monkey (Table 1). Curves representing the activity of this population as a function of time during the trial indicate that the level of anticipated reward exerted marked effects on neuronal activity (Fig. 10A). There was a substantial increase of firing rate from shortly after the reward signal until execution of the saccade on big-reward trials (Fig. 10B). Furthermore, under big reward conditions, the directional signal was moderately stronger (Fig. 10D).
|
INDIVIDUAL NEURONS BY EPOCH. Neurons firing significantly more strongly under the big-reward condition (blue symbols) outnumbered those firing more weakly (red symbols) during epochs I–V (Fig. 10C). This trend achieved significance during epoch IV (2 test, P < 0.05). During no epoch was there a significant difference between the counts of neurons in which directional signals were enhanced or reduced under the big-reward condition (Fig. 10E).
INDIVIDUAL NEURONS ACROSS TRIAL. The firing rate averaged across the entire trial was significantly dependent on reward size in 32% of neurons. Among these, 24 fired more strongly under the big-reward condition and 4 fired more weakly. The difference in these counts was significant (2 test, P < 0.001). Of 5 neurons exhibiting an interaction between reward and direction, 3 showed enhanced direction selectivity under the big-reward condition and 2 showed reduced selectivity.
SUMMARY. Increasing the size of the anticipated reward induced a strong enhancement of firing rate among SMAr neurons. This was evident in both population measures and counts of neurons showing significant effects of reward size. A weak enhancement of directional signals was evident only at the population level.
Comparison among areas
To determine whether the impact of anticipated reward varied systematically across areas, we carried out area-to-area comparisons based on counts of neurons in which firing depended significantly on reward, direction, and a reward x direction interaction (Table 1; Fig. 11). Several systematic interareal differences were clearly evident on comparison.
|
REWARD ENHANCEMENT. Neurons firing significantly more strongly under the big-reward condition became steadily more frequent with progress in a posteriorward direction across the cortical surface (Fig. 11A, blue bars). This was true in both the lateral frontal lobe (PFC, FEF, FEF/PM, and PM) and the medial frontal lobe (SEF and SMAr). Most pairwise interareal differences in the frequency of neurons exhibiting stronger firing on big-reward trials were highly significant (Table 2). Even among neurons selected on the basis of firing significantly more strongly under the big-reward condition, the strength of the signal tended to increase in a posteriorward direction (4.5, 5.1, 5.3, and 5.2 spikes/s in PFC, FEF, FEF/PM, and PM, respectively, and 3.6 and 4.3 spikes/s in SEF and SMAr, respectively).
|
REWARD SUPPRESSION. Neurons firing significantly more strongly under the small-reward condition were observed infrequently and in a pattern that did not seem to reflect systematic differences between areas (Fig. 11A, red bars). Only one pairwise interareal comparison (FEF vs. SEF) revealed a statistically significant difference in the frequency with which such neurons were observed (2 test, P < 0.05).
DIRECTION SELECTIVITY. Neurons exhibiting a statistically significant dependency of firing rate on response direction (blue and red bars in Fig. 11B) were more numerous in the eye fields (FEF and SEF) than in other areas. The frequency of such neurons (as measured by a 2 test) was significantly higher in the FEF than in the PFC (P < 0.01) and PM (P < 0.05) and was significantly higher in the SEF than in the PFC (P < 0.0001), PM (P << 0.0001), and SMAr (P < 0.05). The distribution across areas of direction selective neurons was highly significantly different from the distribution across areas of neurons firing more strongly under big-reward conditions (2 test, P << 0.0001).
IMPACT OF REWARD ON DIRECTIONAL SIGNALS. Neurons exhibiting a statistically significant dependency of firing rate on the interaction between reward size and response direction were rare in all areas (Fig. 11C). The frequency with which these neurons were observed in the total sample (0.043) was in fact no greater than the frequency of type 1 errors expected by chance (0.05). Thus statistical tests carried out on single neuron data do not afford independent support for the observation based on the population histograms (Figs. 5, 6, 7, 8, 9, 10) that the directional signal tended to be stronger under big-reward conditions and that the strength of this effect varied to some degree across areas.
Location of reward sites relative to gross morphological landmarks
To analyze the fine distribution of neurons exhibiting reward effects, we projected recording sites onto the cortical surface. For 3 lateral chambers (in monkeys P, N, and F), we collected MR images parallel to the cortical surface. Using a set of slices that contained both the cortex and fiducial markers at known locations relative to the recording grid, we determined the locations of recording sites relative to the arcuate (AS) and principal (PS) sulci. The results are shown in Fig. 12, A–C. In this figure, the size of each symbol indicates the proportion of neurons at the corresponding site in which there was a significant enhancement of firing rate on big-reward trials. The general tendency for reward enhancement to occur at posterior sites, and, in particular, at sites behind the arcuate sulcus, is clear. In the vicinity of the FEF and FEF/PM, neurons represented by a single symbol occupied depths ranging from 0 to 4 mm (mean = 2.23 mm; median = 2.00 mm). However, there was no consistent trend with respect to depth. Neurons showing a significant effect of reward size were observed throughout the range of recording depths (mean = 2.34 mm; median = 2.25 mm). For 3 midline chambers (in monkeys A, N, and F), we collected frontoparallel MR images showing the cortex and fiducial markers at known locations relative to the recording grid. These images were used to determine the locations of recording sites relative to the interhemispheric midline and the frontal plane containing the genu of the arcuate sulcus (AS genu). The results are shown in Fig. 12, D–F. The general tendency for reward enhancement to occur at relatively posterior sites—in monkey F—as contrasted to relatively anterior sites—in monkeys A and N—is clear.
|
Impact on reward-related activity of reversing the cue-reward associations
At the end of every 40 successful trials in the Variable Reward task, the cue previously associated with big reward became associated with small reward and vice versa. Consequently, in each 80-trial data collection session, there was one block conforming to each cue convention. This manipulation possessed the virtue of allowing us to consider the influence of reward size on neuronal activity independently of any selectivity neurons may have possessed for the visual attributes of the stimuli. However, it may have resulted in an attenuation of reward-related activity. This would be true if it took monkeys many trials to adjust their expectations after each switch. We addressed this concern by asking how long it took monkeys to adjust to the new cue-reward contingencies after a switch.
To do so, we assessed, as a function of trial number relative to the time of the switch, the effect of predicted reward on behavioral reaction time. To achieve adequate power in this analysis, we combined reaction time measures across all data collection sessions in all monkeys, considering only those trials that the monkey completed successfully, receiving a reward and thus acquiring information about the current relation between the cues and the reward sizes. The results are shown in Fig. 13A. This plots, as a function of trial number, the reaction time to the preswitch big-reward cue minus the reaction time to the preswitch small-reward cue. The values were negative before the switch because the monkeys responded more swiftly on trials when big reward was predicted. The values became positive after the switch for the same reason. The shift to reliable positivity was established by the third postswitch trial. Thus on average, it took the monkeys only 2 trials to register in their performance the switched cue-reward contingencies.
|
We also assessed how quickly neuronal reward-related activity was reestablished after the switch. To do so, we combined data from all neurons with a significant tendency to fire more strongly during big-reward trials, collapsing the data across areas and monkeys. We restricted consideration to trials that the monkey completed successfully, receiving reward and thus receiving information about the cue-reward contingencies. The results are shown in Fig. 13B. This plots, as a function of trial number, the mean firing rate on trials involving the preswitch big-reward cue minus the mean firing rate on trials involving the preswitch small-reward cue. The values were positive before the switch because the neurons fired more strongly on big-reward trials and negative after the switch for the same reason. The shift to reliable negativity was established by the third trial after the switch.
It is conceivable that neuronal activity after the switch might have adjusted at different rates in different areas. To assess this possibility, we analyzed data from each area separately. To compensate for the smaller sample sizes, we carried out a coarser analysis, analyzing firing rates on blocks of 4 consecutive correct trials, with blocks demarcated so that the time of the switch fell at a between-block boundary. The results are shown in Fig. 13, C–G. These plots indicate that neuronal activity adjusted quickly to the new rule regardless of area. Furthermore, insofar as there were interareal differences in the rate of adjustment, these could not explain interareal differences in the frequency of reward-related activity. On the contrary, area PM, in which neuronal activity was maximally affected by reward (Fig. 11A), was apparently the slowest to adjust to the switch (Fig. 13F). We conclude that the rule-switching design did not lead to a major attenuation of reward-related neuronal activity.
Relation of reward-related activity to other functional properties of neurons
It might be the case that reward-related activity is correlated with the presence of certain other functional properties in neurons, for example, visual responsiveness, delay period activity, or perisaccadic firing. To assess this possibility, we compared the patterns of task-related activity of neurons exhibiting a significant increase in firing rate under the big-reward condition (Table 1, B > S) and of all other neurons. The classification of neurons was effective as reflected by the fact that neurons in the reward-sensitive category exhibited dramatically enhanced firing on big-reward trials (Fig. 14A, blue vs. red curves), whereas neurons in the reward-insensitive category (Fig. 14B) did not. To compare the functional properties of neurons in the 2 groups, we constructed area-specific population histograms. We restricted consideration to trials in which the saccade was in the neuron's preferred direction because patterns of task-related activity are most evident under this condition. We considered small-reward trials because the average firing rates of reward-sensitive and reward-insensitive neurons were most similar when a small reward was anticipated (Fig. 14A, red curves, vs. Fig. 14B, red curves). The results are shown in Fig. 14, C–H. In general, reward-sensitive neurons (green curves) and reward-insensitive neurons (orange curves) exhibited phasic increases of activity at the same times during the trial. The similarity was most marked in areas FEF/PM and PM (Fig. 14, E and F), which contained the largest proportion of reward-sensitive neurons and in which sampling noise consequently was lowest. There was, however, one factor with respect to which the 2 groups of neurons did differ consistently across areas. Reward-sensitive neurons, considered as a population, fired more strongly than reward-insensitive neurons during the period extending from completion of the saccade to delivery and ingestion of the reward. This effect could have been associated with either of 2 processes. First, the reward-sensitive population might have been active in conjunction with the preparation and execution of consummatory movements. Second, the firing rate of reward-sensitive neurons might have been correlated with the intensity of reward anticipation and this might have peaked late in the trial. To choose between these possibilities would require additional experiments. Our inability to distinguish between them on the basis of the present results reflects the general inability of variable-reward tasks to differentiate between neuronal activity that represents reward value and neuronal activity that reflects motor preparation and output.
|
The analysis described above was confined to data from trials in which the response was in the neuron's preferred direction and therefore could not reveal any dependency between reward sensitivity and direction selectivity. To assess the relation between these functional properties, we carried out an analysis of data from areas containing the highest proportion of reward-sensitive neurons (FEF/PM and PM). Among 129 neurons, 13 were modulated by both reward size and response direction, 50 were modulated by reward only, 13 were modulated by direction only, and 53 were modulated by neither. A 2 test on this distribution indicated that there was no correlation between the 2 traits (P > 0.8).
Relation of reward-related activity to saccadic reaction time
Behavioral reaction times were significantly shorter on big-reward than on small-reward trials in all monkeys. This raises the question: Was reward-related neuronal activity directly correlated with the size of the expected reward or was it, alternatively, directly correlated with the behavioral reaction time and correlated with reward only through this dependency? In the striatum, reward-related activity cannot be accounted for purely in terms of reaction time (Hollerman et al. 1998); however, this conclusion does not necessarily apply to cortex. To resolve this issue, for each neuron, we used a multiple least-squares regression approach to optimize the parameters of 3 models representing firing rate as a linear function of reaction time and reward size: 1) a reduced model incorporating reaction time only, 2) a reduced model incorporating reward size only, and 3) a full model incorporating both. This analysis was based on the entire trial period from onset of the reward cue to initiation of the saccadic response because shorter epochs offered too little statistical power. It was carried out independently for trials involving each neuron's preferred and antipreferred directions because the relation between reaction time and neuronal activity might vary as a function of response direction. We compared each of the reduced models to the full model using a nested F-test (see METHODS). Then we calculated the percentage of neurons in each area that showed a significant improvement of fit only when the variable of reward was added to the model (Fig. 15, black bars), only when reaction time was added (Fig. 15, white bars), or when either was added (Fig. 15, hatched bars).
|
The results are consistent with data presented in preceding sections in that 1) neurons in which firing rate was positively correlated with reward size outnumbered those in which there was a negative correlation (Fig. 15, upward pointing vs. downward pointing black and darkly hatched bars); 2) the percentage of reward-sensitive neurons was greatest in FEF/PM, PM, and SMAr; and 3) the tendency for firing to be stronger on big-reward trials was present regardless of response direction (Fig. 15, A vs. B). The results also indicate that neuronal activity was related to the behavioral reaction time. In areas where reward effects were common (FEF/PM, PM, and SMAr), neurons that fired more strongly on trials with short reaction times outnumbered those showing the opposite effect (Fig. 15, downward pointing vs. upward pointing white and palely hatched bars). The reward-related and reaction time– related effects were not yoked strongly. Reward-related effects frequently occurred in the absence of reaction time–related effects. Nevertheless, the fact that firing related to reward and reaction time was maximal in the same set of areas suggests that the 2 effects have a common origin. To investigate further the relation between the 2 effects, we computed, for each neuron in FEF/PM, PM, and SMAr, 1) the mean firing rate on big-reward trials minus the mean firing rate on small-reward trials and 2) the slope of a line fitted to points representing firing rate versus reaction time across all trials involving saccades in each direction. We then analyzed the correlation across neurons between the two measures. On preferred-direction trials, the correlation was weak (r2 =0.07) but positive and significant (P < 0.0001). On antipreferred-direction trials, the correlation was weaker still (r2 =0.005) but also significant (P < 0.0001). Thus reward-sensitive neurons tend to fire more when the behavioral reaction time is going to be short but the effect is subtle.
How can we account for the existence of a substantial number of neurons in which the firing rate depends significantly on the size of the anticipated reward and yet is not significantly related to the saccadic reaction time? If these neurons were concentrated in the prefrontal cortex, we might argue that they are outside the motor circuit and are at a level of processing where activity represents reward value rather than motor preparation. However, they are most concentrated in the premotor areas, not in the prefrontal cortex (Fig. 15, black bars), and so are probably involved in motor processes including saccade preparation. There are at least 2 principled explanations for the existence of such neurons. First, they may participate in aspects of movement planning unrelated to reaction time. To take one example, motivational enhancement of their activity might contribute to stabilizing the program for the instructed saccade, protecting it against disruption by competing tendencies. In this case, the advantage conferred by motivational enhancement would become evident only in the presence of distractors. Second, they might be involved in preparation and execution of axial or ingestive movements preceding or accompanying but altogether distinct from the instructed saccade. This possibility is considered further in the DISCUSSION.
EMG measures
Reward-related neuronal activity might have been present during the delay period in PM, FEF/PM, and SMAr because monkeys were engaging in skeletomotor behavior or were preparing to make skeletomotor responses at the trial's end. The monkeys in this study did not engage in consistent overt licking during the delay period, such as has been observed in tasks involving multisecond delays (Hassani et al. 2001). Nevertheless, they may have contracted the muscles of the jaw and neck during the delay period or may have maintained a state of preparation for contraction of these muscles during later phases of the task. Insofar as they did so, the intensity of motor output or motor preparation might have depended on the size of the expected reward. To assess this possibility, we recorded EMG activity in the 3 monkeys (P, N, and F) still available for study after the collection and analysis of single-neuron data, focusing on the splenius capitus, responsible for ipsilateral version and torsion of the head, and the masseter, responsible for jaw closure and a commonly used indicator of oral activity (Apicella et al., 1991; Huang et al. 1989; Murray et al. 1991). We collected data from the splenius capitus during 8, 12, and 10 400-trial sessions and from the masseter during 8, 8, and 9 400-trial sessions in monkeys N, P, and F, respectively. Combining the data for each muscle in each monkey across all trials in all sessions, we analyzed the dependency of the EMG on experimental condition by carrying out an ANOVA with reward and direction as factors for each of the epochs (I–VII in Fig. 16) on which neuronal data analysis had been based.
|
DELAY-PERIOD EMG IN SPLENIUS CAPITUS. The pattern of EMG activation and its dependency on condition varied across monkeys. In monkey P the level of activation was approximately constant throughout the delay period until, just before the saccade, it became slightly higher on small-reward trials (Fig. 16A). This effect was significant (epochs IV–V, P < 0.05). In monkey N the level of activation declined steadily during the delay period and showed no effect of condition (Fig. 16B). In monkey F, activation rose steadily during the delay period, was stronger under big-reward than under small-reward conditions (blue vs. red traces), and was stronger when the impending saccade was in the muscle's "preferred" (ipsiversive) direction than when it was in the muscle's "antipreferred" (contraversive) direction (thick vs. thin lines). The tendency for activity to be stronger when a larger reward was expected was highly significant in all epochs (I–V, P << 0.0001). To judge from the EMG results in monkey F, expectation of a larger reward resulted in greater tonus of the neck muscles and in application of stronger torque to the head.
DELAY-PERIOD EMG IN MASSETER. Effects of predicted reward size on masseter EMG for the most part paralleled those on splenius capitus EMG (Fig. 16, D–F). In monkey P (Fig. 16D), EMG activity was significantly reduced under big-reward conditions immediately before the saccade (epochs IV–V, P < 0.05). In monkey N (Fig. 16E), EMG activity was unaffected by trial condition. In monkey F (Fig. 16F), EMG activity was stronger under big-reward conditions in all delay-period epochs (I–V, P < 0.0001). To judge from the EMG results in monkey F, the expectation of big reward induced stronger contraction of the jaw muscles throughout the delay period.
DELAY-PERIOD EMG IN RELATION TO DELAY-PERIOD NEURONAL ACTIVITY. Can reward-related neuronal activity observed during the delay period in PM, FEF/PM, and SMAr be explained entirely as a correlate of reward-dependent muscle contractions? It seems that the answer to this question is in the negative because monkeys F and N differed dramatically in their patterns of muscle contraction and yet gave closely similar results with respect to neuronal activity in PM and FEF/PM. Muscle contraction tracked reward size dramatically in monkey F (Fig. 16, C and F) and not at all in monkey N (Fig. 16, B and E). However, the incidence of reward-related activity in PM and FEF/PM appeared comparable in the 2 monkeys (Fig. 12, B and C) and indeed was statistically indistinguishable between monkeys (as described in preceding sections on individual areas). We conclude that reward-dependent contractions of the neck and jaw muscles during the delay period were not necessary for the appearance of reward-related delay-period neuronal activity. However, it remains possible that reward size affected preparation for skeletomotor movements occurring after the delay period and that reward-related variations in neuronal activity were correlated with the monkey's state of motor preparation. Lacking a direct measure of preparation to contract the neck and jaw muscles, we utilized an indirect measure, the EMG at the time of the response, on the assumption that motor output at the time of the response was prepared before the time of the response.
RESPONSE-PERIOD EMG IN SPLENIUS CAPITUS. All monkeys exhibited strong reward-dependent EMG activation during the late phase of the trial. In monkey P (Fig. 16A) neck EMG was significantly reduced under big-reward conditions during the postsaccadic epoch (VI, P < 0.05) but was significantly enhanced after reward delivery (epoch VII, P < 0.001). In monkey N (Fig. 16B) EMG activation was significantly enhanced under big-reward conditions from initiation of the saccade onward (epochs VI–VII, P < 0.01). In monkey F (Fig. 16C) this was also true (epochs VI–VII, P << 0.0001).
RESPONSE-PERIOD EMG IN MASSETER. All monkeys exhibited strong reward-dependent EMG activation during the late phase of the trial. In monkey P (Fig. 16D) EMG activation was significantly reduced under big-reward conditions during the postsaccadic epoch (VI, P < 0.05) but was significantly enhanced after reward delivery (epoch VII, P < 0.001). In monkey N (Fig. 16E) EMG activity was significantly enhanced under big-reward conditions only during the period after reward delivery (epoch VII, P < 0.001). In monkey F (Fig. 16F) this was also true (epochs VI–VII, P < 0.0001).
RESPONSE-PERIOD EMG IN RELATION TO DELAY-PERIOD NEURONAL ACTIVITY. All 3 monkeys showed reward-dependent variations in neck and jaw muscle EMG during the late phase of the trial encompassing the epochs of the saccade and reward delivery. The fact that motor output during this phase of the trial varied as a function of reward suggests that the state of motor readiness and movement preparation during the preceding phase of the trial was reward dependent. Reward-dependent neuronal activity, as observed during the delay period in PM, FEF/PM, and SMAr, could thus have been a correlate of reward-induced changes in the level of motor readiness and movement preparation.
|
|
DISCUSSION |
|---|
|
We recorded single-neuron activity in a large set of frontal cortical areas in monkeys performing an ocular delayed response task under conditions in which a large or small reward was expected on successful completion of the trial. The areas studied include PFC, FEF, FEF/PM, PM, SEF, and SMAr. Throughout these areas, we found that the size of the expected reward affected the average delay-period firing rate and, to a lesser extent, the strength with which neuronal activity signaled the direction of the impending response. In general, when a larger reward was expected, the firing rate increased and directional signals became stronger. However, areas were not uniform in this respect. Surprisingly, the incidence of reward-related neuronal activity increased steadily along the anterior–posterior axis in both the lateral frontal lobe (PFC < FEF < FEF/PM < PM) and medial frontal lobe (SEF < SMAr). The robust reward-related activity of neurons in the premotor areas was presumably related to the monkey's motivationally modulated level of motor readiness and movement preparation. This observation highlights the fact that reward-related neuronal activity, in all areas where it has been studied by use of standard behavioral paradigms, is ambiguous insofar as it could represent either the value of the expected reward or the monkey's state of motor preparation and motor output. It is important that alternative approaches be developed that allow assessing independently the impact on neuronal activity of reward value and motor preparation.
Effects of predicted reward in dorsolateral prefrontal cortex
Among 201 neurons located in the PFC, we observed only very weak effects of predicted reward. A few more neurons than expected by chance (11%) showed significant enhancement of the mean firing rate on big-reward trials; furthermore, there was a barely significant trend toward enhanced direction selectivity on big-reward trials. These effects are weaker than observed in several previous studies. We suspect that the discrepancy between our results and results previously described arises in part from our having carefully distinguished PFC from the FEF and PM and in part from our having employed an approach in which large and small reward were contrasted, rather than reward and no reward or a preferred food and a nonpreferred food.
ENHANCEMENT OF MEAN FIRING RATE. Leon and Shadlen (1999), like us using a design in which big reward was contrasted to small reward, and like us distinguishing PFC from the FEF, obtained, in the PFC, results consistent with ours. Out of 91 PFC neurons, 14% exhibited significant enhancement on big-reward trials whereas only 2% showed suppression. These proportions are not significantly different from the ones obtained in the present study. In other studies, data from the PFC and FEF have been combined (Kobayashi et al. 2002); the 2 areas have not been distinguished (Watanabe 1996; Watanabe et al. 2002); or neurons from PFC, FEF, and PM have been combined without distinction (Watanabe 1990, 1992). These studies have either contrasted reward to no reward (Kobayashi et al. 2002; Watanabe 1990, 1992; Watanabe et al. 2002) or have contrasted a preferred food to a nonpreferred food (Watanabe 1996). Estimates of the frequency with which neurons exhibit enhancement when the more valued outcome is expected are 5–17% in Kobayashi et al. (2002), approximately 26% in Watanabe (1990), 40% in Watanabe (1996), and 46% in Watanabe et al. (2002). Estimates of the frequency with which neurons exhibit suppression when the more valued outcome is expected are 11–20% in Kobayashi et al. (2002), approximately 13% in Watanabe (1990), 10% in Watanabe (1996), and 30% in Watanabe et al. (2002). Most of these frequencies are greater than observed in the PFC in our study. This may be a result of the fact that the possible outcomes in these studies covered a broader range in their appetitive value.
ENHANCEMENT OF DIRECTION SELECTIVITY. Leon and Shadlen (1999) found that the firing-rate enhancement associated with expectation of a large reward was greater during trials requiring a saccade in the neuron's preferred direction than during trials requiring a saccade in the opposite direction. Kobayashi et al. (2002) confirmed this effect among neurons exhibiting enhancement in association with the expectation of reward and showed that it was not present among neurons exhibiting suppression. In both cases, the conclusions were based on the pattern of activity across an entire neuronal population, not on significant interaction effects observed at the level of individual neurons. Our results are consistent with the earlier reports insofar as we have found that expectation of a large reward induces a subtle sharpening of directional signals at the population level, which rarely attains significance at the level of individual neurons.
CONCLUSION. The effects observed in PFC in this study, consisting of an increase in the population firing rate and an enhancement of population directional signals under conditions in which a more valued outcome is expected, are consistent with effects described previously. The effects are commensurate in strength with those reported by Leon and Shadlen (1999), although weaker than those reported by other groups (Kobayashi et al. 2002; Watanabe 1990, 1996; Watanabe et al., 2002). Their relative weakness could arise from several methodological features common to this study and to that of Leon and Shadlen (1999). It is crucial to note that identical behavioral methods were used in this study to characterize reward-dependent activity in PFC and other areas. The relative weakness of effects observed in PFC makes all the more striking the fact that strong effects were observed in some of these areas.
Effects of predicted reward in the frontal eye field
Among 132 neurons located in the FEF, we found that the mean firing rate and the strength of the directional signal were moderately enhanced when the monkey expected a large as opposed to a small reward. Reward-related effects were significantly more frequent in the FEF than in the PFC. This result is superficially at variance both with Leon and Shadlen's (1999) claim that reward-related activity is less common in the FEF than in the PFC and with the failure of Kobayashi et al. (2002) to observe a clear distinction between the two areas. However, Leon and Shadlen's sample of FEF neurons was so small that the difference between the FEF and the PFC does not achieve significance by a 2 test, whereas Kobayashi et al. (2002) report no explicit quantitative or statistical comparison between the 2 areas.
Effects of predicted reward in FEF/PM and PM
We subdivided postarcuate sites into two categories on the basis of whether electrical stimulation at low current strengths yielded both eye and face/limb movements (FEF/PM) or just face/limb movements (PM). Because the essential findings were the same for the two regions and because they were both located behind the arcuate sulcus in a territory commonly designated as PM, we will discuss as one group the 129 neurons in FEF/PM and PM. We found that the enhancement of neuronal firing rates on big-reward trials was very much stronger in this population than in the FEF or PFC. In previous studies, only Watanabe (1990, 1992) described neuronal activity in PM under conditions in which the value of the expected reward is manipulated. Using a behavioral paradigm in which a cue delivered early in each trial indicated whether reward would or would not be delivered at the trial's end, he combined neurons in PFC and PM for analysis, commenting that there was no obvious difference between the areas (Watanabe 1990). However, he notes in passing that reward-related suppression was as common in PM as reward-related enhancement (Watanabe 1992), a characterization that does not apply to the population as a whole. These statements are difficult to assess because they are not supported by quantitative or statistical analysis. However, on the surface, they seem to be at variance with our results. The discrepancy concerns both the relation between PFC and PM (we found a large difference between the areas in the frequency of reward-related signals) and the nature of reward-related signals in PM (we found a marked preponderance of neurons displaying enhanced activity in conjunction with large reward). What factors could underlie the apparently great difference between the outcomes of the studies is not clear.
Effects of predicted reward in SEF
Among 164 neurons recorded in the SEF, we found only extremely weak effects of the size of the predicted reward. This result might seem to be at variance with the results of previous studies demonstrating reward-related neuronal activity in the SEF. However, these studies employed different behavioral paradigms to investigate different reward-related phenomena. Some neurons in the SEF have been shown to fire during the period immediately after an operant response and immediately preceding or accompanying the delivery of reward (Amador et al. 2000; Stuphorn et al. 2000). The late timing of the activity of these neurons suggests that they are involved in preparation of the consummatory response. The existence of neurons with this property is entirely compatible with our finding that there is little or no reward-related activity during earlier phases of the trial when the monkey is aware of the goal and is engaged with the demands of the task but when reward is not yet imminent. In another case, SEF neurons were shown to carry bias signals reflecting the monkey's impending choice of a particular oculomotor response with a history of being well rewarded (Coe et al. 2002). The fact that SEF neurons represent a response bias, which has incidentally been determined by reward history, is easily reconciled with the fact that they do not represent the value of an expected reward. Thus the very weak effects observed in the present study are consistent with other published observations on the SEF.
The need to deconfound reward value and motivational intensity
The finding that neuronal activity in premotor areas is robustly affected by the size of an expected reward suggests that caution should be exercised in the interpretation of reward-related activity throughout the brain. It is natural to suppose that neurons with reward-related activity have as their function to represent the value of an anticipated outcome (Hassani et al. 2001; Hikosaka and Watanabe 2000; Platt and Glimcher 1999; Rolls 2000; Tremblay and Schultz 2000a; Watanabe 1998). However, neuronal signals that represent the value of an expected reward cannot easily be distinguished from neuronal signals that reflect the degree to which the monkey is in a state of generalized motor readiness or specific movement preparation. Behavioral performance is exquisitely sensitive to the value of an expected reward. When a more valued reward is expected, reaction times tend to be shorter (Hassani et al. 2001; Hollerman et al. 1998; Kawagoe et al. 1998; Kobayashi et al. 2002; Lauwereyns et al. 2002b; Takikawa et al. 2002b; Tremblay et al. 1998; Watanabe 1990; Watanabe et al. 2001), movements are faster (Kawagoe et al. 1998; Kobayashi et al. 2002; Leon and Shadlen 1999; Takikawa et al. 2002b), targeting is more precise (Kobayashi et al. 2002; Leon and Shadlen 1999), percent-correct scores are higher (Hassani et al. 2001; Kobayashi et al. 2002; Lauwereyns et al. 2002a; Leon and Shadlen 1999; Takikawa et al. 2002b), and fixation breaks are fewer (Leon and Shadlen 1999; Takikawa et al. 2002b). From these observations, it is evident that the neural representation of the planned response must be stronger, more stable, and more precise when a more valued reward is expected. Furthermore, enhanced preparation for the planned response may be accompanied by enhanced preparation to ingest the reward and by enhancement of overt postural adjustments accompanying preparation. Thus it is difficult to distinguish between neuronal activity representing of the value of an expected reward and neuronal activity reflecting motivational modulation of motor signals. This point applies to all behavioral paradigms used for characterizing reward-related activity including the one employed in our study.
THE RESPONSE BIAS PARADIGM. Monkeys sometimes are required to make responses in different directions on different trials but are rewarded or rewarded abundantly only for responses in one direction. If given a choice in such a situation, the monkey will select the response associated with greater reward (Coe et al. 2002), whereas, if given no choice, it will carry out more swiftly and accurately the action associated with the greater reward (Takikawa et al. 2002b; Watanabe 1990, 1992; Watanabe et al. 2001). Neuronal activity in this situation comes to represent with special robustness the more rewarded response direction. This is true of neuronal activity in the caudate nucleus (Kawagoe et al. 1998; Lauwereyns et al. 2002a,b; Takikawa et al. 2002a), the substantia nigra pars reticulata (Sato and Hikosaka 2002), the frontal eye field (Coe et al. 2002), the supplementary eye field (Coe et al. 2002), the lateral intraparietal area (Platt and Glimcher 1999), and the superior colliculus (Ikeda et al. 2001). Bias signals have been described as representing "expected gain" (Platt and Glimcher 1999) but this is not a necessary interpretation. A comparable effect is observed when factors other than reward modulate the monkey's predisposition to respond in a certain direction. Increasing the probability that a target will appear in a neuron's response field increases its rate of firing in area LIP (Colby et al. 1996; Platt and Glimcher 1999), the FEF (Umeno and Goldberg 2001), and the superior colliculus (Basso and Wurtz 1997, 1998). Likewise, increasing the signal-to-noise ratio of the cue instructing a certain response leads to enhanced activity among neurons representing that response in area LIP (Roitman and Shadlen 2002; Shadlen and Newsome 2001), dorsolateral prefrontal cortex (Kim and Shadlen 1999), and the superior colliculus (Horwitz and Newsome 2001). Bias signals thus reflect the intensity of movement preparation as distinct from reward value.
THE FIXED-RATIO PARADIGM. Monkeys sometimes are required to perform under a schedule in which a certain number of trials must be completed before reward is delivered (Bowman et al. 1996; Liu and Richmond 2000; Shidara and Richmond 2002; Shidara et al. 1998). Studies using this approach have focused on the question of whether neurons in particular areas carry information about the position of a given trial in the sequence. However, they have also revealed, using measures of reaction time and percent correct, that the monkey's motivational level increases steadily over the course of the sequence of trials. The increase in motivation could arise from an increase in the value of the expected reward because it is subject to progressively less time discounting, or from an increase in the cost of failure, measured as the additional effort involved in starting the sequence over from the beginning. In the context of this task, monotonic shifts of firing rate over the course of the sequence of trials have been observed in the ventral striatum (Bowman et al. 1996; Shidara et al. 1998) and anterior cingulate cortex (Shidara and Richmond 2002). It is not clear whether these represent the value of the promised reward or the value of the threatened penalty or are correlated with the monkey's level of motor readiness and movement preparation.
THE CUED REWARD PARADIGM. A cue delivered early in each trial is sometimes used to indicate to the monkey the nature of the reward available on completion of a manual response directed to a single standard location. Reward manipulations employed in this design have included reward versus no reward (Hollerman et al. 1998; Tremblay et al. 1998; Watanabe 1990, 1992) and more favored foodstuff versus less favored foodstuff (Hikosaka and Watanabe 2000). Behavioral responses are faster when the more valued reward is at stake (Hollerman et al. 1998; Tremblay et al. 1998; Watanabe 1990, 1992). Both in the caudate nucleus (Hollerman et al. 1998) and in dorsolateral prefrontal cortex (Watanabe 1990, 1992), the majority of neurons with reward-related activity exhibits increased firing on these trials. Because the monkey's motivational level varies with the value of the expected reward, as indicated by behavioral measures, it is not possible to dissociate neuronal activity representing the value of the reward from neuronal activity reflecting the reward-induced state of motor readiness and movement preparation.
THE CUED REWARD AND DIRECTION PARADIGM. Finally, as in the present study, the monkey may be informed by a combination of cues delivered early in each trial both what action to perform and how much reward to expect. Reward manipulations have included reward versus no reward (Hollerman et al. 1998; Kobayashi et al. 2002; Watanabe 1990, 1992), more favored foodstuff versus less favored foodstuff (Hassani et al. 2001; Tremblay and Schultz 1999; Watanabe 1996), and larger versus smaller volume of fluid (Leon and Shadlen 1999). The sets of possible actions have included saccades in different directions (Kobayashi et al. 2002; Leon and Shadlen 1999) and reaching movements to targets at different locations (Hassani et al. 2001; Hollerman et al. 1998; Tremblay and Schultz 1999; Watanabe 1990, 1992, 1996). Behavioral performance is superior by a variety of measures when the more valued reward is at stake (Hassani et al. 2001; Hollerman et al. 1998; Kobayashi et al. 2002; Leon and Shadlen 1999; Watanabe 1990, 1992; Watanabe et al. 2001). Predicting a more valued reward tends to induce an increase in the strength of the directional signal, defined as the difference in firing rate between neurons representing the planned response and neurons representing alternative responses (Leon and Shadlen 1999; Kobayashi et al. 2002). With respect to mean firing rate, considered independently of response direction, the most common effect of predicting a valued reward is an increase in firing rate. This generalization applies both to the caudate nucleus (Hassani et al. 2001; Hollerman et al. 1998) and to the dorsolateral prefrontal cortex (Leon and Shadlen 1999; Watanabe 1990, 1992, 1996; but see Kobayashi et al. 2002). Whether the enhancement of mean firing rate reflects the value assigned to the expected reward or motivational modulation of the monkey's preparatory state is uncertain because the two are correlated.
In conclusion, neurons with reward-related activity may well, in some cortical and subcortical areas, represent the values of rewards as goals. That the brain should form explicit representations of the values of goals is indeed predicted by some theories of goal-directed behavior (Dickinson and Balleine 1994) and emotion (Papez 1937). However, reward-related activity, as characterized in standard testing situations, is always subject to the interpretation that it reflects the monkey's motivationally modulated state of motor preparation. Furthermore, in one area with especially robust reward-related activity, the premotor cortex, this is the interpretation most consonant with the area's known functions. To clarify the behavioral significance of reward-related activity in other areas where both interpretations seem plausible, for example, the dorsolateral prefrontal cortex and the caudate nucleus, will require the use of nonstandard tasks designed to decouple the value of the anticipated reward from the intensity of motor preparation.
Behavioral significance of motivational modulation in premotor areas
If one grants the proposition that the prominent reward-related activity of neurons in FEF/PM, PM, and SMAr was attributed to motivational modulation of motor signals, then the question arises: What was the behavioral significance of the modulation? As noted in the INTRODUCTION, there are at least 4 plausible scenarios that involve motivational modulation of motor signals in delay tasks such as the one used here. 1) Neurons representing the plan for the instructed response might fire more strongly when a more valued reward is at stake. This scenario can account for the relatively infrequent effect whereby directional signals were enhanced on big-reward trials (Fig. 11C, blue bars). However, it cannot explain the much more frequent cases in which enhancement occurred independently of response direction (Fig. 11A, blue bars). Nor can it account for the fact that areas in which directionally selective neurons were most prominent (Fig. 11B, FEF and SEF) do not coincide with areas in which reward effects were most prominent (Fig. 11A, FEF/PM, PM, and SMAr). 2) Neuronal activity sensitive to arousal or responsible for generalized motor readiness might be enhanced on trials involving a more valued reward. It is not clear why a state of behavioral arousal should be accompanied by an increase in population activity in premotor cortex or how such an increase could give rise to generalized motor readiness. However, with such issues resolved, this scenario could explain the fact that neuronal activity tends to increase on big-reward trials. 3) Neuronal activity governing overt behaviors that automatically accompany response planning, such as increases of axial tonus, might be enhanced when a more valued reward is at stake. This scenario is compatible with our finding that delay-period neck EMG activity was enhanced on big-reward trials in one monkey. The absence of reward-induced modulations of neck EMG in the other two monkeys seems to argue against it. However, the other monkeys may have engaged, during the delay period, in overt behaviors involving other muscles not subject to EMG monitoring. 4) Neurons involved in preparing ingestive movements might be more active before delivery of a more valued reward. This scenario is compatible with the finding that neurons showing enhanced delay-period activity on big-reward trials tended to fire more than other neurons immediately before and during the delivery of reward. Firing during the delay period might have been stronger on big-reward trials because monkeys prepared consummatory movements farther in advance of reward delivery on these trials. Firing during the period surrounding reward delivery might have occurred regardless of reward size because the monkeys prepared at the end of the trial to ingest whatever reward was impending. None of these scenarios can be absolutely ruled out. Each may contribute to reward-related activity. To assess the contribution of each will require further experiments.
|
|
DISCLOSURES |
|---|
|
|
|
ACKNOWLEDGMENTS |
|---|
|
|
|
FOOTNOTES |
|---|
Address for reprint requests and other correspondence: M. R. Roesch, Center for the Neural Basis of Cognition, Mellon Institute, Room 115, 4400 Fifth Ave., Pittsburgh, PA 15213-2683 (E-mail: roesch{at}cnbc.cmu.edu).
|
|
REFERENCES |
|---|
|
Amador N, Schlag-Rey M, and Schlag J. Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J Neurophysiol 84: 2166–2170, 2000.
Apicella P, Ljungberg T, Scarnati E, and Schultz W. Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85: 491–500, 1991.[Web of Science][Medline]
Basso MA and Wurtz RH. Modulation of neuronal activity by target uncertainty. Nature 389: 66–69, 1997.[Medline]
Basso MA and Wurtz RH. Modulation of neuronal activity in superior colliculus by changes in target probability. J Neurosci 18: 7519–7534, 1998.
Baylis LL and Gaffan D. Amygdalectomy and ventromedial prefrontal ablation produce similar deficits in food choice and in simple object discrimination learning for an unseen reward. Exp Brain Res 86: 617–622, 1991.[Web of Science][Medline]
Bowman EM, Aigner TG, and Richmond BJ. Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J Neurophysiol 75: 1061–1073, 1996.
Brinkman C. Lesions in supplementary motor area interfere with a monkey's performance of a bimanual coordination task. Neurosci Lett 27: 267–270, 1981.[Web of Science][Medline]
Burton MJ, Rolls ET, and Mora F. Effects of hunger on the responses of neurons in the lateral hypothalamus to the sight and taste of food. Exp Neurol 51: 668–677, 1976.[Web of Science][Medline]
Butter CM, McDonald JA, and Snyder DR. Orality, preference behavior, and reinforcement value of nonfood object in monkeys with orbital frontal lesions. Science 164: 1306–1307, 1969.
Butter CM and Snyder DR. Alterations in aversive and aggressive behaviors following orbital frontal lesions in rhesus monkeys. Acta Neurobiol Exp (Warsz) 32: 525–565, 1972.[Medline]
Butter CM, Snyder DR, and McDonald JA. Effects of orbital frontal lesions on aversive and aggressive behaviors in rhesus monkeys. J Comp Physiol Psychol 72: 132–144, 1970.[Web of Science][Medline]
Coe B, Tomihara K, Matsuzawa M, and Hikosaka O. Visual and anticipatory bias in three cortical eye fields of the monkey during an adaptive decision-making task. J Neurosci 22: 5081–5090, 2002.
Colby CL, Duhamel J-R, and Goldberg ME. Visual, presaccadic and cognitive activation of single neurons in monkey lateral intraparietal area. J Neurophysiol 76: 2841–2852, 1996.
Crist CF, YamasakiDSG,Komatsu H, and Wurtz RH. A grid system and a microsyringe for single cell recording. J Neurosci Methods 26: 117–122, 1988.[Web of Science][Medline]
Critchley HD and Rolls ET. Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. J Neurophysiol 75: 1673–1686, 1996.
Dias EC and Segraves MA. Muscimol-induced inactivation of monkey frontal eye field: effects on visually and memory guided saccades. J Neurophysiol 81: 2191–2214, 1999.
Dickinson A and Balleine B. Motivational control of goal-directed action. Anim LearnBehav 22: 1–18, 1994.
Fujii N, Mushiake H, and Tanji J. An oculomotor representation area within the ventral premotor cortex. Proc Natl Acad Sci USA 95: 12034–12037, 1998.
Fujii N, Mushiake H, and Tanji J. Rostrocaudal distinction of the dorsal premotor area based on oculomotor involvement. J Neurophysiol 83: 1764–1769, 2000.
Fukuda M, Ono T, and Nakamura K. Functional relations among inferotemporal cortex, amygdala, and lateral hypothalamus in monkey operant feeding behavior. J Neurophysiol 57: 1060–1077, 1987.
Fukuda M, Ono T, Nishino H, and Sasaki K. Visual responses related to food discrimination in monkey lateral hypothalamus during operant feeding behavior. Brain Res 374: 249–259, 1986.[Web of Science][Medline]
Gaffan D and Murray EA. Amygdalar interaction with the mediodorsal nucleus of the thalamuus and the ventromedial prefrontal cortex in stimulus-reward associative learning in the monkey. J Neurosci 10: 3479–3493, 1990.[Abstract]
Hassani OK, Cromwell HC, and Schultz W. Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85: 2477–2489, 2001.
Hikosaka K and Watanabe M. Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cereb Cortex 10: 263–271, 2000.
Hikosaka O, Sakamoto M, and Usui S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61: 814–832, 1989.
Hikosaka O, Takikawa Y, and Kawagoe R. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev 80: 953–978, 2000.
Hollerman JR, Tremblay L, and Schultz W. Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80: 947–963, 1998.
Hollerman JR, Tremblay L, and Schultz W. Involvement of basal ganglia and orbitofrontal cortex in goal-directed behavior. Prog Brain Res 126: 193–215, 2000.[Web of Science][Medline]
Horwitz GD and Newsome WT. Target selection for saccadic eye movements: prelude activity in the superior colliculus during a direction discrimination task. J Neurophysiol 86: 2543–2558, 2001.
Huang CS, Hiraba H, Murray GM, and Sessle BJ. Topographical distribution and functional properties of cortically induced rhythmical jaw movements in the monkey (Macaca fascicularis). J Neurophysiol 61: 635–650, 1989.
Ikeda T, Takikawa Y, and Hikosaka O. Visuo-motor and anticipatory activities of monkey superior colliculus neurons are modulated by reward. Soc Neurosci Abstr 27: 59.1, 2001.
Jones B and Mishkin M. Limbic lesions and the problem of stimulus-reinforcement associations. Exp Neurol 36: 362–377, 1972.[Web of Science][Medline]
Kalivas PW and Nakamura M. Neural systems for behavioral activation and reward. Curr Opin Neurobiol 9: 223–227, 1999.[Web of Science][Medline]
Kawagoe R, Takikawa Y, and Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1: 411–416, 1998.[Web of Science][Medline]
Kim J-M and Shadlen MN. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 12: 176–185, 1999.
Kobayashi S, Lauwereyns J, Koizumi M, Sakagami M, and Hikosaka O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J Neurophysiol 87: 1488–1498, 2002.
Kurata K and Hoffman DS. Differential effects of muscimol microinjection into dorsal and ventral aspects of the premotor cortex of monkeys. J Neurophysiol 71: 1151–1164, 1994.
Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, and Hikosaka O. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33: 463–473, 2002a.[Web of Science][Medline]
Lauwereyns J, Watanabe K, Coe B, and Hikosaka O. A neural correlate of response bias in monkey caudate nucleus. Nature 418: 413–417, 2002b.[Medline]
Leon MI and Shadlen MN. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24: 415–425, 1999.[Web of Science][Medline]
Liu Z and Richmond BJ. Response differences in monkey TE and perirhinal cortex: stimulus association related to reward schedules. J Neurophysiol 83: 1677–1692, 2000.
Ljungberg T, Apicella P, and Schultz W. Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res 567: 337–341, 1991.[Web of Science][Medline]
Ljungberg T, Apicella P, and Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67: 145–163, 1992.
Málková L, Gaffan D, and Murray EA. Excitotoxic lesions of the amygdala fail to produce impairment in visual learning for auditory secondary reinforcement but interfere with reinforcer devaluation effects in rhesus monkeys. J Neurosci 17: 6011–6020, 1997.
Mirenowicz J and Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72: 1024–1027, 1994.
Mogenson GJ, Jones DL, and Yim CY. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 14: 69–97, 1980.[Web of Science][Medline]
Mora F, Rolls ET, and Burton MJ. Modulation during learning of the responses of neurons in the lateral hypothalamus to the sight of food. Exp Neurol 53: 508–519, 1976.[Web of Science][Medline]
Murray EA, Gaffan EA, and Flint RW. Anterior rhinal cortex and amygdala: dissociation of their contributions to memory and food preference in rhesus monkeys. Behav Neurosci 110: 30–42, 1996.[Web of Science][Medline]
Murray GM, Lin LD, Moustafa EM, and Sessle BJ. Effects of reversible inactivation by cooling of the primate face motor cortex on the performance of a trained tongue-protrusion task and a trained biting task. J Neurophysiol 65: 511–530, 1991.
Nakamura K, Mikami A, and Kubota K. Activity of single neurons in the monkey amygdala during performance of a visual discrimination task. J Neurophysiol 67: 1447–1463, 1992.
Nishijo H, Ono T, and Nishino H. Single neuron responses in amygdala of alert monkey during complex sensory stimulation with affective significance. J Neurosci 8: 3570–3583, 1988a.[Abstract]
Nishijo H, Ono T, and Nishino H. Topographic distribution of modality-specific amygdalar neurons in alert monkey. J Neurosci 8: 3556–3569, 1988b.[Abstract]
Ono T, Nishijo H, and Nishino H. Functional role of the limbic system and basal ganglia in motivated behaviors. J Neurol 247(Suppl 5): V23–V32, 2000.
Papez JW. A proposed mechanism of emotion. Arch Neurol Psychiatry 38: 725–743, 1937.
Platt ML and Glimcher PW. Neural correlates of decision variables in parietal cortex. Nature 400: 233–238, 1999.[Medline]
Robinson DA. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans Biomed Eng 10: 137–145, 1963.[Medline]
Roitman JD and Shadlen MN. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22: 9475–9489, 2002.
Rolls ET. The orbitofrontal cortex and reward. Cereb Cortex 10: 284–294, 2000.
Rolls ET and Baylis LL. Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. J Neurosci 14: 5437–5452, 1994.[Abstract]
Rolls ET, Burton MJ, and Mora F. Hypothalamic neuronal responses associated with the sight of food. Brain Res 111: 53–66, 1976.[Web of Science][Medline]
Rolls ET, Critchley HD, Mason R, and Wakeman EA. Orbitofrontal cortex neurons: role in olfactory and visual association learning. J Neurophysiol 75: 1970–1981, 1996.
Rolls ET, Murzi E, Yaxley S, Thorpe SJ, and Simpson SJ. Sensory-specific satiety: food-specific reduction in responsiveness of ventral forebrain neurons after feeding in the monkey. Brain Res 368: 79–86, 1986.[Web of Science][Medline]
Rolls ET, Sanghera MK, and Roper-Hall A. The latency of activation of neurones in the lateral hypothalamus and substantia innominata during feeding in the monkey. Brain Res 164: 121–135, 1979.[Web of Science][Medline]
Romo R and Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63: 592–606, 1990.
Sanghera MK, Rolls ET, and Roper-Hall A. Visual responses of neurons in the dorsolateral amygdala of the alert monkey. Exp Neurol 63: 610–626, 1979.[Web of Science][Medline]
Sato M and Hikosaka H. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J Neurosci 22: 2363–2373, 2002.
Schultz W. Multiple reward signals in the brain. Nat Rev Neurosci 1: 199–207, 2000.[Web of Science][Medline]
Schultz W, Apicella P, and Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13: 900–913, 1993a.[Abstract]
Schultz W, Apicella P, Ljungberg T, Romo R, and Scarnati E. Reward-related activity in the monkey striatum and substantia nigra. Prog Brain Res 99: 227–235, 1993b.[Web of Science][Medline]
Schultz W, Apicella P, Scarnati E, and Ljungberg T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12: 4595–4610, 1992.[Abstract]
Shadlen MN and Newsome WT. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86: 1916–1936, 2001.
Shidara M, Aigner TG, and Richmond BJ. Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18: 2613–2625, 1998.
Shidara M and Richmond BJ. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296: 1709–1711, 2002.
Sommer MA and Tehovnik EJ. Reversible inactivation of macaque frontal eye field. Exp Brain Res 116: 229–249, 1997.[Web of Science][Medline]
Sommer MA and Tehovnik EJ. Reversible inactivation of macaque dorsomedial frontal cortex: effects on saccades and fixation. Exp Brain Res 124: 429–446, 1999.[Web of Science][Medline]
Stellar E. Brain mechanisms in hedonic processes. In: The Physiological Mechanisms of Motivation, edited by Pfaff DW. New York: Springer-Verlag, 1982, p. 377–407.
Stellar JR and Stellar E. The Neurobiology of Motivation and Reward. New York: Springer-Verlag, 1985.
Stuphorn V, Taylor T, and Schall JD. Performance monitoring by the supplementary eye field. Nature 408: 857–860, 2000.[Medline]
Takikawa Y, Kawagoe R, and Hikosaka O. Reward-dependent spatial selectivity of anticipatory activity in monkey caudate neurons. J Neurophysiol 87: 508–515, 2002a.
Takikawa Y, Kawagoe R, Itoh H, Nakahara H, and Hikosaka O. Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142: 284–291, 2002b.[Web of Science][Medline]
Thorpe SJ, Rolls ET, and Maddison S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp Brain Res 49: 93–115, 1983.[Web of Science][Medline]
Tremblay L, Hollerman JR, and Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80: 964–977, 1998.
Tremblay L and Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 398: 704–708, 1999.[Medline]
Tremblay L and Schultz W. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J Neurophysiol 83: 1864–1876, 2000a.
Tremblay L and Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J Neurophysiol 83: 1877–1885, 2000b.
Umeno MM and Goldberg ME. Spatial processing in the monkey frontal eye field. II. memory responses. J Neurophysiol 86: 2344–2352, 2001.
Watanabe M. Prefrontal unit activity during associative learning in the monkey. Exp Brain Res 80: 296–309, 1990.[Web of Science][Medline]
Watanabe M. Frontal units of the monkey coding the associative significance of visual and auditory stimuli. Exp Brain Res 89: 233–247, 1992.[Web of Science][Medline]
Watanabe M. Reward expectancy in primate prefrontal neurons. Nature 382: 629–632, 1996.[Medline]
Watanabe M. Cognitive and motivational operations in primate prefrontal neurons. Rev Neurosci 9: 225–241, 1998.[Web of Science][Medline]
Watanabe M, Cromwell HC, Tremblay L, Hollerman JR, Hikosaka K, and Schultz W. Behavioral reactions reflecting differential reward expectations in monkeys. Exp Brain Res 140: 511–518, 2001.[Web of Science][Medline]
Watanabe M, Hikosaka K, Sakagami M, and Shirakawa S. Coding and monitoring of motivational context in the primate prefrontal cortex. J Neurosci 22: 2391–2400, 2002.
This article has been cited by other articles:
|
|
 |
|
  J. L. Pardo-Vazquez, V. Leboran, and C. Acuna A role for the ventral premotor cortex beyond performance monitoring PNAS, November 3, 2009; 106(44): 18815 - 18819. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C.F. Geier, R. Terwilliger, T. Teslovich, K. Velanova, and B. Luna Immaturities in Reward Processing and Its Influence on Inhibitory Control in Adolescence Cereb Cortex, October 29, 2009; (2009) bhp225v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. J. Buckley, F. A. Mansouri, H. Hoda, M. Mahboubi, P. G. F. Browning, S. C. Kwok, A. Phillips, and K. Tanaka Dissociable Components of Rule-Guided Behavior Depend on Distinct Medial and Prefrontal Regions Science, July 3, 2009; 325(5936): 52 - 58. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C.-H. Luk and J. D. Wallis Dynamic Encoding of Responses and Outcomes by Neurons in Medial Prefrontal Cortex J. Neurosci., June 10, 2009; 29(23): 7526 - 7539. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K.-i. Okada, K. Toyama, Y. Inoue, T. Isa, and Y. Kobayashi Different Pedunculopontine Tegmental Neurons Signal Predicted and Actual Task Rewards J. Neurosci., April 15, 2009; 29(15): 4858 - 4870. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Behseta, T. Berdyyeva, C. R. Olson, and R. E. Kass Bayesian Correction for Attenuation of Correlation in Multi-Trial Spike Count Data J Neurophysiol, April 1, 2009; 101(4): 2186 - 2193. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. W. Kennerley and J. D. Wallis Reward-Dependent Modulation of Working Memory in Lateral Prefrontal Cortex J. Neurosci., March 11, 2009; 29(10): 3259 - 3270. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. K. Berdyyeva and C. R. Olson Monkey Supplementary Eye Field Neurons Signal the Ordinal Position of Both Actions and Objects J. Neurosci., January 21, 2009; 29(3): 591 - 599. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H. Seo and D. Lee Cortical mechanisms for reinforcement learning in competitive games Phil Trans R Soc B, December 12, 2008; 363(1511): 3845 - 3857. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Rezvani and B. D. Corneil Recruitment of a Head-Turning Synergy by Low-Frequency Activity in the Primate Superior Colliculus J Neurophysiol, July 1, 2008; 100(1): 397 - 411. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Tanji and E. Hoshi Role of the Lateral Prefrontal Cortex in Executive Behavioral Control Physiol Rev, January 1, 2008; 88(1): 37 - 57. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J.-W. Sohn and D. Lee Order-Dependent Modulation of Directional Signals in the Supplementary and Presupplementary Motor Areas J. Neurosci., December 12, 2007; 27(50): 13655 - 13666. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. Uchida, X. Lu, S. Ohmae, T. Takahashi, and S. Kitazawa Neuronal Activity Related to Reward Size and Rewarded Target Position in Primate Supplementary Eye Field J. Neurosci., December 12, 2007; 27(50): 13750 - 13755. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Seo and D. Lee Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game J. Neurosci., August 1, 2007; 27(31): 8366 - 8377. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Milstein and M. C. Dorris The Influence of Expected Value on Saccadic Preparation J. Neurosci., May 2, 2007; 27(18): 4810 - 4818. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. R. Roesch, T. A. Stalnaker, and G. Schoenbaum Associative Encoding in Anterior Piriform Cortex versus Orbitofrontal Cortex during Odor Discrimination and Reversal Learning Cereb Cortex, March 1, 2007; 17(3): 643 - 652. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. N. Tobler, J. P. O'Doherty, R. J. Dolan, and W. Schultz Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems J Neurophysiol, February 1, 2007; 97(2): 1621 - 1632. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Ding and O. Hikosaka Temporal Development of Asymmetric Reward-Induced Bias in Macaques J Neurophysiol, January 1, 2007; 97(1): 57 - 61. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K.-i. Amemori and T. Sawaguchi Contrasting Effects of Reward Expectation on Sensory and Motor Memories in Primate Prefrontal Neurons Cereb Cortex, July 1, 2006; 16(7): 1002 - 1015. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Ding and O. Hikosaka Comparison of reward modulation in the frontal eye field and caudate of the macaque. J. Neurosci., June 21, 2006; 26(25): 6695 - 6703. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. G. Shuler and M. F. Bear Reward timing in the primary visual cortex. Science, March 17, 2006; 311(5767): 1606 - 1609. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
O. Hikosaka, K. Nakamura, and H. Nakahara Basal Ganglia Orient Eyes to Reward J Neurophysiol, February 1, 2006; 95(2): 567 - 584. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. R. Roesch and C. R. Olson Neuronal Activity in Primate Orbitofrontal Cortex Reflects the Value of Time J Neurophysiol, October 1, 2005; 94(4): 2457 - 2471. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Campos, B. Breznen, K. Bernheim, and R. A. Andersen Supplementary Motor Area Encodes Reward Expectancy in Eye-Movement Tasks J Neurophysiol, August 1, 2005; 94(2): 1325 - 1335. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. R. Roesch and C. R. Olson Neuronal Activity Dependent on Anticipated and Elapsed Delay in Macaque Prefrontal Cortex, Frontal and Supplementary Eye Fields, and Premotor Cortex J Neurophysiol, August 1, 2005; 94(2): 1469 - 1497. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Tsujimoto and T. Sawaguchi Context-dependent Representation of Response-outcome in Monkey Prefrontal Neurons Cereb Cortex, July 1, 2005; 15(7): 888 - 898. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Tsujimoto and T. Sawaguchi Neuronal Activity Representing Temporal Prediction of Reward in the Primate Prefrontal Cortex J Neurophysiol, June 1, 2005; 93(6): 3687 - 3692. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Z. L. Rossetti and S. Carboni Noradrenaline and Dopamine Elevations in the Rat Prefrontal Cortex in Spatial Working Memory J. Neurosci., March 2, 2005; 25(9): 2322 - 2329. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Musallam, B. D. Corneil, B. Greger, H. Scherberger, and R. A. Andersen Cognitive Control Signals for Neural Prosthetics Science, July 9, 2004; 305(5681): 258 - 262. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Isoda and J. Tanji Participation of the Primate Presupplementary Motor Area in Sequencing Multiple Saccades J Neurophysiol, July 1, 2004; 92(1): 653 - 659. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. Hoshi and J. Tanji Area-Selective Neuronal Activity in the Dorsolateral Prefrontal Cortex for Information Retrieval and Action Planning J Neurophysiol, June 1, 2004; 91(6): 2707 - 2722. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Lee Behavioral Context and Coherent Oscillations in the Supplementary Motor Area J. Neurosci., May 5, 2004; 24(18): 4453 - 4459. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. R. Roesch and C. R. Olson Neuronal Activity Related to Reward Value and Motivation in Primate Frontal Cortex Science, April 9, 2004; 304(5668): 307 - 310. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Amador, M. Schlag-Rey, and J. Schlag Primate Antisaccade. II. Supplementary Eye Field Neuronal Activity Predicts Correct Performance J Neurophysiol, April 1, 2004; 91(4): 1672 - 1689. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
TOP
ABSTRACT
INTRODUCTION
