| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1Department of Physiology, Juntendo University, School of Medicine, Tokyo 113-8421, Japan; and 2Laboratory of Sensorimotor Research, National Eye Institute, National Institute of Health, Bethesda, Maryland 20892-4435
Submitted 28 July 2003; accepted in final form 28 September 2003
 |
ABSTRACT |
|---|
|
|
|
INTRODUCTION |
|---|
|
). This is done mainly by serial inhibitory connections: one from the caudate (CD) to the substantia nigra pars reticulata (SNr) and the other from the SNr to the superior colliculus (SC). Spatial information necessary for saccades may be provided by the inputs from the frontal eye field and the surrounding prefrontal cortices (Selemon and Goldman-Rakic 1985; Yeterian and Pandya 1991; Parthasarathy et al. 1992). Recent studies from our laboratory indicated that this basal ganglia mechanism modifies a saccade depending on whether the saccade is followed by reward (Kawagoe et al. 1998; Sato and Hikosaka 2002; Takikawa et al. 2002a,b). An important question is: Where does the reward-related information originate? A candidate for the input would be midbrain dopaminergic neurons that are located in and around the substantia nigra pars compacta (SNc) (Andén et al. 1964). Indeed, animal experiments provided some hints that DA neurons are related to reward and motivation (Robbins and Everitt 1996). DA neurons in monkeys respond to reward and to sensory stimuli that predict reward (Schultz 1998). Specifically, DA neurons respond to unexpected reward and show depression of activity when reward is unexpectedly omitted (Schultz et al. 1997) or shifted (Hollerman and Schultz 1998). Other lines of evidence suggest a common role of DA neurons in animals and humans. Thus reinforcement of brain self-stimulation in animals and drug addiction in humans seem to be mediated by DA neurons (Wise 1996). Human imaging studies have suggested that rewarding experiences are associated with changes in neural activity (Delgado et al. 2000; Knutson et al. 2000) or increases in DA transmission in the striatum (Koepp et al. 1998). These observations provided experimental evidence for recent theories of reinforcement learning (Barto 1994; Houk et al. 1995; Montague et al. 1996).
Despite a large body of data on DA neurons, it is still unclear how their signals are used to modify motor outputs. A likely site where DA neurons may influence motor signals is the dorsal striatum (including the CD and putamen), since a majority of DA neurons project to the striatum (Lynd-Balta and Haber 1994) and its dorsal part is thought to have motor functions (Alexander and Crutcher 1990). Inside the striatum are cellular and molecular mechanisms with which DA neurons can exert strong influences on striatal projection neurons (Calabresi et al. 1997; Nicola et al. 2000). DA depletion in the unilateral CD induces deficits in initiation of saccades to the contralateral side (Kato et al. 1995; Kori et al. 1995) or contralateral hemineglect (Miyashita et al. 1995). Since CD projection neurons are a major origin of eye movement signals in the basal ganglia (Hikosaka et al. 2000), DA neurons are in a good position to induce the reward-contingent modulation of activity of CD neurons and other basal ganglia neurons.
Using a modified memory-guided saccade task (called 1DR task) (Kawagoe et al. 1998), in which reward was given for one particular direction out of four targets, we found that activity of CD neurons is strongly modulated by the presence and absence of an upcoming reward (Kawagoe et al. 1998) and the position at which reward is given (Lauwereyns et al. 2002b; Takikawa et al. 2002a). To study the function of midbrain DA neurons in reward-oriented saccades, we recorded from CD neurons and DA neurons in the same monkeys using 1DR. We found that activity of DA neurons was even more clearly modulated by an upcoming reward. Interestingly, one monkey showed no reward-dependent modulation of saccades, unlike the other monkeys. In this monkey neither CD neurons nor DA neurons exhibited reward-dependent modulation of visual cue responses. Such individual differences were consistent with the hypothesis that the reward dependencies of DA neurons and CD neurons are causally related such that saccades are modulated depending on an upcoming reward (Takikawa et al. 2002b).
|
|
METHODS |
|---|
|
We used four male Japanese monkeys (Macaca fuscata): monkeys G, H, K, and M. Under general anesthesia, we implanted a head holder, chambers for unit recording, and a scleral search coil (Hikosaka et al. 1993). All surgical and experimental protocols were approved by the Juntendo University Animal Care and Use Committee and are in accordance with the National Institutes of Health Guide for the Care and Use of Animals. The monkeys were trained to perform saccade tasks, especially a memory-guided saccade task (Hikosaka and Wurtz 1983b). Eye movements were recorded using the search coil method. We recorded extracellular spike activity of presumed projection neurons in the CD in all four monkeys (G, H, K, and M) and presumed DA neurons in and around the SNc in three monkeys (G, H, and M).
Task procedures
The monkeys performed the memory-guided saccade task in two different reward conditions: all-directions-rewarded condition (ADR) and one-direction-rewarded condition (1DR). We used a set of four target locations of equal eccentricity (either 10° or 20°), arranged in either normal or oblique angles, depending on the neuron's response field. For every neuron recorded, we required the monkeys to perform one block of ADR and four blocks of 1DR (i.e., four different rewarded directions).
A task trial started with the onset of a central fixation point that the monkeys had to fixate (see Fig. 1). A cue (spot of light, duration: 100 ms) came on 1 s after onset of the fixation point at one of the four target locations, and the monkeys had to remember its location. After a random period (1-1.5 s), the fixation point turned off, and the monkeys were required to make a saccade to the previously cued location. The target came on 400 ms later for 150 ms at the cued location. The saccade was judged to be correct if the eye position was within a window around the target (usually within 3°) when the target turned off. The correct saccade was indicated by a tone stimulus. The next trial started after an intertrial interval of 3.5-4 s.
|
The monkeys were first trained to perform the ADR task. They were then trained to perform the 1DR task. Before the recording experiment started, the monkeys had been trained on the 1DR task extensively for more than 300 blocks.
Recording procedures
Eye movements were recorded using the search coil method (Enzanshi Kogyo MEL-20U) (Judge et al. 1980; Matsumura et al. 1992; Robinson 1963). Eye positions were sampled at 500 Hz. The behavioral tasks as well as storage and display of data were controlled by a computer (PC 9801RA, NEC, Tokyo, Japan). The unitary action potentials were passed through a window discriminator (Bak INC, Model DDIS-1), and the times of their occurrences were stored with a resolution of 1 ms.
Before the single-unit recording experiment, we obtained MR images (Hitachi, AIRIS, 0.3T) such that they were perpendicular to the recording chamber. We then determined the recording sites in the substantia nigra based on the chamber-based coordinates (Kawagoe et al. 1998). The recording sites were further verified by MR-imaging a plastic guide tube through which the electrodes were inserted.
Single-unit recordings were performed using tungsten electrodes (diameter: 0.25 mm, 1-5 M
, measured at 1 KHz, Frederick Haer). To introduce the electrode into the brain, we first inserted a stainless steel guide tube (OD 0.85 mm, ID 0.60 mm) containing the electrode. A hydraulic microdrive (Narishige, MO95-S) was used both to insert the guide tube and subsequently to advance the electrode into the brain. We sometimes implanted a plastic guide tube (OD 1.1 mm, ID 0.8 mm) semipermanently at the location where DA neurons were concentrated. The location of the guide tube was visualized on MRIs and was confirmed to be directed to the SNc.
Identification of DA neurons and CD projection neurons
We searched for DA neurons in and around the SNc (François et al. 1985). Guided by MRIs, we first searched for the SNr, where neurons showed high-frequency spontaneous firing (Hikosaka and Wurtz 1983a). The identity of the SNr was assured if such rapid firing stopped in relation to saccade tasks (Sato and Hikosaka 2002). Often intermingled with or just medial to SNr neurons (usually 
2 mm) were presumed DA neurons that fired irregularly and tonically at about 5 spikes/s with broad spike potentials. Extracellular spikes may have an initial positive component or may be followed by prolonged positive component (Schultz and Romo 1987). A neuron with these features was thus determined to be a DA neuron. We then examined whether it responded to the delivery of reward. A drop of water as a reward was given with a random time interval (4-9 s) while the monkey was sitting in the dim experimental room. If the neuron responded to reward, we had the monkeys perform at least one block of ADR and four blocks of 1DR (i.e., four different rewarded directions). In addition, we sometimes repeated 1DR blocks to confirm the reproducibility of the neuron's behavior.
We recorded extracellular spike activity of presumed projection neurons, which showed very low spontaneous activity (<1 Hz) (Hikosaka et al. 1989a), not of presumed interneurons, which showed irregular tonic discharge (Aosaki et al. 1994). To find CD projection neurons, we let the monkeys perform 1DR continuously because CD neurons were often very quiet if the monkey was not performing a task. If a CD projection neuron was found and it fired at any period while the monkey was performing 1DR, we had the monkeys to perform at least one block of ADR and four blocks of 1DR.
Data analysis
DETERMINATION OF A RESPONSE. We analyzed the postcue response of CD neurons and DA neurons and the postreward response of DA neurons. For each response, we determined its presence by comparing the spike frequency in the control period and the spike frequency in the test period using the Wilcoxon signed-rank test (P < 0.001) separately for rewarded and nonrewarded trials. For CD neurons two kinds of test period after the cue stimulus were set for each neuron: 200-500 ms for visual response and 500-1000 ms for sustained response. If the neuron showed the postcue response significantly at 200-500 and/or 500-1000 ms, we determined that a postcue response was present. We focused on reward enhanced type whose test activity was higher in rewarded trial than nonrewarded trial. For DA neurons the test period for the postcue response was 100-300 ms (for excitatory responses) and 200-500 ms (for inhibitory responses). The control period was set just before the onset of the fixation point for both of CD and DA neurons, its duration being the same as that of the test period. The test period for the response to reward was set to be 100-400 ms after the time when reward would be delivered. We then determined the presence of the reward response in the same way as the cue response (Wilcoxon signed-rank test, P < 0.001).
STATISTICAL ANALYSIS FOR THE REWARD AND SPATIAL SELECTIVITY. A normalized cue response was calculated, for each trial, as the spike frequency during the test period minus the spike frequency during the control period. We performed nonparametric analysis for reward and spatial selectivity for the four-block data of 1DR for the neuron's normalized response. The first eight trials in each block were excluded from the analysis. Reward selectivity of the postcue response in 1DR was tested by the Mann-Whitney U test (P < 0.01) by comparing the responses in the rewarded trials and the responses in the nonrewarded trials. Spatial selectivity of the postcue response was determined by Kruskal-Wallis test (P < 0.01) by comparing the responses in four cued directions in 1DR. We used nonparametric tests because in many trials there was no spike in a test or control period and hence the sampled spike frequencies did not show the normal distribution.
REWARD/NONREWARD DISCRIMINATION OF SACCADE PARAMETERS (SEE FIG. 2 AND TABLE 1). For each experiment including the four blocks of 1DR, we obtained mean values of saccade parameters (velocity, latency, and amplitude) separately for rewarded trials (52 trials: 13 trials x 4 blocks) and nonrewarded trials (156 trials: 39 trials x 4 blocks). The first eight trials in each block were excluded from the analysis. We then performed a statistical comparison for the entire set of experiments between the rewarded and nonrewarded conditions. For this purpose, we used the Wilcoxon signed-rank test (P < 0.0001) by pairing the mean value of rewarded trials and the mean value of nonrewarded trials for each experiment.
|
|
Histology
After recording was completed, monkeys H, G, and M were anesthetized with an overdose of pentobarbital sodium and perfused through the heart with 4% paraformaldehyde. The brain was blocked and equilibrated with 20% sucrose. Frozen sections were cut every 50 µm in the planes parallel to the electrode penetrations so that complete tracks were visible in single sections. The sections were stained with cresyl violet. Reconstruction of the recording sites was based on microlesions (5 µA for 180 s) made at the end of some recording experiments. Other recording sites were estimated based on these microlesions. To examine the dopaminergic activity in the brain of monkeys G and M, we used tyrosine hydroxylase (TH) immunohistochemistry to visualize dopaminergic neurons in and around the SNc and axon terminals in the CD, according to the procedure described previously (Kato et al. 1995). One of every five sections was subject to TH immunohistochemistry and this was done for the anterior-posterior dimension including the entire substantia nigra.
|
|
RESULTS |
|---|
|
We trained four monkeys to perform 1DR of the memory-guided saccade task. Before the recording experiment started, all monkeys had been trained for the 1DR task for more than 300 blocks. As shown in Fig. 2A, in three monkeys (H, G, and K), the peak saccadic velocity was significantly faster when the saccade was followed by reward (ordinate) than when the saccade was not followed by reward (abscissa) (Wilcoxon signed-rank test, P < 0.0001, also see Table 1). This phenomenon has been reported previously (Takikawa et al. 2002b). In monkey M (Fig. 2B), however, the peak saccadic velocity was not different between the two conditions (Wilcoxon signed-rank test, P > 0.01, Table 1). In monkeys H and K, the saccade latency was significantly shorter in rewarded than in nonrewarded trials (Wilcoxon signed-rank test, P < 0.0001, Table 1). Furthermore, the coefficients of variation of saccade parameters were significantly lower in monkeys H, G, and K in rewarded trials than in nonrewarded trials (Wilcoxon signed-rank test, P < 0.0001, parenthesis in Table 1). The error rate was significantly lower in monkeys H, G, and K in rewarded trials than in nonrewarded trials (Wilcoxon signed-rank test, P < 0.0001, Table 1). These tendencies were absent in monkey M. On the other hand, the saccade amplitude was not significantly changed (Table 1). Therefore we judged that three monkeys (H, G, and K) were sensitive and one monkey (M) was insensitive to an upcoming reward. The saccade velocity was generally higher in monkey M than the other monkeys regardless of reward condition (Table 1).
Search for DA neurons and CD projection neurons
A total of 368 neurons in three monkeys (H, G, and M) were determined to be DA neurons (see METHODS). Among them, 203 (55%) responded to the delivery of reward by phasically increasing its discharge rate. We further examined these reward-responsive DA neurons using the ADR and 1DR. The other reward-unresponsive neurons were not examined further, although we cannot exclude the possibility that they also were DA neurons. The data presented in this paper are based on 71 DA neurons for which we were able to examine at least one block of ADR and four blocks of 1DR. They were recorded from monkeys H (n = 16), G (n = 32), and M (n = 23).
A total of 397 neurons in four monkeys (H, G, K, and M) were determined to be CD projection neurons (see METHODS) and were related to the performance of the memory-guided saccade task. Among them, 293 CD neurons responded to the visual cue stimulus in the memory-guided saccade task (Kawagoe et al. 1998); other types of neurons (Lauwereyns et al. 2002a; Takikawa et al. 2002a) are not included if they showed no visual response. The data presented in this paper are based on 206 CD neurons for which we were able to examine at least one block of ADR and four blocks of 1DR. They were recorded from monkeys H (n = 84), G (n = 59), K (n = 36), and M (n = 27).
The behavioral sensitivity (indicated by the saccade velocity) to reward was well correlated with the neuronal sensitivity to reward. In the three reward-sensitive monkeys (H, G, and K), a majority of both CD and DA neurons tended to be sensitive to an upcoming reward. In the reward-insensitive monkey (M) a majority of CD and DA neurons were insensitive to reward. In the following we mainly concentrate on three monkeys in which we recorded from both DA neurons and CD neurons (monkeys H, G, and M).
Comparison of neuronal activity between reward-sensitive and reward-insensitive monkeys
Figure 3 shows typical examples of a DA neuron (top) and a CD neuron (bottom) recorded in reward-sensitive monkey H. The data represent the neuronal sensitivity to an upcoming reward. The DA neuron responded to reward with a burst of spikes when the reward was delivered while the monkey was not engaged in a task (data not shown). However, when the same reward was delivered in response to the correct performance of the monkey in ADR or 1DR, DA neurons showed no discernable response except for the first few rewarded trials (Fig. 3, top, shaded parts). Instead, they responded to the cue stimulus in a selective manner in 1DR. In the first block of 1DR in which the right-up (RU) direction was rewarded (RU column), the DA neuron exhibited a burst of spikes after RU cue that indicated an upcoming reward. In the second block in which the left down (LD) direction was rewarded (LD column), the neuron responded to LD cue with a similar burst of spikes, but no longer responded to RU cue. In this way, the DA neuron changed its response completely such that it always responded to the cue that indicated the upcoming reward (Wilcoxon signed-rank test, P < 0.0001). The magnitude of the cue response was invariable whichever cue was rewarded (Kruskal-Wallis test, P = 0.6105). For the other three directions that indicated no reward, the neuron showed some suppression in activity (Wilcoxon signed-rank test, P < 0.0001). These results indicate that DA neurons acquired a particular position-reward association very quickly and flexibly. In contrast, the neuron showed no sign of response in ADR in which reward was delivered in every trial (ALL column).
|
). On the other hand, the CD neuron showed no reward response (Fig. 3, bottom, shaded parts). We found that neuronal activity was remarkably different in the reward-insensitive monkey M (Fig. 4). The DA neuron shown in Fig. 4, top, did not respond to the cue stimulus in either 1DR or ADR, but responded with a burst to reward itself. This pattern was completely opposite to the DA neuron in the reward-sensitive monkeys (Fig. 3, top).
|
The difference in neuronal activity between the reward-sensitive and reward-insensitive monkeys is summarized as population histograms in Fig. 5. DA neurons in the two reward-sensitive monkeys (H and G) were very similar (Fig. 5A, left). While the response to reward was negligible (not shown), the response to the cue stimulus was robust and selective for the expected outcome of reward. The response to the reward-indicating cue (thick black line) appeared as a single positive peak that reached about 16 spikes/s in both monkeys. The response to the nonreward-indicating cue (thick gray line) consisted of an initial small positive peak that was followed by a delayed negative peak, again in both monkeys. On the other hand, DA neurons in reward-insensitive monkey M were completely different as shown in Fig. 5B, left. The cue response was very weak and did not discriminate between the reward-indicating and nonreward-indicating cues.
|
|
) reward selectivity occurred in the reward-sensitive monkeys (Fig. 5A, right), but not in the reward-insensitive monkey M (Fig. 5B, right). A majority of postcue-responsive CD neurons in the reward-sensitive monkeys (104/127, 82%) showed significant reward-dependent modulation of the postcue response (Mann-Whitney U test, P < 0.01, Table 3). Among them 76 neurons showed stronger responses to the reward-indicating cue than to the no-reward-indicating cue (positive reward modulation) as shown in Fig. 5A, whereas in 28 neurons the response was stronger to the no-reward-indicating cue (negative reward modulation). In the reward-insensitive monkey M, only 2 of 20 neurons showed reward-dependent modulation (both positive) (Table 3).
|
The data shown so far indicate that the reward-dependent activity in DA neurons and CD neurons exists in the reward-sensitive monkeys, but not in the reward-insensitive monkey. This raises the possibility that DA and CD neuronal responses are either causally related or share common causes. To study the relationship between DA and CD neurons in the reward-sensitive monkeys, we examined spatial and temporal factors of the neuronal responses (Fig. 6). In Fig. 6A the DA and CD cue responses in the reward-sensitive monkey H are shown separately for the contralateral cue (left) and the ipsilateral cue (right). Only 4 of 42 DA neurons showed significant contralateral versus ipsilateral spatial selectivity (Kruskal-Wallis test, P < 0.01 Table 2). In contrast, CD neurons responded better to the contralateral cue than to the ipsilateral cue provided that it indicated an upcoming reward (Fig. 6). A majority of CD neurons (82/127 in the reward-sensitive monkeys and 16/20 in the reward-insensitive monkey) showed significant spatial selectivity (Table 3). These data indicate that CD neurons receive spatially selective inputs that do not originate from DA neurons.
|
In the second subblock or thereafter, both DA and CD neurons differentiated their cue responses depending on the presence and absence of the upcoming reward. In DA neurons, the response to the reward-indicating cue became positive (i.e., increase from the background activity) while the response to the no-reward-indicating cue became negative (i.e., decrease from the background activity). In CD neurons, the response to the reward-indicating cue increased slightly while the response to the no-reward-indicating cue decreased monotonically.
The data in Fig. 6B suggested that the activity of DA neurons and the activity of CD neurons covaried within one block of 1DR. To illustrate their relationship we replotted the time-course data in Fig. 6B for monkey H as a scatterplot in Fig. 7, but now for monkey G and M as well. Each data point indicates the mean cue response of DA neurons (abscissa) and the mean cue response of CD neurons (ordinate). Only the responses to the contralateral cue are shown. In the reward-sensitive monkeys (H and G) there was a clear positive correlation between the mean cue responses of DA and CD neurons (Spearman 
= 0.869 P < 0.0001, = 0.840 P < 0.0001, respectively). If we consider the CD cue response as a function of the DA cue response, any positive response of DA neurons (compared with the background activity) would result in a positive shift of CD responses (compared with their initial responses in the first subblock); any negative response of DA neurons would result in a negative shift of CD responses. No such relationships were found in the reward-insensitive monkey M (Spearman = -0.317, P > 0.03), possibly because the dynamic range of DA cue responses was negligible.
|
All four monkeys were healthy and had a good appetite throughout the experiments. However, monkey M was remarkably different from the other three monkeys. When humans showed up in front of their home cages, most monkeys would show some signs of affection, aggression, obedience, or withdrawal. Monkey M showed no such sign. When an investigator tried to touch them, most monkeys would avoid the touch. Only when an intimate relationship has been established, monkeys may invite tactile contacts by presenting themselves toward a human mate. Monkey M made no such sign of avoidance or invitation, remaining expressionless. When presented with a favorite food such as banana, most monkeys would grab it and eat it hastily. Monkey M showed no sign of an immediate interest; he would not reach for the food for a while, although he would eat it eventually. A monkey usually discriminated humans, showing emotional attachments with the investigator who worked with the monkey. Monkey M showed no sign of such social discrimination. Although monkey M was usually indifferent to human observers, he was more sensitive to a sudden sensory stimulus, especially sound, by orienting to the source of the stimulus.
Interestingly, monkey M was able to learn saccade tasks, including the memory-guided saccade task, as quickly and accurately as the other monkeys. He even had no difficulty in learning 1DR. Most monkeys were reluctant to, and occasionally refused to, make a saccade to the nonrewarded direction. Monkey M showed no sign of reluctance. However, monkey M tended to get drowsy during experiments, a tendency so robust as to be unusual.
Recording sites of DA neurons
After making more than 20 electrode penetrations to the region around the SNc for each monkey, we could determine the region where DA-like neurons were recorded, on which the subsequent recordings were centered. Near the end of the recording session, we selected representative locations for electrode penetration and, when DA neurons were recorded, we made electrolytic microlesions at the recording sites. Histological examination revealed that the microlesions were located in the region inside or just mediodorsal to the SNc. This was true for both the reward-sensitive monkeys, G (shown in Fig. 8, A-D) and H (not shown), and the reward-insensitive monkey M (Fig. 8, F and H-J). By aligning the sections that contained the microlesions on the adjacent TH-immunostained sections, we found that the microlesions and reconstructed recording sites were among TH-positive neurons that were most likely to be DA neurons for both monkey G (Fig. 8E) and monkey M (Fig. 8G). The lack of behavioral and neuronal sensitivity to the upcoming reward in monkey M raised the possibility that the DA mechanism of monkey M may be abnormal. However, TH immunohistochemistry showed no gross abnormalities.
|
|
|
DISCUSSION |
|---|
|
In the memory-guided saccade task, a saccade-motor instruction is given by a positional cue stimulus. Many CD neurons respond preferentially to the contralateral one (Hikosaka et al. 1989b). By using a one-direction version of the memory-guided saccade task (1DR), we previously found that the CD postcue response was strongly modulated by reward condition (Kawagoe et al. 1998). Typically, the postcue response was much larger when the cue indicated an upcoming reward. 1DR task also gave rise to a robust behavioral outcome: saccades to the rewarded direction were faster and earlier than saccades to the nonrewarded directions (Takikawa et al. 2002b). These observations suggested that CD neurons create the reward-dependent modulation of saccade parameters. This idea was supported by a recent study indicating that CD neuronal activity, including the postcue response, is positively correlated with the velocity of contralateral saccades (Itoh et al. 2003). How then might DA neurons contribute to the neuronal and behavioral outcomes of reward?
Relation between DA neurons and CD projection neurons
There were similarities and differences between DA neurons and CD projection neurons. They were similar in that their cue responses were stronger when the cue indicated an upcoming reward in 1DR than when the cue indicated no reward (positive reward modulation). Whereas this rule applies to all DA neurons, a small fraction of CD neurons showed the opposite reward dependency (negative reward modulation, data not shown). Distinct differences were found in their task specificity, spatial selectivity, and precue activity. DA neurons showed postcue response in 1DR exclusively, whereas CD neurons responded both in ADR and 1DR. DA neurons showed no spatial preference, whereas CD projection neurons usually responded to the contralateral cue rather than to the ipsilateral cue. The spatial preference of CD neurons was present also in ADR. Precue activity was evident in CD neurons (Takikawa et al. 2002a), but not in DA neurons, while postcue activity were evident in both CD and DA neurons. To summarize, the cue response of DA neurons seems to be determined exclusively by the reward contingencies (not by the spatial localizations), whereas that of CD projection neurons is determined by both the reward contingency and spatial localization.
If DA neurons project to the CD and make synaptic connections, the reward contingency for the cue response of CD neurons may, at least partly, be derived from DA neurons. Indeed, the within-block development of DA cue responses was similar to that of CD cue responses.
These results imply that the spatial factor of CD cue responses is derived from non-DA neurons. One plausible candidate would be the inputs from the cerebral cortex, although other inputs such as those from the thalamus (Matsumoto et al. 2001) could also contribute. Cortical inputs to the CD that carry spatial attention or working memory would include the frontal eye field, supplementary eye field, intraparietal sulcus area, and dorsolateral prefrontal cortex (Parthasarathy et al. 1992; Selemon and Goldman-Rakic 1985; Yeterian and Pandya 1991). These cortical signals carrying spatial information may be modulated by DA signals carrying reward information. The modulation would usually be positive because in a majority of CD neurons the cue response increases when reward is expected (Kawagoe et al. 1998). Such a facilitatory effect of DA neurons may be mediated by D1 receptors (Hernández-López et al. 1997; Reynolds et al. 2001). In the small number of CD neurons for which the cue response is depressed if reward is expected (Kawagoe et al. 1998), the inhibitory effect of DA neurons may be mediated by D2 receptors (Hernández-López et al. 2000; West and Grace 2002). In any case, it is plausible to think that the interaction of the spatial signals from the cerebral cortex and the reward-related signals from DA neurons in CD projection neurons are the reason why the monkeys behaved (or made saccades) in a reward-sensitive manner. In monkey M, DA neurons showed no response to the cue stimulus in 1DR, and under our hypothesis CD neurons could not acquire the ability to discriminate between the reward-indicating and no-reward-indicating cues.
Although we favor the hypothesis above, we cannot rule out the possibility that the relationship is reversed: from CD to DA. A main target of CD neurons is the SNr (François et al. 1994; Hedreen and DeLong 1991; Parent and Hazrati 1994). Many SNr neurons send recurrent collateral axons to DA neurons in and around the SNc (Tepper et al. 1995). Since CD projection neurons and SNr projection neurons are both GABAergic, spike activity of CD neurons would lead to disinhibition of DA neurons. Under this alternative, postcue activity of CD neurons might lead to or facilitate postcue activity of DA neurons.
Precue anticipatory activity
The precue anticipatory activity was present in many CD neurons in the reward-sensitive monkeys, but not in the reward-insensitive monkey M. It is unlikely that the precue activity affected saccade parameters directly, because the activity occurred before the motor instruction. Nonetheless, the precue activity might influence the saccade parameters indirectly. As illustrated in previous studies from our laboratory (Lauwereyns et al. 2002a; Takikawa et al. 2002a), the precue activity was present selectively when a particular direction, especially a direction contralateral to the recorded neuron, was rewarded. Many of the CD neurons with precue activity also showed postcue responses with similar direction selectivity. Note, however, the meaning of the direction selectivity was different: selectivity for rewarded direction for precue activity; selectivity for instructed direction for postcue response. This combination of precue activity and postcue activity gave rise to selective boosting of the postcue response when the preferred direction was rewarded. The enhancement appeared as a positive bias or boosting rather than a gain change of the postcue response (Lauwereyns et al. 2002a). The enhanced postcue response would then facilitate saccades.
Our recent studies have suggested that the precue activity could indeed contribute to changes in saccade parameters. When the monkey was instructed to make a saccade to the cue stimulus on its appearance, the precue activity was followed by a saccade immediately (Lauwereyns et al. 2002b). When a contralateral side was rewarded, the precue activity was higher and the saccade latency was shorter. The result is consistent with the hypothesis that stronger precue activity leads to facilitation of a saccade to the contralateral direction. Monkey M would certainly lack the hypothetical facilitatory effects of CD precue activity on contralateral saccades. This might partly account for the lack of reward-dependent modulation of saccades in monkey M.
Possible pathological mechanisms
Monkey M's behavior was clearly different from other monkeys that we have tested. He rarely showed motivated behavior, emotional reactions, or any sign of affect. And yet, monkey M was more sensitive to sudden sensory stimuli than other monkeys. We, the authors, agreed that these behaviors were pathological, likened to human psychiatric or neurological disorders. These behavioral symptoms were somewhat similar to parkinsonism, although monkey M showed no obvious sign of motor deficits. Parkinsonian patients often have poor expression (called "mask-like expression"), like monkey M; they are often reluctant to initiate an action spontaneously (hypokinesia), like monkey M; they often showed reflex blink hyperexcitability (see Basso et al. 1996), as monkey M continued to respond to sudden sensory stimuli. The similarities suggested that the basal ganglia of monkey M did not function normally. In this monkey, DA neurons showed almost no response to the cue stimulus while CD neurons were not influenced by expected reward, unlike the other monkeys. One obvious difference between parkinsonian patients and monkey M was that DA neurons become scarce in parkinsonian patients, but were present and active in monkey M. Histological examination suggested that there were plenty of TH-immunoreactive neurons in the substantia nigra, which probably correspond to DA neurons. Despite the apparently normal chemical feature, these DA neurons did not function normally. It is possible that CD neurons recorded in monkey M may have not acquired the ability to predict an upcoming reward, perhaps due to the lack of reward-predicting signal in DA inputs. Alternatively, we may have recorded from different types of CD neurons in these monkeys. However, the latter explanation seems unlikely because we rarely found reward-predicting responses in CD neurons in monkey M.
On the other hand, monkey M's behavior was somewhat similar to symptoms of schizophrenia or autism. Schizophrenic and autistic patients are often alienated and lack emotional or social contact with other people, like monkey M. In fact, it has long been suspected that DA dysfunctions are involved in schizophrenic symptoms (Weinberger 1987). However, pathophysiology of schizophrenia remains largely elusive. It is largely because there has practically been no primate model of schizophrenia. One way to solve this problem would be to rely on the natural incidence of behavioral disorders, but the probability of finding such behavioral disorders would be very low. One way to circumvent this difficulty would be to collect animals with behavioral disorders from research institutions world wide. Researchers would probably be willing to give away such "psychiatric" animals since they are usually inappropriate for most experiments. The collection would be a precious data base for research on disordered mind and behavior.
|
|
ACKNOWLEDGMENTS |
|---|
|
GRANTS
This work was supported by a Grant-in-Aid for Scientific Research on Priority Areas (C) of Ministry of Education, Culture, Sports, Science, and Technology; Core Research for Evolutional Science and Technology of Japan Science and Technology Corporation; and Japan Society for the Promotion of Science Research for the Future program.
|
|
FOOTNOTES |
|---|
Address for reprint requests and other correspondence: O. Hikosaka, Laboratory of Sensorimotor Research, National Eye Institute, National Institute of Health, 49 Convent Drive, Bldg. 49, Rm. 2A50, Bethesda, MD 20892-4435 (E-mail: oh{at}lsr.nei.nih.gov).
|
|
REFERENCES |
|---|
|
Andén N-E, Carlsson A, Dahlström A, Fuxe K, Hillarp N-A, and Larsson K. Demonstration and mapping out of nigro-neostriatal dopamine neurons. Life Sci 3: 523-530, 1964.
Aosaki T, Graybiel AM, and Kimura M. Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science 265: 412-415, 1994.
Barto AG. Reinforcement learning control. Curr Opin Neurobiol 4: 888-893, 1994.[CrossRef][Medline]
Basso MA, Powers AS, and Evinger C. An explanation for reflex blink hyperexcitability in Parkinson's disease. I. Superior colliculus. J Neurosci 16: 7308-7317, 1996.
Calabresi P, De Murtas M, and Bernardi G. The neostriatum beyond the motor function: experimental and clinical evidence. Neuroscience 78: 39-60, 1997.[CrossRef][Web of Science][Medline]
Delgado MR, Nystrom LE, Fissell C, Noll DC, and Fiez JA. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84: 3072-3077, 2000.
François C, Percheron G, Yelnik J, and Heyner S. A histological atlas of the macaque (Macca mulatta) substantia nigra in ventricular coordinates. Brain Res Bull 14: 349-367, 1985.[CrossRef][Web of Science][Medline]
François C, Yelnik J, Percheron G, and Fénelon G. Topographic distribution of the axonal endings from the sensorimotor and associative striatum in the macaque pallidum and substantia nigra. Exp Brain Res 102: 305-318, 1994.[Web of Science][Medline]
Hedreen JC and DeLong MR. Organization of striatopallidal, striatonigral, and nigrostriatal projections in the macaque. J Comp Neurol 304: 569-595, 1991.[CrossRef][Web of Science][Medline]
Hernández-López S, Bargas J, Surmeier DJ, Reyes A, and Galarraga E. D1 receptor activation enhances evoked discharge in neostriatal medium spiny neurons by modulating an L-type Ca2+ conductance. J Neurosci 17: 3334-3342, 1997.
Hernández-López S, Tkatch T, Perez-Garci E, Galarraga E, Bargas J, Hamm H, and Surmeier DJ. D2 dopamine receptors in striatal medium spiny neurons reduce L-type Ca2+ currents and excitability via a novel PLC[beta]1-IP3-calcineurin-signaling cascade. J Neurosci 20: 8987-8995, 2000.
Hikosaka O, Sakamoto M, and Miyashita N. Effects of caudate nucleus stimulation on substantia nigra cell activity in monkey. Exp Brain Res 95: 457-472, 1993.[Web of Science][Medline]
Hikosaka O, Sakamoto M, and Usui S. Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. J Neurophysiol 61: 780-798, 1989a.
Hikosaka O, Sakamoto M, and Usui S. Functional properties of monkey caudate neurons. II. Visual and auditory responses. J Neurophysiol 61: 799-813, 1989b.
Hikosaka O, Takikawa Y, and Kawagoe R. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev 80: 953-978, 2000.
Hikosaka O and Wurtz RH. Visual and oculomotor functions of monkey substantia nigra pars reticulata. I. Relation of visual and auditory responses to saccades. J Neurophysiol 49: 1230-1253, 1983a.
Hikosaka O and Wurtz RH. Visual and oculomotor functions of monkey substantia nigra pars reticulata. III. Memory-contingent visual and saccade responses. J Neurophysiol 49: 1268-1284, 1983b.
Hollerman JR and Schultz W. Dopamine neurons report an error in the temporal prediction reward during learning. Nat Neurosci 1: 304-309, 1998.[CrossRef][Web of Science][Medline]
Houk JC, Adams JL, and Barto A. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of Information Processing in the Basal Ganglia, edited by JC Houk, JL Davis, and DG Beiser. Cambridge, MA: MIT Press, 1995, p. 249-270.
Itoh H, Nakahara H, Hikosaka O, Kawagoe R, Takikawa Y, and Aihara K. Correlation of primate caudate neural activity and saccade parameters in reward-oriented behavior. J Neurophysiol 89: 1774-1783, 2003.
Judge SJ, Richmond BJ, and Chu FC. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res 20: 535-538, 1980.[CrossRef][Web of Science][Medline]
Kato M, Miyashita N, Hikosaka O, Matsumura M, Usui S, and Kori A. Eye movements in monkeys with local dopamine depletion in the caudate nucleus. I. Deficits in spontaneous saccades. J Neurosci 15: 912-927, 1995.[Abstract]
Kawagoe R, Takikawa Y, and Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1: 411-416, 1998.[CrossRef][Web of Science][Medline]
Knutson B, Westdorp A, Kaiser E, and Hommer D. FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 12: 20-27, 2000.[CrossRef][Web of Science][Medline]
Koepp MJ, Gunn RN, Lawrence AD, Cunningham VJ, Dagher A, Jones T, Brooks DJ, Bench CJ, and Grasby PM. Evidence for striatal dopamine release during a video game. Nature 393: 266-268, 1998.[CrossRef][Medline]
Kori A, Miyashita N, Kato M, Hikosaka O, Usui S, and Matsumura M. Eye movements in monkeys with local dopamine depletion in the caudate nucleus. II. Deficits in voluntary saccades. J Neurosci 15: 928-941, 1995.[Abstract]
Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, and Hikosaka O. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33: 463-473, 2002a.[CrossRef][Web of Science][Medline]
Lauwereyns J, Watanabe K, Coe B, and Hikosaka O. A neural correlate of response bias in monkey caudate nucleus. Nature 418: 413-417, 2002b.[CrossRef][Medline]
Lynd-Balta E and Haber SN. The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum. Neuroscience 59: 625-640, 1994.[CrossRef][Web of Science][Medline]
Matsumoto N, Minamimoto T, Graybiel AM, and Kimura M. Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. J Neurophysiol 85: 960-976, 2001.
Matsumura M, Kojima J, Gardiner TW, and Hikosaka O. Visual and oculomotor functions of monkey subthalamic nucleus. J Neurophysiol 67: 1615-1632, 1992.
Miyashita N, Hikosaka O, and Kato M. Visual hemineglect induced by unilateral striatal dopamine dificiency in monkeys. Neuroreport 6: 1257-1260, 1995.[Web of Science][Medline]
Montague PR, Dayan P, and Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936-1947, 1996.
Nicola SM, Surmeier J, and Malenka RC. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu Rev Neurosci 23: 185-215, 2000.[CrossRef][Web of Science][Medline]
Parent A and Hazrati L-N. Multiple striatal representation in primate substantia nigra. J Comp Neurol 344: 305-320, 1994.[CrossRef][Web of Science][Medline]
Parthasarathy HB, Schall JD, and Graybiel AM. Distributed but convergent ordering of corticostriatal projections: analysis of the frontal eye field and the supplementary eye field in the macaque monkey. J Neurosci 12: 4468-4488, 1992.[Abstract]
Reynolds JN, Hyland BI, and Wickens JR. A cellular mechanism of reward-related learning. Nature 413: 67-70, 2001.[CrossRef][Medline]
Robbins TW and Everitt BJ. Neurobehavioural mechanisms of reward and motivation. Curr Opin Neurobiol 6: 228-236, 1996.[CrossRef][Web of Science][Medline]
Robinson DA. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans Biomed Eng 10: 137-145, 1963.[Medline]
Sato M and Hikosaka O. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J Neurosci 22: 2363-2373, 2002.
Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 80: 1-27, 1998.
Schultz W, Dayan P, and Montague PR. A neural substrate of prediction and reward. Science 275: 1593-1599, 1997.
Schultz W and Romo R. Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol 57: 201-217, 1987.
Selemon LD and Goldman-Rakic PS. Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey. J Neurosci 5: 776-794, 1985.[Abstract]
Takikawa Y, Kawagoe R, and Hikosaka O. Reward-dependent spatial selectivity of anticipatory activity in monkey caudate neurons. J Neurophysiol 87: 508-515, 2002a.
Takikawa Y, Kawagoe R, Itoh H, Nakahara H, and Hikosaka O. Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142: 284-291, 2002b.[CrossRef][Web of Science][Medline]
Tepper JM, Martin LP, and Anderson DR. GABAA receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons. J Neurosci 15: 3092-3103, 1995.[Abstract]
Weinberger DR. Implications of normal brain development for the pathogenesis of schizophrenia. Arch Gen Psychiatry 44: 660-669, 1987.
West AR and Grace AA. Opposite influences of endogenous dopamine D1 and D2 receptor activation on activity states and electrophysiological properties of striatal neurons: studies combining in vivo intracellular recordings and reverse microdialysis. J Neurosci 22: 294-304, 2002.
Wise RA. Neurobiology of addiction. Curr Opin Neurobiol 6: 243-251, 1996.[CrossRef][Web of Science][Medline]
Yeterian EH and Pandya DN. Prefrontostriatal connections in relation to cortical architectonic organization in rhesus monkeys. J Comp Neurol 312: 43-67, 1991.[CrossRef][Web of Science][Medline]
This article has been cited by other articles:
|
|
 |
|
  K. Nakamura, M. Matsumoto, and O. Hikosaka Reward-Dependent Modulation of Neuronal Activity in the Primate Dorsal Raphe Nucleus J. Neurosci., May 14, 2008; 28(20): 5331 - 5343. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. Uchida, X. Lu, S. Ohmae, T. Takahashi, and S. Kitazawa Neuronal Activity Related to Reward Size and Rewarded Target Position in Primate Supplementary Eye Field J. Neurosci., December 12, 2007; 27(50): 13750 - 13755. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
B. Pasquereau, A. Nadjar, D. Arkadir, E. Bezard, M. Goillandeau, B. Bioulac, C. E. Gross, and T. Boraud Shaping of Motor Responses by Incentive Values through the Basal Ganglia J. Neurosci., January 31, 2007; 27(5): 1176 - 1183. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Nakamura and O. Hikosaka Facilitation of Saccadic Eye Movements by Postsaccadic Electrical Stimulation in the Primate Caudate J. Neurosci., December 13, 2006; 26(50): 12885 - 12895. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. N. Haber, K.-S. Kim, P. Mailly, and R. Calzavara Reward-Related Cortical Inputs Define a Large Striatal Region in Primates That Interface with Associative Cortical Connections, Providing a Substrate for Incentive-Based Learning. J. Neurosci., August 9, 2006; 26(32): 8368 - 8376. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Ding and O. Hikosaka Comparison of reward modulation in the frontal eye field and caudate of the macaque. J. Neurosci., June 21, 2006; 26(25): 6695 - 6703. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Nakamura and O. Hikosaka Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J. Neurosci., May 17, 2006; 26(20): 5360 - 5369. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 O. Hikosaka, K. Nakamura, and H. Nakahara Basal Ganglia Orient Eyes to Reward J Neurophysiol, February 1, 2006; 95(2): 567 - 584. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Lehericy, E. Bardinet, L. Tremblay, P.-F. Van de Moortele, J.-B. Pochon, D. Dormont, D.-S. Kim, J. Yelnik, and K. Ugurbil Motor control in basal ganglia circuits using fMRI and brain atlas approaches Cereb Cortex, February 1, 2006; 16(2): 149 - 161. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Watanabe and O. Hikosaka Immediate Changes in Anticipatory Activity of Caudate Neurons Associated With Reversal of Position-Reward Contingency J Neurophysiol, September 1, 2005; 94(3): 1879 - 1887. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. R. Roesch and C. R. Olson Neuronal Activity Dependent on Anticipated and Elapsed Delay in Macaque Prefrontal Cortex, Frontal and Supplementary Eye Fields, and Premotor Cortex J Neurophysiol, August 1, 2005; 94(2): 1469 - 1497. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. N. Tobler, C. D. Fiorillo, and W. Schultz Adaptive Coding of Reward Value by Dopamine Neurons Science, March 11, 2005; 307(5715): 1642 - 1645. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. Takikawa, R. Kawagoe, and O. Hikosaka A Possible Role of Midbrain Dopamine Neurons in Short- and Long-Term Adaptation of Saccades to Position-Reward Mapping J Neurophysiol, October 1, 2004; 92(4): 2520 - 2529. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
TOP
ABSTRACT
INTRODUCTION
