| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1Institute of Human Science and Biomedical Engineering, National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, Japan; and 2Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland
Submitted 6 January 2005; accepted in final form 1 May 2005
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
). Although there are numerous ways to characterize "rules," this study was concerned about generalizability or transferability. For example, learning becomes progressively easier as a result of previous experience with similar problems. This increased ability to solve similar problems, namely learning set, is thought to reflect the affirmation or rejection of hypotheses or rules (Harlow 1949). In a simplest form, learning sets can be studied as "reversal sets" (Harlow 1950; Meyer 1951; Treichler and Petros 1983), where animals learn to switch several stimulus-reward associations. This study aimed at examining neuronal indications of a reversal set in the primate caudate nucleus, more specifically, immediate changes in caudate anticipatory activity on switches of the position-reward contingency.
Many studies of behavioral switches in primates exist, but studies of the underlying neural mechanisms have begun only recently, after advances in neuronal recording and experimental paradigms allowed researchers to study single neurons over entire learning episodes within an experimental session (Assad et al. 1998; Chen and Wise 1995a,b; Mitz et al. 1991; Nakamura et al. 1998; Tremblay and Schultz 2000; Tremblay et al. 1998). Neuronal changes paralleling behavioral changes during learning were found in various cortical and subcortical regions (Chen and Wise 1995a,b; Mitz et al. 1991; Nakamura et al. 1998; Niki et al. 1990; Pasupathy and Miller 2005; Rolls et al. 1996; Thorpe et al. 1983; Tremblay and Schultz 2000; Tremblay et al. 1998; Watanabe 1990). Some studies showed rapid neuronal changes following changes in stimulus-response (e.g., Assad et al. 1998) or stimulus-reward contingency (e.g., Tremblay et al. 1998).
It has been suggested that the basal ganglia play a critical role in various selection/switching behaviors (e.g., Redgrave et al. 1999; Rolls 1994). The striatum (caudate nucleus, putamen, and ventral striatum) is the main input station of the basal ganglia and receives glutamate-mediated excitatory inputs from all areas of the cortex, as well as afferents from the thalamus and limbic structures such as the hippocampus and amygdala (Parent 1990; Parthasarathy et al. 1992). Neurons in the primate striatum show a variety of activity related to rewards, expectation of external events that are behaviorally relevant (reward-predicting or movement-eliciting stimuli), and movement preparation (Alexander et al. 1986; Apicella et al. 1992; Hikosaka et al. 1989a–c; Rolls 1994; Schultz et al. 1992; Watanabe et al. 2003b). The striatum is therefore thought to be one of the primary sites where sensory, motor, cognitive, and motivational signals interact. In most cases, the expectation of reward seems to be a central component of the striatal activities (Hollerman et al. 1998; Schultz et al. 1992). We chose the head and body of the caudate nucleus as the target recording sites because these subregions are known to be related to the generation of saccadic eye movements (Hikosaka et al. 1989a, 2000; Watanabe et al. 2003b) as well as the coding of position-reward contingencies (Kawagoe et al. 1998; Lauwereyns et al. 2002b; Takikawa et al. 2002).
The main purpose of this study was to find manifestations of immediate and robust changes of neural activity. To this end, we examined changes in oculomotor behavior (saccade latency) after switches of position-reward contingency and recorded neuronal activity from the primate caudate nucleus. We focused on the spatially tuned anticipatory activity of primate caudate neurons. In many caudate neurons, the anticipatory activity is tuned to the position-reward contingency, not simply to the position or the reward expectation. Some caudate neurons exhibit reward associations with other stimulus features such as color (Lauwereyns et al. 2002a), indicating that the caudate anticipatory activity can represent various types of reward association and expectation. It also shows a clear relationship with saccade latency, presumably reflecting the animal's motivational state or response bias (movement preparation) toward the rewarded position (Lauwereyns et al. 2002b). Our current hypothesis for the functional significance of spatially tuned anticipatory activity is that it represents motivationally biased motor signals, which may be projected to the superior colliculus (Hikosaka et al. 2000), where both bias-type and gain-type modulations are observed (Ikeda and Hikosaka 2003). Yet, importantly, the anticipatory activity itself does not trigger eye movements but modulates subthreshold activity in the superior colliculus. This way, they eventually modify the goal-directed behavior (eye movements in this case) in a reward-dependent manner. Thus the caudate anticipatory activity seems to be suitable for examining the relationship between internal motivational sets (e.g., position-reward associations) and oculomotor behaviors.
To efficiently study changes in behavior and neuronal activity, we employed a reversal-learning paradigm (Assad et al. 1998; Rolls et al. 1996). Within a block of 20 completed trials, the reward was mapped consistently onto one target position (Fig. 1). The position-reward association remained constant within a block but was reversed frequently and automatically between blocks. The switch of the position-reward contingency occurred without any indication to the monkey. Consequently, the animal had to learn the switched contingency by trial and error, specifically by detecting an unexpected reward delivery or an unexpected reward omission. This feature allowed us to study the role of previous experiences of position-reward associations. Specifically, we were interested in whether changes in saccadic latency and neuronal activity depend on the reward history (i.e., whether the first trial was rewarded or not, and whether the first and second trials were in same or different reward conditions).
|
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
Two adult male Japanese monkeys (monkey A and monkey B; Macaca fuscata) were used (body weight, 6.0–7.5 kg). The monkeys received dry pellets and small amounts of fresh fruit or vegetables in their home cages. During periods of training and experiments, the monkeys' access to water in the cage was controlled and monitored.
We implanted a head-holding device, a chamber for unit recording, and a scleral search coil under general anesthesia. The monkey was sedated with ketamine (4.6–6.0 mg/kg) and xylazine (1.8–2.4 mg/kg) given intramuscularly, and general anesthesia was induced by intravenous injection of pentobarbital sodium (4.5–6.0 mg/kg/h) with butorphanol tartrate (0.02 mg/kg/h). After the skull was exposed, 10–15 acrylic screws were bolted into it. The screws acted as anchors to which a plastic head holder and chamber were fixed to the skull with dental acrylic resin. A recording chamber (antero-posterior: 42 mm; lateral: 30 mm; depth: 10 mm) was placed over the fronto-parietal cortices, tilted laterally by 35° in the coronal plane and was aimed at the head and the body of the caudate nucleus based on magnetic resonance imaging (AIRIS, 0.3 T; Hitachi, Tokyo, Japan). A scleral eye coil was implanted in one eye for monitoring eye position (Judge et al. 1980). The monkey received antibiotics (sodium ampicillin 25–40 mg/kg im each day) after the operation. All surgical and experimental procedures conformed to the National Institutes of Health Principles of Laboratory Animal Care (National Institutes of Health publication no. 86–23, revised 1985) and were approved by the Juntendo University Animal Care and Use Committee.
Behavioral task
The monkey sat in a primate chair inside a dark sound-attenuated room with his head being immobilized. The visual stimuli were small red spots of light, 0.2° diam, back-projected onto a tangent screen by LED projectors. In each trial the monkey was required to direct and maintain his gaze at a central fixation spot during a first fixation ("pretarget") period of 1,500 ms. After the pretarget period, the fixation spot disappeared and a peripheral target appeared at 20° to the left or to the right. The monkey had to make a saccade within 500 ms to within 3° of the target position. An auditory tone of a 800-Hz rectangular waveform followed each completed saccade. If the monkey made a fixation break or a late or inaccurate saccade, the same trial was repeated.
To study the influence of incentive on eye movement behavior and caudate neuronal activity, we used an asymmetrical reward schedule (biased-saccade task; Fig. 1). Within a block of 20 completed trials, reward (a drop of water) was mapped consistently onto one target position and never on the other position. Because the position of the target was randomized and counterbalanced within a block, the monkey was rewarded in only one-half of the trials in a given block. The position-reward contingency remained constant within a block but was reversed frequently (22–51 times during behavioral sessions, 6–16 times during recording sessions) and automatically, without any indication to the monkey or any pause between blocks. Consequently, the switch of the position-reward contingency could be detected only on the basis of the unexpected reward delivery and the unexpected lack of reward. During behavioral sessions, the intertrial interval was fixed within a session but varied among sessions (1,500, 3,000, and 6,000 ms).
Electrophysiological recording
Eye position was measured with a standard magnetic search-coil (MEL-25, Enzanshi-Kogyo, Tokyo, Japan) technique (Judge et al. 1980), digitized at 500 Hz, and stored with event times for off-line analysis. During recording sessions, action potentials of single neurons were recorded with tungsten electrodes (impedance, 1.5–3 MOhm; FHC, Bowdoinham, ME). Microelectrodes were advanced perpendicularly to the cortical surface using an oil-driven micro-manipulator (MO-95, Narishige, Tokyo, Japan). The action potentials were amplified, filtered (500 Hz to 2 kHz), and processed by a window discriminator (MDA-4 and DDIS-1, BAK Electronics, Germantown, MD). We selected extracellular neural activity of presumed projection neurons, which show very low spontaneous activity (0.01–1 Hz), but not of presumed interneurons, which show irregular tonic activity (2–10 Hz; Aosaki et al. 1994; Kimura 1986). For the purpose of this study, we searched selectively for neurons that showed anticipatory activity during the pretarget period while the monkey performed the task. When we encountered such a projection neuron by visual inspection, we proceeded with recording the neuron in as many trials as possible. During neuronal recording sessions, the ITI was fixed at 3,000 ms. We conducted behavioral and recording sessions separately because it was impractical to keep single neurons isolated for multiple sessions, and a comparison between neural activity and saccade latency within the same session was practically difficult.
Data analysis
We used the following procedure to determine the time of saccade initiation. An eye movement was judged as a possible saccade if its velocity and acceleration exceeded predetermined threshold values (30 and 90°/s2, respectively). To be accepted as a saccade 1) the velocity must exceed 45°/s after the onset, 2) this suprathreshold velocity must be maintained for 
10 ms, and 3) the total duration must be >25 ms. The end of the eye movement was determined when the velocity became <40°/s. These threshold values were determined empirically by applying them to sample saccades, which led to almost perfect detection of saccades in the trained monkeys.
Neurons were classified as anticipatory neurons (Fig. 2) if they showed a statistically reliable increase in the average number of spikes in the window of –1,500 to 0 ms from target onset (anticipatory activity) compared with the activity before the onset of the fixation spot (from –1,000 to 0 ms from fixation onset). All comparisons of average firing rates were evaluated by a two-tailed t-test, using a significance level of P < 0.05. After selecting anticipatory neurons, we examined whether their anticipatory activity systematically changed depending on the position-reward contingency and showed a significant difference between the two position-reward contingencies (unpaired t-test between contralateral vs. ipsilateral reward conditions; P < 0.05).
|
After the switch of the position-reward contingency, the animal would experience a surprising position-reward event. For instance, in a left-reward block, the animal would make quick saccades to the left position and delayed saccades to the right position (Watanabe et al. 2003a). In the first trial after the contingency switch (now in a right-reward block), if the target appears on the left, the animal would make a quick saccade to it (because the animal could not know about the contingency switch) but receive no reward (unexpected omission of reward). In the second trial, if the target again appears on the left, the saccade latency would become longer. However, what would the saccade latency be if the target appears on the right in the second trial? After the contingency switch, the animal had not encountered an event where a rightward saccade was associated with reward. If the latency of the saccade to the right position becomes significantly shorter than those in the previous block, this suggests that the reversal set for this particular task is effectively learned (Harlow 1950; Meyer 1951; Treichler and Petros 1983). Observing these behavioral results, we asked the same question about the anticipatory activity of caudate projection neurons (Fig. 2). Would caudate anticipatory neurons change their activity immediately after the first (surprising) trial?
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
For behavioral sessions, monkey A completed 469 blocks, and monkey B completed 319 blocks. For both monkeys, saccade latency clearly changed after the switches of the position-reward contingency (Fig. 3, top). Learning curves were stable during the experiment after the extensive training (>6 mo). After learning reached asymptote levels (trials 6–20), saccade latency was significantly shorter in rewarded trials than unrewarded trials for both monkeys (unpaired t-test; P < 0.001). The changes in saccade latency occurred mainly within the first few trials.
|
In recording sessions, we encountered a total of 426 neurons in three caudate nuclei of the two monkeys. Among them, 338 neurons were judged as projection neurons by their low spontaneous activity, and the remaining 88 neurons were classified as tonically active neurons (Aosaki et al. 1994). Of the 338 putative projection neurons, 104 neurons (104/338, 31%) appeared to have task-related activity, with 46 neurons (46/104; 44%) that were judged to show elevated anticipatory activity as compared with intertrial activity. Forty-one of these neurons (16 neurons from monkey A, 25 neurons from monkey B) had sufficient data for statistical analyses (160 trials, i.e., 8 reversals). All of these 41 neurons showed a statistically reliable increase in the neuronal activity in the pretarget period compared with the control period (2-tailed paired t-test, P < 0.05) and so were classified as anticipatory neurons (Fig. 2). Thirty-one of the 41 anticipatory neurons (76%) systematically changed their anticipatory activity depending on the position-reward contingency. Twenty-five of the 31 contingency-sensitive anticipatory neurons showed stronger anticipatory activity when the contralateral position was associated with reward than when the ipsilateral position was associated with reward (25/31; 81%; contra-bias neurons). The remaining six neurons (6/31; 19%) will be referred to as ipsi-bias neurons.
Neuronal results
We focused on the neuronal data from the 31 anticipatory neurons that showed sensitivity to the position-reward contingency and analyzed the changing dynamics of anticipatory activity after contingency switches and its dependency on previous reward history. Neuronal activity was normalized within each neuron with respect to the neuron's maximum firing rate, and the data from all appropriate trials (in error-free blocks) were combined. The bottom panels of Fig. 3 show the mean (normalized) firing rate as a function of the number of trials after the contingency switch. The black line shows mean anticipatory activity in blocks where reward was associated with the neurons' preferred positions (preferred blocks; contralateral for contra-bias neurons and ipsilateral for ipsi-bias neurons). The gray line represents the mean anticipatory activity in blocks where reward was associated with positions opposite to the neurons' preferred positions (nonpreferred blocks; ipsilateral for contra-bias neurons and contralateral for ipsi-bias neurons). The anticipatory activity showed a remarkable plasticity contingent on the position-reward mapping (Fig. 3, bottom), which appeared to parallel the changes in saccade latency (Fig. 3, top), consistent with Lauwereyns et al. (2002b). The changes in anticipatory activity occurred mainly within the first few trials.
Reward-history analyses
For simplicity, only blocks in which the monkeys made correct responses for the next two trials after a contingency change were included in the reward-history analyses.1
FOR BEHAVIORAL DATA.
Obviously, in the first trial after a contingency switch, the monkey would expect reward based on the previous (opposite) contingency. We refer to rewarded and unrewarded trials after the contingency change as R and U, respectively (Fig. 4). The second trials after the contingency change consisted of four types: RR (1st rewarded 
2nd rewarded), UR (1st unrewarded 2nd rewarded), RU (1st rewarded 2nd unrewarded), and UU (1st unrewarded 2nd unrewarded). Mean saccade latencies in the initial two trials after the contingency switch are shown in the top panels of Fig. 5. In the first trials after the contingency switch (R and U), the saccade latency was shorter for unrewarded directions (U) than for reward directions (R; unpaired t-test P < 0.01). This indicates that the monkeys followed the previous (opposite) position-reward contingency. In the second trials, the saccade latency already showed significant changes; rewarded trials tended to produce shorter saccade latencies than unrewarded trials (unpaired t-test, P < 0.01). This held true for all four trial types: RR, UR, RU, and UU. In other words, the saccade latency changed just after completing a single (surprising reward/unrewarded) trial following the position-reward contingency switch.
|
|
; see also Fig. 2). That is, the reward condition (and target position) in the ongoing trial is irrelevant for the analysis of the neuronal data. If the change in neural activity is based on simple position-reward association, anticipatory activity in the second trial would depend on the particular reward condition and target position of the first trial. On the other hand, if the reversal set has been well established, anticipatory activity would be determined mainly by the reward-position contingency of the current block (p = in the neuron's preferred block, or n = in the neuron's nonpreferred block). In the first trial of a preferred block (n p), the neuron's anticipatory activity occurs before the current trial is known to be R or U. Thus the neuron's state is denoted as 0-p. After receiving the unexpected reward in the first trial (R), the neuron's state becomes R-p; if the first trial was unrewarded, the neuron's state becomes U-p. Similarly, in the first trial of the nonpreferred block (p n), the neuron's state becomes 0-n. The state in the second trial is R-n if the first trial is rewarded (R) or U-n if the first trial is unrewarded (U). The main question we addressed in the reward-history analysis was as follows: is a single trial sufficient to induce the change in neuronal activity, irrespective of the trial type? In the first trials after the contingency switch, anticipatory activity was high in nonpreferred blocks (0-n; red lines) and low in preferred blocks (0-p; blue lines; unpaired t-test P < 0.01). This was expected because there was no explicit indication of the switch of the position-reward contingency between blocks. There were significant changes in neuronal activity in all the transitions from the first to the second trials (0-n to R-n; 0-n to U-n; 0-p to R-p; 0-p to U-p; unpaired t-test, P < 0.01). Thus both the unexpected reward delivery and the unexpected lack of reward delivery had immediate effects on the anticipatory activity of the caudate projection neurons. To summarize, the results supported the established reversal set at the level of caudate anticipatory activity.
Additionally, an inspection of the neuronal results (Fig. 5, bottom) suggested that the anticipatory neural activity in the second trials tended to be larger when the previous (1st) trial was rewarded (R-p > U-p; R-n > U-n; i.e., solid lines > hatched lines). A two-way ANOVA on the data of the second trials (R-p, R-n, U-p, U-n; R-U vs. p-n) confirmed this observation; both main effects of position-reward contingency [F(1,167) = 11.14, P < 0.01] and of reward condition in the first trials [F(1,167) = 3.98, P < 0.05] were significant, with no significant interaction [F(1,167) = 0.029, P = 0.86]. In short, the position-reward contingency and the presence of reward seemed to have independent influences on the anticipatory activity.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
). In this experiment, there was no cue that the monkey could use to predict the occurrence of the contingency switch (unless the monkeys explicitly counted successfully completed trials, of which we found no indication). Therefore in the first trial after the contingency switch, as the monkey followed the position-reward contingency of the previous block, the saccade latency was longer for a rewarded position and shorter for an unrewarded position. Then, contrary to expectation, the monkey received the water reward or did not receive it. After those surprising reward events, the saccade latencies showed significant changes compared with those in the first trials. This is not so surprising when the first and the subsequent trials were in the same reward condition (RR and UU). However, the significant changes in saccade latency were observed even when the previous reward conditions were different from that in the current trial (UR and RU). In these conditions, the monkey had not encountered the position-reward association of the current trial after the contingency switch. This suggests that the monkeys established a reversal set which is related to the rule of the asymmetrical reward paradigm (i.e., if 1 position was rewarded, the other position was not rewarded) and employed it effectively following each position-reward contingency switch (Harlow 1950).
Anticipatory activity in many caudate neurons is sensitive to position-reward contingency (Lauwereyns et al. 2002b; Takikawa et al. 2002). Similar to the changes in saccade latency, the changes in the anticipatory activity required only a single trial after the contingency switch. Importantly, this study showed that the two types of unexpected reward events were both effective in inducing the significant changes in the anticipatory activity. In other words, to induce the change in the caudate anticipatory activity, it was not required for the monkey to experience the two possible position-reward associations. This is again a clear sign of the application of the reversal set. Evidently, the roles of the primate caudate nucleus are not limited to simple stimulus-response or stimulus-reward associations. These findings lead us to propose that the primate caudate nucleus can reflect cognitive sets (exemplified by reversal sets) to induce immediate changes in neural activity and oculomotor behaviors.
Another interesting observation is that the anticipatory neural activity in the second trials tended to be larger when the previous (1st) trial was rewarded (Fig. 5). For both monkeys, the experience of reward in the first trials heightens caudate anticipatory activity, irrespective of the neuron's preferred position. However, such overall activities in neuronal activity were not consistently related to the saccade latencies (Fig. 6, left). On the other hand, the neuronal bias (expressed as difference between mean activity in the preferred and nonpreferred conditions) can reliably be related to the bias in saccade latency (expressed as difference between mean latency of rewarded and that of unrewarded saccades; Fig. 6, right). The neuronal bias roughly corresponds to the difference in anticipatory activity between the two caudate nuclei, because the preferred condition for most neurons in the right caudate (i.e., left-rewarded) corresponds to the nonpreferred condition for most neurons in the left caudate. That is, the bias in saccade behavior could be comprehended more reliably in terms of a neuronal competition between two opposing motor preparation processes in the caudate nuclei on the opposite sides.
|
Researchers have studied neural plasticity in various brain areas by using learning sets, where single neurons can be studied over entire learning episodes (Assad et al. 1998; Chen and Wise 1995a,b; Mitz et al. 1991; Nakamura et al. 1998; Tremblay and Schultz 2000; Tremblay et al. 1998). For example, Schultz et al. (2003) examined how reward expectation-related activity is modulated during learning in the monkey striatum, showing how the striatum neurons learn novel stimulus-reward associations. Although the present study has similarities with previous learning set experiments, it has at least two new and important features.
First, this study investigated the effect of an uncued switch of the stimulus(position)-reward contingency on striatal, in particular caudate, neurons. For example, in previous learning set experiments on striatal neurons (e.g., Tremblay et al. 1998), there were cues indicating the beginning of a new session. In previous studies of caudate neurons in our laboratory (Kawagoe et al. 1998, 2004; Takikawa et al. 2002), there were long breaks between blocks, which could signal the change of the position-reward contingency. Several studies have employed paradigms similar to ours (i.e., no transition cue between blocks) to force monkeys to learn stimulus-response or stimulus-reward contingencies by trial and error (Assad et al. 1998, 2000; Rolls et al. 1996; Thorpe et al. 1983), but these studies focused mainly on prefrontal neurons and did not take into account possible roles of reward history (type of feedback) in behavioral switch and neural change.
Second, this study is the first to examine the effect of reward-history on both saccade latency and caudate neural activity during reversal learning. Effects of previous trials on saccade latency and neural activity have been reported in the superior colliculus by using simple sensorimotor tasks (Dorris et al. 1999, 2000; Fecteau and Munoz 2003) and in the caudate nucleus by using a memory-guided saccade task (Itoh et al. 2003). These studies showed the effects of reward history on established performance, which could be traces of simple residual or priming effects. Our study focused on the learning process before the reward-based performance was established within a block. During associative learning, behavioral and neuronal changes associated with the contingency switch involve more than such passive effects of reward-history (Assad et al. 2000; Miller et al. 2003; Wallis et al. 2001). This is particularly true when changes in behavior and neural activity are based partly on rules rather than simple stimulus-response associations.
Origins of immediate neuronal changes in caudate neurons
The immediate neuronal changes could be due to afferent modulatory inputs to caudate projection neurons. One possible brain region that may provide such modulatory input is the substantia nigra pars compacta (SNc) (Kawagoe et al. 2004; Takikawa et al. 2004). Midbrain dopamine neurons in primates respond to unexpected reward and conditioned reward-predicting stimuli by phasic activation. They also show depression of activity when reward is unexpectedly omitted (Schultz et al. 1997) or shifted (Hollerman and Schultz 1998). Dopamine neurons thus seem to signal the extent to which the rewarding outcome deviates from the prediction (prediction error; Schultz 2002; Schultz et al. 1997) and have the formal characteristics of reinforcement signals for acquiring new behavioral reactions (Barto 1994; Montague et al. 1996; Schultz 2002; Waelti et al. 2001). In this study, both unpredicted reward delivery and unpredicted reward omission were effective in inducing the immediate neural plasticity of the caudate anticipatory neurons. Because the dopamine response transmits a reward prediction error with a short latency through diverging connections to the striatum and frontal cortex (Schultz 2002), it can serve as "broadcasting" signal for quick changes in multiple brain regions, which might induce the immediate change in caudate neuronal activity (Kawagoe et al. 2004; Takikawa et al. 2004).
Another candidate brain region that participates in the immediate change of caudate anticipatory activity is the prefrontal cortex. The primate prefrontal cortex has massive anatomical connections with the basal ganglia, forming several distinct parallel functional loops (Alexander et al. 1986). The striatum, including the caudate nucleus, is the primary region that receives the input from the prefrontal cortex. Recent studies on prefrontal functions have pointed to their role in the adaptive changes of stimulus representation. During behavioral reversal, neurons in the prefrontal cortex show remarkable sensitivity to stimulus-reward (Rolls et al. 1996; Watanabe 1990; Watanabe et al. 2002) and stimulus-response (Assad et al. 1998; Niki et al. 1990) associations. Alternations of these associations produce rapid gain and loss of response to the alternate stimuli, which are accompanied by equally rapid changes in the animal's behavior. Such changes in prefrontal neural activity may signify changes in the rules that the animal follows. Moreover, recent studies have shown that the prefrontal cortex is the key site where abstract rules are acquired and represented. Some sensory and motor-related neurons in the prefrontal cortex change their activity on the basis of the abstract rule (or behavioral context) that the animal is presently employing (Assad et al. 2000; Miller et al. 2003; Sakagami et al. 2001; Wallis et al. 2001; White and Wise 1999). This rule-sensitive neural activity in the prefrontal cortex could be involved in changing the neural processing that underlies the behavioral change (e.g., in the premotor cortex, Wallis and Miller 2003; in the anterior cingulate, Shima and Tanji 1998) and could signal changes of the behavioral context to caudate projection neurons.
In line with the possible involvement of the prefrontal cortex, anticipatory activation for task specific events has also been reported in the prefrontal and premotor cortex in primates (Coe et al. 2002; Kobayashi et al. 2002; Sakagami and Niki 1994; Tremblay and Schultz 2000; Watanabe 1996). Because the central part of the caudate nucleus (from which we recorded the projection neurons) receives massive projections from these cortical regions (Parthasarathy et al. 1992; Selemon and Goldman-Rakic 1985; Yeterian and Pandya 1991), it is possible that caudate anticipatory activity is derived from cortical anticipatory inputs. In fact, anticipatory activity in the frontal eye field shows characteristics similar to that in the caudate nucleus (Bruce and Goldberg 1985). The anatomical connection between these brain structures (monosynaptic projections from the frontal eye field to the caudate) implies a possible functional relationship between them. Also, Kobayashi et al. (2002) have shown that some neurons in the dorsolateral prefrontal cortex show similar anticipatory activity before the explicit instruction. Furthermore, a recent study have reported that, during associative learning in monkeys, neuronal activity in the striatum shows more rapid, almost bistable, changes than that in the prefrontal cortex (Pasupathy and Miller 2005). Therefore it is possible that the basal ganglia lead the prefrontal cortex in learning new position-reward associations (Bar-Gad et al. 2003; Houk and Wise 1995). Given these findings, it would be particularly interesting to record neuronal activity simultaneously in the frontal eye field and the caudate in future studies.
The above-mentioned hypotheses of afferent modulatory inputs partly and implicitly assume that motor preparation or bias in the caudate nuclei on the opposite sides independently influence on the activity of the superior colliculus. Another, not mutually exclusive, possibility is that the immediate changes in caudate activity (and consequently changes in saccadic latency) are implemented by a competition between two opposing motor preparation processes that are mutually inhibitory (as implied by Fig. 6). When one of this population of neurons experiences a "surprise" event on the first trial after the switch, its activity either increases or decreases depending on whether it was a rewarded or unrewarded trial. This increased or decreased activity causes the activity in the opposing network to decrease or increase accordingly. However, it is unclear if such a mutually inhibitory network is based on pre-existing anatomical connections or is a consequence of long-term experimental training. It would be interesting to study how neuronal networks that enable the animal to form reversal sets are established.
|
ACKNOWLEDGMENTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
|
FOOTNOTES |
|---|
1 Plotting saccade latency and neural activity as a function of trial number (Fig. 3) partly showed the rapid behavioral and neural changes but did not unquestionably show an immediate and robust change. This was because there were four possible trial types for both behavioral measure (saccade latency) and neural measure (anticipatory activity), and the data shown in Fig. 3 were from combinations and means of these trial types. 
Address for reprint requests and other correspondence: K. Watanabe, Inst. of Human Science and Biomedical Engineering, National Inst. of Advanced Industrial Science and Technology, AIST Tsukuba Central 6, 1-1-1, Higashi, Tsukuba, Ibaraki 305-8566, Japan (E-mail: katsumi.watanabe{at}aist.go.jp)
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
Aosaki T, Tsubokawa H, Ishida A, Watanabe K, Graybiel AM, and Kimura M. Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning. J Neurosci 14: 3969–3984, 1994.[Abstract]
Apicella P, Scarnati E, Ljungberg T, and Schultz W. Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J Neurophysiol 68: 945–960, 1992.
Assad WF, Rainder G, and Miller EK. Neural activity in the primate prefrontal cortex during associative learning. Neuron 21: 1399–1407, 1998.[CrossRef][ISI][Medline]
Assad WF, Rainder G, and Miller EK. Task-specific activity in the primate prefrontal cortex. J Neurophysiol 84: 451–459, 2000.
Bar-Gad I, Morris G, and Bergman H. Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog Neurobiol 71: 439–473, 2003.[CrossRef][ISI][Medline]
Barto AG. Reinforcement learning control. Curr Opin Neurobiol 4: 888–893, 1994.[CrossRef][Medline]
Bruce CJ and Goldberg ME. Primate frontal eye fields. I. Single neurons discharging before saccades. J Neurophysiol 53: 603–635, 1985.
Chen LL and Wise SP. Neuronal activity in the supplementary eye field during acquisition of conditional oculomotor associations. J Neurophysiol 73: 1101–1121, 1995a.
Chen LL and Wise SP. Supplementary eye field contrasted with the frontal eye field during acquisition of conditional oculomotor associations. J Neurophysiol 73: 1122–1134, 1995b.
Coe B, Tomihara K, Matsuzawa M, and Hikosaka O. Visual and anticipatory bias in three cortical eye fields of the monkey during an adaptive decision-making task. J Neurosci 22: 5081–5090, 2002.
Dorris M, Paré M, and Munoz DP. Immediate neural plasticity shapes motor performance. J Neurosci 20: 1–5, 2000.
Dorris MC, Taylor TL, Klein RM, and Munoz DP. Influence of previous visual stimulus or saccade on saccadic reaction times in monkey. J Neurophysiol 81: 2429–2436, 1999.
Fecteau JH and Munoz DP. Exploring the consequences of the previous trial. Nat Rev Neurosci 4: 1–9, 2003.[Medline]
Harlow HF. The formation of learning sets. Psychol Rev 56: 51–65, 1949.[Medline]
Harlow HF. Analysis of discrimination learning by monkeys. J Exp Psychol 40: 26–39, 1950.[CrossRef][ISI]
Hikosaka O, Kawagoe R, and Takikawa Y. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev 80: 953–978, 2000.
Hikosaka O, Sakamoto M, and Usui S. Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. J Neurophysiol 61: 780–798, 1989a.
Hikosaka O, Sakamoto M, and Usui S. Functional properties of monkey caudate neurons. II. Visual and auditory responses. J Neurophysiol 61: 799–813, 1989b.
Hikosaka O, Sakamoto M, and Usui S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61: 814–832, 1989c.
Hikosaka O and Wurtz RH. Visual and oculomotor functions of monkey substantia nigra pars reticulata. IV. Relation of substantia nigra to superior colliculus. J Neurophysiol 49: 1285–1301, 1983.
Hollerman JR and Schultz W. Dopamine neurons report an error in the temporal prediction reward during learning. Nat Neurosci 1: 304–309, 1998.[CrossRef][ISI][Medline]
Hollerman JR, Tremblay L, and Schultz W. Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80: 947–963, 1998.
Houk JC and Wise SP. Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb Cortex 5: 95–110, 1995.
Ikeda T and Hikosaka O. Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39: 693–700, 2003.[CrossRef][ISI][Medline]
Itoh H, Nakahara H, Hikosaka O, Kawagoe R, Takikawa Y, and Aihara K. Correlation of primate caudate neural activity and saccade parameters in reward-oriented behavior. J Neurophysiol 89: 1774–1783, 2003.
Judge SJ, Richmond BJ, and Chu FC. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res 20: 535–538, 1980.[CrossRef][ISI][Medline]
Kawagoe R, Takikawa Y, and Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1: 411–416, 1998.[CrossRef][ISI][Medline]
Kawagoe R, Takikawa Y, and Hikosaka O. Reward-predicting activity of dopamine and caudate neurons: a possible mechanism of motivational control of saccadic eye movement. J Neurophysiol 91: 1013–1024, 2004.
Kimura M. The role of primate putamen neurons in the association of sensory stimuli with movement. Neurosci Res 3: 436–443, 1986.[CrossRef][Medline]
Kobayashi S, Lauwereyns J, Koizumi M, Sakagami M, and Hikosaka O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J Neurophysiol 87: 1488–1498, 2002.
Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, and Hikosaka O. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33: 463–473, 2002a.[CrossRef][ISI][Medline]
Lauwereyns J, Watanabe K, Coe B, and Hikosaka O. A neural correlate of response bias in monkey caudate nucleus. Nature 418: 413–417, 2002b.[CrossRef][Medline]
Meyer DR. Intraproblem–interproblem relationships in learning by monkeys. J Comp Physiol Psychol 44: 162–167, 1951.[CrossRef][ISI][Medline]
Miller EK, Nieder A, Freeman DJ, and Wallis J. Neural correlates of categories and concepts. Curr Opin Neurobiol 13: 198–203, 2003.[CrossRef][ISI][Medline]
Mitz AR, Godschalk M, and Wise SP. Learning-dependent neuronal activity in the premotor cortex: activity during the acquisition of conditional motor associations. J Neurosci 11: 1855–1872, 1991.[Abstract]
Montague PR, Dayan P, and Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936–1947, 1996.
Nakamura K, Sakai K, and Hikosaka O. Neuronal activity in medial frontal cortex during learning of sequential procedures. J Neurophysiol 80: 2671–2687, 1998.
Niki H, Sugita S, and Watanabe M. Modification of the activity of primate frontal neurons during learning of a go/no-go discrimination and its reversal. In: Vision, Memory and the Temporal Lobe, edited by Iwai E and Mishkin M. New York: Elsevier, 1990, p. 295–304.
Parent A. Extrinsic connections of the basal ganglia. Trends Neurosci 13: 254–258, 1990.[CrossRef][ISI][Medline]
Parthasarathy HB, Schall JD, and Graybiel AM. Distributed but convergent ordering of corticostriatal projections: analysis of the frontal eye field and the supplementary eye field in the macaque monkey. J Neurosci 12: 4468–4488, 1992.[Abstract]
Pasupathy A and Miller EK. Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433: 873–876, 2005.[CrossRef][Medline]
Redgrave P, Prescott TJ, and Gurney K. The basal ganglia: a vertebrate to the selection problem? Neuroscience 89: 1009–1023, 1999.[CrossRef][ISI][Medline]
Rolls ET. Neurophysiology and cognitive functions of the striatum. Rev Neurol (Paris) 150: 648–660, 1994.[Medline]
Rolls ET, Critchley HD, Mason R, and Wakeman EA. Orbitofrontal cortex neurons: role in olfactory and visual association learning. J Neurophysiol 75: 1970–1981, 1996.
Sakagami M and Niki H. Encoding of behavioral significance of visual stimuli by primate prefrontal neurons: relation to relevant task conditions. Exp Brain Res 97: 423–436, 1994.[ISI][Medline]
Sakagami M, Tsutsui K, Lauwereyns J, Koizumi M, Kobayashi S, and Hikosaka O. A code for behavioral inhibition on the basis of color, but not motion, in ventrolateral prefrontal cortex of macaque monkey. J Neurosci 21: 4801–4808, 2001.
Schultz W. Getting formal with dopamine and reward. Neuron 36: 241–263, 2002.[CrossRef][ISI][Medline]
Schultz W, Apicella P, Scarnati E, and Ljungberg T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12: 4595–4610, 1992.[Abstract]
Schultz W, Dayan P, and Montague PR. A neural substrate of prediction and reward. Science 275: 1593–1599, 1997.
Schultz W, Tremblay L, and Hollerman JR. Changes in behavior-related neuronal activity in the striatum during learning. Trends Neurosci 26: 321–328, 2003.[CrossRef][ISI][Medline]
Selemon LD and Goldman-Rakic PS. Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey. J Neurosci 5: 776–794, 1985.[Abstract]
Shima K and Tanji J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282: 1335–1338, 1998.
Takikawa Y, Kawagoe R, and Hikosaka O. Reward-dependent spatial selectivity of anticipatory activity in monkey caudate neurons. J Neurophysiol 87: 508–515, 2002.
Takikawa Y, Kawagoe R, and Hikosaka O. A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping. J Neurophysiol 92: 2520–2529, 2004.
Taylor AE and Satint-Cyr JA. The neuropsychology of Parkinson's disease. Brain Cogn 28: 281–296, 1995.[CrossRef][ISI][Medline]
Thorpe SJ, Rolls ET, and Maddison S. Neuronal activity in the orbitofrontal cortex of the behaving monkey. Exp Brain Res 49: 93–115, 1983.[ISI][Medline]
Treichler FR and Petros TV. Interference characteristics in concurrent discrimination performance by monkeys. Bull Psychol Soc 21: 206–208, 1983.
Tremblay L, Hollerman JR, and Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80: 964–977, 1998.
Tremblay L and Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J Neurophysiol 83: 1877–1885, 2000.
Wallis JD, Anderson KC, and Miller EK. Single neurons in the prefrontal cortex encode abstract rules. Nature 411: 952–956, 2001.
Waelti P, Dickinson A, and Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412: 43–48, 2001.[CrossRef][Medline]
Wallis JD and Miller EK. From rules to responses: neuronal processes in the premotor and prefrontal cortex. J Neurophysiol 90: 1790–1808, 2003.
Watanabe K, Lauwereyns J, and Hikosaka O. Effects of motivational conflicts on visually elicited saccades in monkeys. Exp Brain Res 152: 361–367, 2003a.[CrossRef][ISI][Medline]
Watanabe K, Lauwereyns J, and Hikosaka O. Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. J Neurosci 23: 10051–10057, 2003b.
Watanabe M. Prefrontal unit activity during associative learning in the monkey. Exp Brain Res 80: 296–309, 1990.[ISI][Medline]
Watanabe M. Reward expectancy in primate prefrontal neurons. Nature 382: 629–632, 1996.[CrossRef][Medline]
Watanabe M, Hikosaka K, Sakagami M, and Shirakawa S. Coding and monitoring of motivational context in the primate prefrontal cortex. J Neurosci 22: 2391–2400, 2002.
White IM and Wise SP. Rule-dependent neuronal activity in the prefrontal cortex. Exp Brain Res 126: 315–335, 1999.[CrossRef][ISI][Medline]
Yeterian EH and Pandya DN. Prefrontostriatal connections in relation to cortical architectonic organization in rhesus monkeys. J Comp Neurol 312: 43–67, 1991.[CrossRef][ISI][Medline]
This article has been cited by other articles:
|
|
 |
|
  K. M. Cham, A. J. Anderson, and L. A. Abel Task-Induced Stress and Motivation Decrease Foveation-Period Durations in Infantile Nystagmus Syndrome Invest. Ophthalmol. Vis. Sci., July 1, 2008; 49(7): 2977 - 2984. [Abstract] [Full Text] [PDF]
|
