| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Program in Neuroscience, Department of Psychological and Brain Sciences, Indiana University, Bloomington, Indiana
Submitted 7 April 2006; accepted in final form 18 December 2006
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
; DeLong 1990). In this view, these nuclei had no extra-basal ganglia inputs of their own, and so they were considered relays for striatal firing patterns. However, there is considerable evidence that the STN receives direct cortical (Hartmann-von Monakow et al. 1978; Nambu et al. 1996), thalamic (Sugimoto et al. 1983), and midbrain (Hassani et al. 1997) input. Because the STN and STR receive direct input from the same cortical and midbrain regions, the STN is in a position to act as a second, independent input area of the basal ganglia. Both of these nuclei project directly to the output areas of the basal ganglia (the substantia nigra pars reticulata and internal globus pallidus), and both are reciprocally connected with the external globus pallidus. Thus an alternative interpretation of basal ganglia connectivity is that the STN and STR provide parallel cortico-nigrothalamic pathways (Kita et al. 2004; Levy et al. 1997; Nambu et al. 2002). If such parallelism exists, we may well ask whether it extends to function as well.
An extensive literature highlights parallel STR (Anderson et al. 1979; Chang et al. 2006; Crutcher and DeLong 1984b; Haracz et al. 1993; Rolls et al. 1983) and STN (Bergman et al. 1994; Carpenter et al. 1950; Cheruel et al. 1996; Georgopoulos 1983; Matsumura et al. 1992; Shi et al. 2004; Wichmann et al. 1994) involvement in normal and pathological movement. However, although striatal involvement in associative and limbic processes has been thoroughly documented (Apicella et al. 1991, 1992, 1997; Jog et al. 1999; Packard and Knowlton 2002; Schultz and Romo 1992; Tremblay et al. 1998), it is unclear if the STN plays a parallel role. Lesions of the STN in rats performing a multi-choice attentional task induced deficits that suggest decreased impulse control (Baunez and Robbins 1997; but see Winstanley et al. 2005), increased behavioral measures of motivation (Baunez et al. 2002), and differentially impaired craving for rewards (Baunez et al. 2005). Single-unit recordings in primate (Darbaky et al. 2005; Matsumura et al. 1992) and cat STN (Cheruel et al. 1996) showed that these neurons increased firing rate during reinforcement.
Lesions (Obeso et al. 1997) or high-frequency stimulation of the STN (Limousin et al. 1995) are effective treatments for the dyskinesia characteristic of Parkinson's disease. If the STN is indeed involved in nonmotor aspects of behavior that relate to impulse control, surgical manipulation of this nucleus would be expected to induce corresponding behavioral deficits; such deficits have already been reported. Parkinson's patients undergoing high-frequency stimulation of the STN exhibit overeating (Moro et al. 1999), uncontrollable laughter (Krack et al. 2001), and hypersexuality (Absher et al. 2000; Romito et al. 2002).
Although behavioral and physiological studies collectively suggest that STN neurons are involved in operant behavior, there are no studies in the rat that examine concurrent behavior and single-unit electrophysiology during operant learning. We therefore recorded STN and STR neuronal firing patterns in rats learning an operant task comprising motor, limbic, and associative components to obtain information about STN firing patterns during operant behavior, examine any involvement of STN neurons in nonmotor aspects of behavior, and compare STN activity with concurrent recordings of STR neuronal activity with an eye toward assessing functional parallelism within the basal ganglia.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
Male Sprague-Dawley rats (250–400 g, Harlan Industries, Indianapolis, IN) were used in all experiments. For the duration of the study, rats were food-restricted to 85% of free-feeding weight but had ad libitum access to water except for two instances of overnight water deprivation. Rats were housed in the Department of Psychology animal colony on a 12:12 h light:dark cycle. All experimental protocols were approved by the local Institutional Animal Care and Use Committee (IACUC) and followed guidelines established by the National Institutes of Health (Institute of Laboratory Animal Resources 1996).
Stereotaxic surgery
Rats were under general anesthesia during all surgical procedures. Atropine sulfate (0.05 mg/kg sc) was given preoperatively to facilitate breathing. Anesthesia was induced with ketamine/xylazine (90/10 mg/kg im) and maintained with 0.2 ml ketamine ip as needed. We blocked all incision areas and pressure points with lidocaine and applied a moisturizing lubricant (Moisture Eyes PM, Bausch and Lomb) to prevent corneal drying. We used blunt ear bars coated with antibiotic jelly to reduce the risk of infection due to accidental rupture of the eardrum. All implanted materials were sterilized in 1:4 diluted 2% glutaraldehyde/dH2O (Cidex, Advanced Sterilization Products, Miami, FL). During recovery, rats were given 10 ml lactated Ringer solution subcutaneously to counteract dehydration and closely monitored until they awoke.
After incision and reflection of the scalp and periosteum, we drilled two holes in the skull over the STN and ipsilateral STR and removed the exposed dura. Multiwire bundle electrodes were lowered into the target structures at a rate of 50–100 µm/30 s. STR coordinates (AP and ML) were +1.0 and +2.5 mm relative to bregma and –5.5 mm ventral to skull surface (Paxinos and Watson 1998). Due to the small volume and deep location of the STN, we estimated its location by averaging the results from two sets of calculations, one measured from bregma (–3.8 mm AP, +2.5 mm ML) and skull surface (–8.35 mm DV), the other from the interaural line (+5.2 mm AP, +2.5 mm ML, +1.65 mm DV). Four to six additional holes were drilled for stainless steel anchor screws. The electrode assemblies were fixed in place with super glue and dental acrylic.
Electrophysiology
Each multiwire electrode bundle comprised seven 25-µm-diam, Formvar-insulated stainless steel wires (California Fine Wire) threaded through a 27-gauge stainless steel needle that served as the ground. The cannula was soldered to an 8-pin male strip connector (Omnetics PS2). The wire bundle was drawn out of a hole at the base of the needle, and each wire was soldered to a connector pin. Wires were trimmed so that they protruded 1–2 mm past the end of the needle. On recording days, an 8-pin plug interfaced with the head-affixed PS2 assemblies. Electrode impedances were typically on the order of 1 M
although this was not quantified systematically.
Rats were placed in the operant chamber and connected to a multichannel electrical commutator (Plastics One) via a length of flexible shielded cable, allowing them complete freedom of movement. Electrophysiological signals were transmitted via the commutator to a preamplifier and then to data-acquisition hardware (MNAP, Plexon, Dallas TX) that allowed PC-based user control of unit discrimination and recording. Signals were also fed to an oscilloscope and audio monitor to facilitate unit discrimination.
Using the Plexon system software (SortClient), we were able to discriminate and simultaneously record up to four signals per channel for a maximum of 56 possible units per recording session. Unit discrimination was accomplished using a combination of template-matching, principal components analysis, and k-means clustering algorithms. All units had at least a 3:1 signal:noise ratio. Prior to recording, the power spectra and either an autocorrelogram or an interspike interval histogram were examined for each putative unit to maximize the probability that units consisted of only one signal [reflected in a trough in the interspike interval (ISI) histogram around the absolute refractory period], and were free of 60-cycle interference. The typical yield was five to seven units per recording session. Although we report here a total sample size of several hundred units, it is likely that many signals were recorded more than once over the course of several days. However, as we had no criteria for determining a common signal source over repeated recordings, we were obliged to treat them as independent entities. Our reported values are presented as the average number of recorded or responsive neurons per training/recording session.
Operant training
Recording sessions took place in a custom-built Plexiglas operant chamber. Two nose-poke detectors recesses were located along one wall, 
5 cm above the floor. Each recess contained a green light-emitting diode (LED) and an infrared beam-break detector. A recess on the opposite wall, 5 cm above the floor, contained a sucrose delivery spout, infrared beam-break detector, and yellow LED. Tone generators (1.9 and 4.2 kHz) provided auditory stimuli. Figure 1 depicts the physical layout of the operant chamber as well as schematic diagrams of the training protocols.
|
After three days of LICK training, rats underwent nine daily sessions in which they learned to perform a discriminatory nose-poke to obtain reward (POKE sessions). Rats were water-deprived the night before the first poke session. A trial began with concurrent auditory and visual cues (auditory: 4.5 kHz, 70 dB, 2 x 500-ms pulses, 100-ms interpulse interval, nose-poke tone; visual: pseudo-random illumination of one of the nose-poke recesses by a green LED). A nose-poke into the lit recess elicited the previously learned feed tone/yellow LED cues signaling the reward-ready period, which in this protocol lasted only 5 s. A trial ended when either the reward-ready period elapsed without any spout licks or 1 s after valve opening. The ITI varied pseudorandomly from 5–8 s. The tone/light combinations were not counterbalanced nor was the association between the green LEDs and "correct" nose-pokes.
To determine the effects of reward prediction, the final two operant sessions used a modified POKE protocol in which reward occurred in 75% of the trials (POKE75 session). In the remaining 25% of trials, all cues were presented normally but spout licking during the reward-ready period did not open the valve. Nonrewarded trials occurred on a pseudorandom basis.
All rats learned the task within the 2-wk training period. One rat underwent LICK training alone; those data were pooled with the other LICK data. A second group of rats underwent the 2-wk training regimen before being implanted with bundle electrodes; they were then recorded for a further 2 wk performing the POKE and POKE75 protocols. We shall refer to this group as the "pretrained" group.
Histology
On completion of all recordings, rats were killed with a mixture of chloral hydrate and sodium pentobarbital (chloropent). A 30-µA, 5-s current pulse was passed through each electrode where a unit was recorded. Rats were then transcardially perfused via the ascending aorta with 10% formosaline/19% wt/vol potassium ferrocyanide [K4Fe(CN)6], producing small blue deposits at the site of the recording electrode ("Prussian blue" reaction). Brains were removed, fixed in 10% formosaline, cryoprotected in 30% phosphate-buffered sucrose, sliced into 80-µm coronal sections, and stained with Cresyl violet. We included for analysis all units from animals with large visible lesions in the STN or along the border with the cerebral peduncle (Fig. 2A).
|
All of our statistical analyses were based on binned perievent firing rates obtained during each recording/training session. Our first experimental aim was to describe the qualitative features of STN and STR perievent responses to operant events. Towards this end, we determined the latency, duration, and magnitude of significant perievent responses. We obtained measures that described the response profile for each neuronal sample, i.e., the probabilities of significant excitations and inhibitions as well as whether either of these response types was predominant at a given time relative to the event. Our second experimental question was whether there were differences in the proportion of neurons within each sample that responded to operant events, either within or across nuclei, training session, or reinforcement condition. The raw data in this case were the numbers of responsive neurons within a sample, responsiveness having been determined during the qualitative phase of analysis. We looked at the number of responses to each event out of the entire neuronal sample for that nucleus or a subsample comprising only neurons that showed significant perievent responses to the chosen event. We compared these response rates using Fisher's exact test with a conservative alpha value to compensate for the large number of comparisons.
Our qualitative analysis began by expressing each binned perievent record as z scores based on the mean and SD of the firing rate during the period from –2 to –1 s preceding the event. Bins with z scores >1.64 SDs (95% confidence interval) away from the baseline mean firing rate were considered significant. A "response" to an event was defined as three or more (
150 ms) consecutive, significant bins. In certain cases, indicated in the text, we analyzed prolonged neuronal responses. These we defined as 10 consecutive significant 100-ms bins, yielding a minimum response duration of 1 s. We included for analysis only those responses that fell within certain time windows. For phasic responses, that window was ±1 s, and for long responses, the window extended from –1 to +5 s as we wished to examine some responses that corresponded to the end of the reinforcement period.
We determined, for each response, the onset latency (defined as the time of occurrence of the first of the consecutive bins), duration (defined as the total number of consecutive bins with significant z scores), and magnitude (defined as the mean z score across all the consecutive bins constituting an individual response). Plotting the response magnitude versus the onset latency gave us the response distribution.
Because neuronal responses might correspond to either the beginning or the end of a particular behavior, we obtained the aggregate response probability for both excitations and inhibitions. Our thinking was that such an analysis would illustrate any overlap between the endings and beginnings of perievent responses, thereby highlighting time points within a given behavioral sequence that were particularly emphasized by changes in firing rate and providing us with a slightly different perspective on the temporal structure of perievent responses than that afforded by traditional perievent analysis. We obtained these aggregate response probabilities by first grouping together all of the neurons that showed significant responses to a particular behavioral event. Each neuron's perievent response was described by a string of values, one for each bin. A value of +1 meant that that bin occurred during a significant excitation; a value of –1 meant a significant inhibition was ongoing, and a value of 0 meant that firing during that bin was not different from baseline. We then summed the +1 responses across the entire sample and divided by the sample size to obtain the aggregate, binwise probability of excitation. We performed a similar summation on the –1 values to obtain the probability of inhibition. To obtain a measure of the strength and polarity of neuronal responses versus time, we also took the total sum of positive and negative responses and divided by the sample size. The sign of this aggregate probability corresponded to whether the predominant response in a given bin was excitatory or inhibitory, and its magnitude gave a measure of the degree of predominance of that response. See Fig. 3 for an example of this technique.
|
To determine the significance of these differences, we took the mean dPEH across all neurons included for analysis. We also generated a control event by taking the dPEH, for each neuron, of pairs of randomly picked time points occurring during the ITI; we repeated this 100 times for each neuron and then obtained the mean dPEH for this "null" event across all the neurons previously selected for analysis. We used this null dPEH to generate a binwise mean and SD; our confidence interval was 3 SDs from the bin mean, corresponding to a P value of 0.0027 (confidence interval = ±99.865). We then applied this confidence interval to the testable event dPEHs. Responses were considered significant if any of the bins exceeded the confidence interval.
After determining the qualitative parameters of each neuron's perievent responses as described in the preceding text, we obtained three sets of sums for each event during each training session type: the numbers of responsive and nonresponsive STN and STR neurons, the sums of neurons that did and did not show responses preceding event onset (t 
= 0), and the numbers of neurons that did and did not respond following event onset (t > 0). By specifying and summing over different combinations of training sessions, we were able to examine specific conditions and factors that might have influenced response rates. For example, by comparing the response data from POKE sessions to that from LICK sessions, we could examine whether the behavioral context affected response rates. Other conditions included pretrained versus non-pretrained, all POKE and POKE75 sessions versus LICK sessions and POKE versus POKE75 sessions. Such comparisons allowed us to determine not only whether STR or STN neurons were more likely to respond to a given stimulus but also whether the responses observed in each nucleus were context or training dependent.
To calculate the significance of differences in within-session/across-nucleus response proportions (disregarding whether the responses preceded or followed the specified behavioral event), we used Fisher's exact test with a conservative value of alpha = 0.005 to compensate for the large number of comparisons. For each session or group of sessions, the numbers of responsive and nonresponsive neurons from one nucleus constituted the expected values, whereas the observed values were obtained from the other nucleus. For example, to compare the proportions of STN and STR responses to reinforcement during poke sessions, we would set the expected values to the numbers of responsive and nonresponsive STN neurons across all poke sessions; the observed values would be taken from the STR group. It made no difference which nucleus was designated as observed or expected. Within-nucleus/across-session significances were calculated in a similar fashion with the expected values taken for one session or group of sessions, and the observed values taken from a second session or group of sessions, e.g., the expected values were taken from the numbers of recorded and responsive STN neurons during all poke sessions, and the observed values were taken from the numbers of recorded and responsive STN neurons during all lick sessions. Although we performed our statistics on the raw totals of recorded neurons, our tables present these values as the average numbers of neurons per training session; the statistical results were unaffected.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
1 ms wide and had mean firing rates of 2.40 ± 2.7 spike/s (range = 0.0005–24.34 spike/s). STR waveforms were also bi- or triphasic with a long afterhyperpolarization; a typical STR waveform was 1.5 ms wide. Mean STR firing rate was 2.69 ± 2.9 spike/s (range = 0.0043–19.15 spike/s). Typical waveforms are illustrated in Fig. 2B. Our reported STN firing rates are lower than those previously reported for awake, unrestrained rats (Olds et al. 1999), a discrepancy that may reflect methodological differences. A recent study using an electrode technique similar to ours also reported relatively slow firing rates (5–10 spike/s) (Shi et al. 2004). To address the possibility that we recorded from distinct subpopulations with different firing rats, we split the neuronal population into two groups based on each neuron's mean firing rate (, 1 spike/s) and re-ran all our analyses. We found no significant differences in the percent of neurons responding to each event or the response parameters (latency, duration, magnitude), although slower firing neurons did show a floor effect with regard to event-related inhibitions. We present here the pooled neuronal populations. Operant behavior
We used two behavioral metrics to monitor learning: discriminative accuracy, measured as the ratio of the number of incorrect nose-pokes to the total number of nose-pokes, and latency to lick the spout following the feed tone. These results are shown in Fig. 4. Dots indicate the mean values for each training session; vertical lines indicate the standard error. Solid lines show the best fit to the raw data. The data from non-pretrained rats were fit with an exponential function, whereas the data from pretrained rats were fit with a line.
|
Latency to lick the spout (Fig. 4B) was initially high in non-pretrained rats but decreased exponentially until it reached an asymptotic value of 2.92 ± 0.22 s (mean of the last 10 training sessions); exponential curve-fitting gave an asymptotic value of 2.78 s. Pretrained rats had an empirically determined asymptotic value of 2.48 ± 0.13 s, whereas linear curve fitting yielded a y intercept of 3.04 s. As with discriminative accuracy, pretrained rats showed decreasing latencies during the 2-wk recording period. Switching from POKE to POKE75 training sessions had no effect on LICK latency in either set of rats. The asymptotic latencies of pretrained and non-pretrained rats were not significantly different [ANOVA, F(1,18) = 3.02, P = 0.1]. Both of these measures demonstrated that rats successfully learned the discriminative task within the 2-wk recording/training period.
STN and STR firing decreases during reinforcement
STN and STR firing patterns during reinforcement were complex, often multiphasic; they comprised both excitations and inhibitions of various durations (Fig. 5A). Responses were characterized relative to the time of the first spout lick during the "reward-ready" period; a "response" was any significant deviation from baseline firing rate beginning within ±1 s of reinforcement. For a qualitative description of reinforcement-related firing patterns, we plotted the onset latency versus the mean amplitude of every response (Fig. 5B, top) for both the STN (left) and STR (right). Additionally, we plotted the probability of an ongoing response versus time, as determined by summing across all neurons and dividing by the sample size (Fig. 5B, bottom). This plot highlighted the times at which different responses intersected, e.g., if, at a particular time, one neuronal response terminated as another one in a different neuron began, or two responses with different onset latencies terminated at the same time. Peaks indicate an overlap between two responses that might not have been obvious from the scatter plot. In Fig. 5B and in subsequent plots of this nature, the black line represents the probability of excitation versus time, the light gray line represents the probability of inhibition versus time, and the dark gray line reflects the aggregate probability i.e., the predominant response versus time for the entire neuronal sample, taken as the sum of the black and light gray lines. This graph shows that the prototype neuronal response of both STN and STR neurons included a period of excitation preceding reinforcement (t <0 s, when the rat was approaching the spout) and a pronounced, persistent decrease in firing rate during and following reinforcement (t > 0 s).
|
1 s. These responses were primarily inhibitory although some excitations were observed. To examine these long responses without contamination from the short responses that typically preceded reinforcement, we re-analyzed the data but increased the minimum response length criterion from 150 ms (3 x 50-ms bins) to 1 s (10 x 100=ms bins). Responses had to occur between t = –1 and +5 s to be counted. We extended the time window to examine events that corresponded to the end of the reinforcement period at t = +1 s. (The analysis of long responses was only performed on reinforcement-related events, as long responses to nose-pokes and tones were rarely observed.) Although it is true that the 5 s postevent window overlaps with the intertrial interval, it was necessary to analyze responses that corresponded to the end of the reinforcement period. The behaviors exhibited during the intertrial interval were minimal; rats typically persisted in (non-reinforced) spout licking, or sat quietly until the start of the next trial. The aggregate probability profiles for short (>150 ms; Fig. 5B, left) and long (>1s; Fig. 5B, right) responses were similar in shape; both indicated a pronounced decrease in firing rate during and following reinforcement, although the profile for long responses also shows a high probability of excitation following reinforcement. In both nuclei, there was a transition point at which a steadily increasing probability of excitation dropped sharply and an initially low probability of inhibition increased. This transition point occurred around t = 0 in the STN and t = –500 ms in the STR; these differences may provide a clue as to the meaning of the increase in firing rate preceding reinforcement.
Although Fig. 5B clearly indicates that reinforcement was typically preceded by excitation and accompanied by inhibition, there were some neurons the firing rates of which instead switched from low to high (Fig. 5A, STN, top right). These results show that STN and STR firing patterns during reinforcement were not uniform and raise the possibility that changes in firing rate during reinforcement may reflect transitions between behavioral states or "chunks" (Graybiel 1995, 1998) of behavioral sequences.
We noted in particular one STR neuron (Fig. 5A. bottom right) in which examination of the raster plot revealed a response that developed over the course of this training session. Early in the recording session (top rows of the raster), the short excitation that follows the end of the 1-s reinforcement period was brief, limited to one or two spikes, and its occurrence was intermittent. Three quarters of the way through the session, the response became more pronounced, and indeed began to widen. Such responses were rarely observed, but provided an exciting in vivo example of real-time neuronal plasticity that accompanied learning.
Having qualitatively described STN and STR responses to reinforcement, we wanted to know whether there were significant differences in the proportions of responsive neurons, either across nuclei or within nuclei but across conditions. i.e., pretrained versus non-pretrained, LICK versus POKE, etc. A neuron was considered responsive if it demonstrated any significant deviation from baseline firing rate as determined during the qualitative analysis (see preceding text, METHODS). The proportions of responsive neurons were compared using Fisher's exact test with the expected and observed values obtained from the two neuronal samples (in the case of across-nuclei comparison) or two groups of sessions (for within-nucleus comparisons). In some cases, which will be noted as appropriate, we confined the samples to only those neurons whose responses occurring before or after reinforcement i.e., valve-opening.
The numbers of reinforcement-responsive STN and STR neurons per session are shown in Table 1. Short responses (>150 ms) were more likely to occur in the STN than in the STR (P < 0.005) when the data were pooled across all training sessions. Within each nucleus, the proportion of responsive neurons was context-dependent, occurring more frequently during POKE sessions than during LICK sessions (STN: P < 0.0001, STR: P < 0.0001). There was no effect of pretraining on the proportion of responsive neurons. STN and STR responses were equally likely to precede or follow reinforcement regardless of whether we expressed responsive/nonresponsive totals based on the entire sample of recorded neurons (before: P < 0.01, after: P < 0.18) or only those neurons showing reinforcement responses (before: P < 0.72, after: P < 0.09). Within-nucleus, across-session comparisons revealed a context-dependence identical to what we observed when we considered all responses together: STN and STR neurons were more likely to respond to reinforcement during POKE sessions than LICK sessions. This was true for responses preceding and after reinforcement (P < 0.0001 in each case), making it unlikely that this context-dependence arose solely from the distinct motor behaviors immediately preceding reinforcement in the two sessions (approach vs. stationary spout-licking). Our analysis revealed a context-dependent bias toward responses preceding reinforcement in the STN (P < 0.005) but not the STR (P = 0.197). During POKE sessions, early STN responses were more likely than late responses (P < 0.002), whereas during LICK sessions. the proportions were equal. This difference may reflect the differing behavioral patterns immediately preceding spout licking.
|
STN, STR inhibition was attenuated when reinforcement was withheld
We used a 75% reinforcement schedule (see METHODS) to control for the motor aspects of spout licking and to assess the influence of prediction error on neuronal firing patterns. Comparisons were made between the reinforcement condition, where t = 0 corresponded to the first lick that opened the spout, and the unexpectedly non-reinforced condition in which the first lick at the spout did not elicit sucrose. Rats did not show any overt changes in behavior during this protocol, although they did lick the spout significantly less during the 1 s after the first non-reinforced lick (5.71 ± 2.54 licks) than during normal reinforcement (8.33 ± 1.58 licks, P < 0.001, t-test).
Figure 6A illustrates paired firing rate histograms centered on valve opening (reward) or on the analogous, non-reinforced first spout lick in the POKE75 trials (no reward). In both examples, the firing pattern observed during the 1-s reinforcement period (indicated by vertical lines) was markedly altered when a predicted reinforcement was withheld. In the STN neuron, the large decrease in firing rate was greatly attenuated when reinforcement was withheld. The STR neuron responded to reinforcement with a strong short excitation coupled with a sustained decrease in firing that persisted throughout the reinforcement period and the remained of the ITI. When reinforcement was withheld, the initial excitation persisted for 1–2 s and the long inhibition disappeared. Furthermore, the large excitation seen at the start of the subsequent trial (reward, t = +5) was absent after the withholding of reinforcement. Thus in both of our examples, firing rate was higher following an incorrect prediction of reinforcement than following normal reinforcement.
|
|
To confirm the significance of the lack of inhibition after withholding of reinforcement, we used a differential perievent histogram (see METHODS) to visualize the relative firing rates of neurons during the two reinforcement conditions. Confidence intervals generated using a control dPEH based on random points within the intertrial interval allowed us to determine the times at which firing rate during one event was significantly different from the firing rate during the paired event. The results are shown in Fig. 7. Difference scores for short responses to reinforcement were not significant. Long responses, however, were significantly different beginning near the zero point of the dPEH, where we observed a large negative region, indicating a higher firing rate after the withholding of reinforcement. This negative region was shorter in the STN than in the STR, and began later (STN: t = +500 ms, STR: t = 0). Our dPEH analysis confirmed that individual neurons’ firing rates were higher following the unexpected absence of reinforcement.
|
STN, STR responses to nose-pokes do not reflect discriminative accuracy
STR and STN neurons both responded to nose-pokes. Figure 8A shows typical examples of responses to correct and incorrect nose-pokes. In these instances, firing rate increased before or coincident with nose-pokes, regardless of their accuracy. Firing rate decreased following a correct nose-poke, such decrease being absent following the incorrect nose-poke.
|
15% responded to incorrect nose-pokes (P = 0.63; Table 3). There were no significant differences in the relative rates of early (t < 0) or late (t > 0 s) responses, either within or between nuclei. Nor did we observe context-dependent response rates or effects of pretraining. Similar proportions (14%) of STN and STR neurons responded to nose-pokes occurring during the ITI; these response rates were independent of context or pretraining. We ignore ITI poke responses for the remainder of our analysis.
|
Although the examples and response probability profiles in Fig. 8 suggest that a decrease in firing rate after a nose-poke accompanies a correct discrimination, analysis of dPEHs for correct and incorrect nose-pokes showed that although there was a trend in this direction, the differences in relative firing rates were not significant. While it is possible that a significant effect might be seen with a larger sample size, we conclude that STN and STR firing rates do not encode the "correctness" or "success" of a discriminative nose-poke.
STN, STR neurons differentially respond to tone stimuli
Figure 9 and Table 4 summarize the STN and STR responses to tone stimuli. Approximately 30% of the recorded neurons in each nucleus responded to the feed tone (P = 0.49), and 18% of the neurons in each nucleus responded to the nose-poke tone presentation (P = 0.06). Responses were characterized in relation to the onset of the first pulse of each of the tones and were only counted if they occurred within ±1 s of the tone onset.
|
|
Equal proportions of STN and STR neuron responses preceded (P = 0.18) or followed (P = 0.18) nose-poke tone presentation. We did not observe any effects of pretraining, context-dependence, or reinforcement condition on the rates of nose-poke tone response in either nucleus. In both nuclei, response probability profiles revealed peaks in the probability of excitation (with a corresponding positive peak in the aggregate probability) 200 ms after tone onset. The probability of inhibition remained flat except for a brief increase 500 ms after tone onset; the aggregate probability showed no negative peaks, confirming the neurons were typically excited during nose-poke presentation.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
We report here, for the first time in rats, that the STN and STR are concurrently involved in the processing of events related to operant performance. We found that neurons in both structures responded to task events including reinforcement and instructive cues and showed differential responses to the presence and absence of reinforcement. Although reinforcement-related firing patterns have been reported in primates (Darbaky et al. 2005; Matsumura et al. 1992), this is the first study in behaving rats to examine STN neuronal responses to reinforcement in detail.
Reinforcement-related activity
Our results show that STN firing patterns can represent operant reinforcement. In fact, STN and STR neurons respond concurrently, but with subtle differences, to reinforcement. The predominant motif of reinforcement-related neuronal firing, both in the STN and the STR, was a phasic excitation preceding reinforcement coupled with a pronounced inhibition during consumption of sucrose reward (Fig. 5A). Response latency histograms revealed differences in the latencies of phasic STN and STR excitations preceding valve opening, suggesting that these neurons’ firing rates related to ongoing behavioral sequences in different ways.
In the STN, the aggregate response probability increased steadily preceding valve opening but dropped off sharply at the time of valve opening. Increased firing could have reflected either an increasing expectation of reward or motor activity as the rats approached and positioned themselves at the spout. The proportion of STN neurons showing responses preceding valve opening was higher during POKE sessions, in which rats had to cross the chamber to lick the spout, than during LICK sessions, in which rats remained in front of the spout. Because the change in the aggregate probability decreased so abruptly at t = 0 s (valve opening) and because the proportion of STN excitations preceding valve opening depended on the training protocol, which differed in the nature of behavior expressed immediately before valve opening, it is likely that excitatory STN responses preceding valve opening were related to approach/positioning behavior. In the STR, the analogous sudden decrease in the aggregate response probability occurred 500 ms before valve opening. Furthermore, the percentage of responsive STR neurons was independent of training protocol, and thus of behavior immediately preceding spout licking. STR responses preceding valve opening may therefore have related to cue presentation; perievent histograms based on consecutive events confirmed that some STR neurons’ firing rates increased after the feed tone but before valve opening.
During sucrose reinforcement, neurons were predominantly inhibited. This inhibition was greatly attenuated, or even reversed (Fig. 6A, right) when spout licking was not reinforced, whether it occurred during the ITI or during non-reinforced trials in the POKE75 sessions. Several reports have shown a "pause" response in tonically active STR neurons during reinforcement (Aosaki et al. 1995; Apicella et al. 1991, 1992, 1997). In primates and cats, STN neurons were typically excited during reinforcement (Cheruel et al. 1996; Darbaky et al. 2005; Matsumura et al. 1992). Although we did observe reinforcement-related excitations in the STN, the majority of our responses were inhibitory, as illustrated by the aggregate response probability graph in Fig. 5B.
Cue-related activity
Both STN and STR neurons increased their firing rates during the period surrounding presentation of auditory cues. Several reports have documented STN and STR responses to auditory stimuli as well as innervation of these areas by auditory cortex afferents (Cromwell and Schultz 2003; Gardiner and Kitai 1992; Jog et al. 1999; Kolomiets et al. 2001; Cheruel et al. 1996; Shi et al. 2005). STN and STR response profiles differed both within and across nuclei. Although we report differences in the rates of response to the feed tone stimulus as a function of training protocol, these differences were confined to the pretone period and may arise from the nature of the behavioral tasks: during lick sessions, rats remained stationed in front of the reinforcement spout, whereas during poke sessions, they had to approach it from the other side of the recording chamber. Thus it is likely that prefeed-tone responses were related to approach behavior.
Neurons in both the STN and STR responded to the nose-poke tone with phasic excitations occurring within 200 ms after the tone presentation, reflected in the positive aggregate response probability in Fig. 9B. STR neurons showed a similar response profile after the feed tone. STN neurons, however, showed equal proportions of inhibitions and excitations after feed tone presentation; there was no peak in the aggregate probability. STR firing patterns may reflect either purely auditory responses or the instructive value of the tone. Conversely, although STN neurons clearly responded to the nose-poke tone, responses occurred throughout the feed tone perievent window, and the only evident peak in the aggregate probability occurred early in the perievent period. This peak was likely due to a residual response to the preceding correct nose-poke. The response profiles suggest that STN neurons were not merely responding to auditory stimuli but rather to the differential salience of each cue within the operant paradigm.
Nose-pokes
Both STN and STR neurons responded to correct and incorrect nose-pokes, although responses to incorrect nose-pokes were much less common. In both nuclei, the aggregate response probability indicated that firing rate increased before all nose-pokes, regardless of whether they were correct or not. This consistency is not surprising, and the most parsimonious explanation would be that increases in firing rate relate to the motor aspects of approach and poke behavior. However, our recordings from the STR were located in the medial portion of the caudate, a region not noted for its innervation by motor cortical afferents (Parent and Hazrati 1995). We think that approach behavior was the most likely explanation for the increase in STN firing because we saw a similar pattern of responses preceding reinforcement (valve opening). The STR response may instead reflect that the nose-poke is the first motor action in a behavioral sequence, which could account for the lack of a difference between correct and incorrect nose-pokes. We observed several STR (and STN) neurons that exhibited sustained periods of increased or decreased firing whose onsets corresponded with nose-poke performance; one such neuron is shown in Fig. 5A, where the pronounced "hump" occurring late in the perievent histogram corresponds with the nose-poke at the start of the subsequent trial.
The response probability graphs further revealed that inhibitions often followed the performance of a correct nose-poke, inhibitions that were attenuated or even reversed after an incorrect nose-poke. This was an exciting finding because it suggested that STN and STR neurons could encode discriminative accuracy or successful fulfillment of part of a sequence. However, the dPEH scores for these events were not significant, and so we must conclude that these neurons were incapable of differentially coding discriminative accuracy.
Functional implications
One interesting hypothesis concerning the role of the basal ganglia in behavior is that they mediate the acquisition of habits or sequences of behavior by coding for the start and stop times of the "chunks" constituting those sequences (Graybiel 1995, 1998, Jog et al. 1999). When we examined the responses to different task events, we frequently observed neurons in both the STN and STR that would exhibit sustained changes in firing rate coincident with a particular behavioral event. During reinforcement, we often observed transitions from one firing regime to another, e.g., a slow-firing neuron transitioned to fast firing during reinforcement, and this fast firing persisted until the start of the next trial. Other neurons showed just the opposite transition (cf. Figs. 5A, STN: top right and bottom left, and Fig. 6A, STR: top and bottom left). Such transitions suggest that STN and STR firing patterns could reflect more than simply a motor or reinforcement roles, including the start of a new trial or behavioral sequence, the discriminative action phase of the trial, and the reinforcement phase of the trial as well as the completion of a behavioral sequence and entry into a behaviorally neutral period. Such "behavioral chunking" could explain the similarity in STR firing patterns preceding correct and incorrect nose-poke because we are not recording from motor areas of striatum, the similarity could reflect that a nose-poke was the first event in a behavioral sequence, regardless of its accuracy. Similarly, it could explain why STN responses were locked to the nose-poke tone but not the feed tone. Although there is already evidence in rats for STR neurons encoding of the beginning and end of operant behavioral sequences (Jog et al. 1999), we are unaware of any reports documenting such coding by STN neurons.
Although STN firing patterns during operant performance have not been widely documented (but see Darbaky et al. 2005), lesion and pharmacological studies have suggested that the STN is involved in motivated behavior. STN-lesioned rats exhibited perseverative and premature responses in an attentional task and demonstrated higher scores on behavioral measures of motivation including latency to consume food and locomotion in anticipation of reward (Baunez and Robbins 1997; Baunez et al. 2002, 2005). Lesions of the prefrontal cortex-STN pathway induced similar behavioral deficits (Chudasama et al. 2003). Clinical reports have described inappropriate, uncontrollable laughter (Krack et al. 2001), increased appetite and overeating (Moro et al. 1999), and increased sex drive (Absher et al. 2000; Romito et al. 2002) in Parkinson's patients having undergone STN deep-brain stimulation. These findings suggest that disruption of STN function may interfere with the suppression of impulsive behaviors. More recent behavioral work, however, suggests that rather than gating impulsive behaviors, the STN may instead mediate the association of conditioned and unconditioned stimuli (CS/US) (Winstanley et al. 2005). Not only did our STN neurons respond to sucrose reinforcement (a US), but they also respond to tone cues that might serve as conditioned stimuli and thus are in an excellent position to mediate the Pavlovian association of stimuli. Indeed STN and STR neurons both responded to these events with similar, but subtly differing, firing patterns. It is intriguing to consider the interplay, at the output nuclei, of these similar firing patterns given the different learning deficits induced by STN and STR lesions (Baunez and Robbins 1997; Baunez et al. 2005; Packard and Knowlton 2002
