Visually guided behavior often involves decisions that are based on evaluating stimuli in the context of those observed previously. Such decisions are made by monkeys comparing two consecutive stimuli, sample and test, moving in the same or opposite directions. We examined whether responses in the motion processing area MT during the comparison phase of this task (test) are modulated by the direction of the preceding stimulus (sample). This modulation, termed comparison signal, was measured by comparing responses to identical test stimuli on trials when it was preceded by sample moving in the same direction (S-trials) with trials when it was preceded by sample moving in a different direction (D-trials). The test always appeared in the neuron's receptive field (RF), whereas sample could appear in the RF or in the contralateral visual field (remote sample). With sample in-RF, we found three types of modulation carried by different sets of neurons: early suppression on S-trials and late enhancement, one on S-trials, and the other on D-trials. Under these conditions, many neurons with and without comparison effects exhibited significant, choice-related activity. Response modulation was also present following the remote sample, even though the information about its direction could only reach MT indirectly via top-down influences. However, unlike on trials with in-RF sample, these signals were dominated by response suppression, shedding light on the contribution of top-down influences to the comparison effects. These results demonstrate that during the task requiring monkeys to compare two directions of motion, MT responses during the comparison phase of this task reflect similarities and differences between the two stimuli, suggesting participation in sensory comparisons. The nature of these signals provides insights into the operation of bottom-up and top-down influences involved in this process.
- working memory
- random-dots stimuli
- top-down signals
- receptive fields
- motion perception
visually guided behavior involves a complex interaction of sensory information with working memory and cognitive control. The bottom-up flow of visual information involves successive stages of processing that are manifest through increasing receptive field (RF) sizes and the complexity of feature selectivity. At most, if not all, levels of this hierarchy, visual processing can be influenced by a top-down flow of information about the contextual relevance of the current visual stimuli. Such dynamic interactions between visual processing and cognitive states have been documented in midlevel visual areas, middle temporal (MT) and area V4, containing neurons with well-localized RFs and stimulus-selective responses that can be affected by selective allocation of spatial or feature-based attention [for review, see Reynolds and Chelazzi (2004)]. Such interactions have also been reported during tasks requiring a comparison between current and previously observed stimuli. For example, during a search task, responses of area V4 neurons to the current stimulus were enhanced when its properties matched the preceding cue (Motter 1994). Similarly, during a working memory task involving a comparison between two sequentially presented shapes—sample and test—responses in the inferotemporal (IT) cortex to the test were modulated by the preceding sample (Freedman et al. 2003; Miller and Desimone 1994; Miller et al. 1993a). Subsequent recordings from the prefrontal cortex (PFC) during the same task also revealed response modulation by the preceding sample (Freedman et al. 2003; Miller et al. 1996). The incidence and the time-course of these signals led to the conclusion that some of the comparison signals observed in IT reflect top-down influences from the PFC.
There is also evidence for comparison signals carried by neurons in area MT during a working memory for motion task (Zaksas and Pasternak 2006). In that task, the monkeys compared the directions of two moving stimuli—sample and test—separated by a 1.5-s delay and reported them as moving in the same or different directions by pressing one of the two response buttons. The authors reported that the average response to the test was modulated by sample direction, and this modulation appeared to be dominated by response suppression. While this observation implicated MT in sensory comparisons required by the task, it provided little information about the nature and the time-course of signals carried by individual neurons.
In the current study, we examined in detail the temporal dynamics and the sign of these effects, termed comparison effects, by comparing responses to the test on trials when its direction matched the preceding sample (S-trials) with trials when it was preceded by the opposite direction (D-trials). We examined the nature of this modulation under conditions when sample direction appeared in the neuron's RF, as well as when it was presented in the opposite visual quadrant. Since MT neurons have RFs localized to the contralateral hemifield, the information about the sample presented in the ipsilateral-noncorresponding portion of the visual field could only reach these neurons indirectly, via top-down inputs, revealing the nature of their influences. Thus the two sample conditions allowed us to assess the contribution of both bottom-up and top-down influences on sensory comparisons recorded in MT. With sample in-RF, we identified three types of modulation carried by different neurons: early suppression on S-trials and late enhancement, one on S-trials, and the other on D-trials. Responses during the test were also affected by the remote sample, exhibiting differential activity during S- and D-trials. These comparison signals, represented by two distinct cell groups, were dominated by later-occurring response suppression, an effect not observed with sample in-RF, most likely reflecting top-down influences. These results demonstrate that during the task requiring monkeys to compare two directions of motion, MT neurons carry signals reflecting similarities and differences between the two comparison stimuli, suggesting their participation in sensory comparisons. The nature of these signals provides insights into the operation of both bottom-up and top-down influences involved in this process.
MATERIALS AND METHODS
We recorded from area MT of two adult, 8- to 12-kg male macaque monkeys (Macaca nemestrina). The data used for the current study were collected during the same recording experiments as those reported previously (Zaksas and Pasternak 2005, 2006). The monkeys received their daily liquid rations in the form of fruit drink or water during the testing sessions. At the end of each testing day, they were given fresh fruit and vitamins. No testing occurred on weekends, and the monkeys received 100 ml/kg allotments of water. Food was continually available in the home cage, and body weights were measured on a daily basis to monitor health and growth. Experiments were carried out in accordance with the guidelines published in the National Institutes of Health Guide for the Care and Use of Laboratory Animals (revised 1996) and were approved by the University of Rochester Committee for Animal Research.
Visual Stimuli and Behavioral Task
The stimuli and the behavioral tasks have been described in Zaksas and Pasternak (2005). Stimuli were presented on a video monitor (17-inch Nanao FlexScan T560i, 1,152 × 870 pixel resolution, 75-Hz refresh rate), placed 42 cm in front of the monkeys. They consisted of random dots placed in a circular aperture and having constant translational step size (Δx) and temporal interval (Δt = 13 ms). The dots were 0.03° of visual angle in diameter with a luminance of 15 cd/m2, shown on a dark background of 0.1 cd/m2. Each dot persisted for the entire duration of the stimulus (500 ms) and during each frame, had an independent direction chosen randomly from a specified distribution (Fig. 1B). Thus the dots in a stimulus could move in a range of directions from 0° to 360° from the specified direction. This parameter was termed the direction range of the stimulus.
The monkeys performed a delayed direction discrimination task in which they compared the directions of two moving random-dot stimuli—sample and test—separated by a delay (Fig. 1A). The two comparison stimuli moved either in the same or in opposite directions, and the monkeys pressed one of the two adjacent buttons to report whether the trials contained the same or different directions. During each session, the directions were selected to match the neuron's direction selectivity: preferred and antipreferred. Either of these directions appeared as the sample or the test with equal probability, and thus the probability of a S-trial or a D-trial could not be predicted on the basis on the sample alone. The sample was placed either in the same location as the test or in a location in the opposite quadrant of the visual field. The two sample conditions were run in separate blocks of 200–300 trials. The monkeys initiated a trial by fixating a small spot for 1,000 ms within a 1.5 × 1.5° window and were required to maintain fixation throughout the trial. Sample and test, each 500 ms in duration, were separated by a 1,500-ms delay. After the test, the monkeys reported whether the two stimuli moved in the same or in different directions by pressing one of two response buttons. Correct responses were rewarded with a drop of juice or water, whereas incorrect responses were not rewarded and were signaled by a 3-s time-out. For each trial, the direction range of the sample was chosen at random (method of constant stimuli) from a set of four values (0°, 150°, 300°, 360°), selected to bracket each monkey's range threshold. The test stimulus always contained coherent motion (0° range). On trials with sample containing no net motion (360° range), the monkeys were rewarded at random. Direction-range thresholds were generally between 280° and 330° (see Fig. 1C) and changed little with eccentricity and the target size. This allowed us to use the same direction range values for all recording sessions, regardless of whether the sample and test were in the same or different spatial locations.
Monkeys were implanted with head-restraint devices and scleral search coils for monitoring eye position. For all surgical procedures, anesthesia was induced with ketamine hydrochloride (10 mg/kg) and diazepam (0.25 mg/kg) and maintained with 1.5–3% isoflurane. MT was localized by examining brain scans of each monkey obtained by MRI (T2-weighted, 2T and 1.5T GE magnets, small surface coil; echo time/repetition time: 5,000/90 or 3,000/85; 1.5-mm-thick slices). The superior temporal sulcus was identified from coronal scans, and a craniotomy was placed at the location from which MT could be accessed. Recording chambers, 20 mm in diameters (Crist Instruments, Hagerstown, MD), were implanted over the craniotomies by means of bone cement and titanium screws.
Recording procedures were identical to those used in previous studies (Zaksas and Pasternak 2005, 2006). During each recording session, a single epoxy-coated tungsten microelectrode (1.5–5 MΩ; FHC, Bowdoin, ME) was advanced into cortex through a steel-guide tube that penetrated the dura positioned in a Cilux grid using a hydraulic microdrive (Narishige, Tokyo, Japan). Waveforms from single neurons were isolated and recorded using a dual window discriminator (BAK Electronics, Mount Airy, MD).
The details of the procedure used for mapping MT RFs have been provided in the preceding paper (Zaksas and Pasternak 2005). Briefly, the borders of each neuron's RF were initially mapped with a joystick-controlled patch of dots while the monkey passively fixated a small cross on the display. Once the RF location and size were determined, it was filled with a stimulus of coherently moving random dots. The cell's preferred direction was determined during a session consisting of 40 trials in which eight directions of motion, separated by 45°, were presented in random order and by computing a vector average from mean firing rates. The computed direction, designated as preferred, and the opposite direction, termed antipreferred, were used during the sample and the test for the remainder of the recording session. Speed and dot density of stimuli were presented at values that elicited the largest responses for each individual neuron. Speeds varied from 3°/s to 35°/s, and dot densities varied from 2 to 5 dots/°2.
Remote sample locations.
The remote sample was always placed in the opposite and noncorresponding quadrant of the visual field at equal eccentricity to the RF. In most cases, a remote location was a reflection of the RF across the fixation point. For RFs close to or on the vertical meridian, the remote location was positioned as far as possible from the horizontal meridian while maintaining maximal distance from the RF and equal eccentricity. The details of RF locations, sizes, locations of remote stimulation, and the separation between RF and remote sample can be found in Fig. 3 of the preceding study (Zaksas and Pasternak 2005).
Analysis of spike data and statistical tests was performed using MATLAB (Mathworks, Natick, MA) and Excel (Microsoft, Redmond, WA). For the purpose of visual inspection, as seen in example plots in Fig. 2, the activity of each neuron was represented with a spike-density function obtained by convoluting the spike train with a Gaussian kernel (σ = 20) in 10-ms steps. Activity of individual neurons (Figs. 2, 3A, and 4A), as well as average responses of multiple cells (Figs. 3, B–F, and 4, B–E), was examined by sliding a 100-ms window in 10-ms increments. To allow better temporal resolution, we used a 50-ms window stepping across spike trains every 10 ms to determine times of maximal effects (Figs. 3, B and C, and 4, B and C). The analysis was based exclusively on trials with test moving in the preferred direction.
To determine whether the response to the test was affected by the direction of the preceding sample, we compared activity recorded on S-trials with D-trials. Only trials with the test stimulus moving in the preferred direction were included in the analysis. Only cells with at least five S-trials and five D-trials with the test moving in the preferred direction were included in the analysis, with the average 17 trials for each neuron. We calculated the area under the receiver operating characteristic (AROC) curve to determine the reliability of the differences between the two types of trials. In this analysis, the value of 0.5 indicates perfect overlap in the distribution of the firing rates recorded on same trials or different trials, whereas a value of 1 or 0 indicates absolute separation between the spike counts during the two types of trials, with values >0.5 indicating higher activity on S-trials and values <0.5, higher activity on D-trials. The significance level of AROC values was established with a permutation test by randomly redistributing firing rates into two groups, i.e., either “same” or “different”, regardless of the actual trial type. An AROC level was calculated from the redistributed groups, and the process was repeated 5,000 times, creating a distribution of AROC values occurring by chance, from which a P value was determined.
We also determined the number of significant epochs (P < 0.05) for our population of neurons in 100-ms bins sliding by 10-ms steps throughout the duration of the test. Because these bins were not independent, we could not assume that 5% of total epochs would be significant beyond the level of P < 0.05. Therefore, we also used bootstrap analysis to determine the number of bins expected to be significantly different from AROC = 0.5 at the level of P < 0.05. Individual trials were randomly designated as either S-trials or D-trials, with the caveat that the number of each trial type remained identical to that of the experimental data. AROC and P values were calculated in 100-ms bins slid by 10 ms across these randomly assigned trials, as was done for the experimental data. That allowed us to determine the number of epochs with significant differences between the two types of trials (P < 0.05): same > different (S > D) and different > same (D > S) epochs over the duration of the stimulus for each shuffle. This process was repeated 2,000 times for each cell, providing a distribution of the number of epochs that would carry significant comparison effect by chance, both on a population level and a single-cell level. The number of significant epochs obtained from experimental data was compared with the distribution resulting from the bootstrap analysis (i.e., fell above or below 2.5% of the shuffled distribution). This method also had the advantage of preserving the temporal structure of spiking activity associated with individual trials.
We examined whether activity during the test was predictive of perceptual decision. This relationship was explored with ROC analysis by grouping trials according to the monkey's choice, i.e., report of S-trials (right-button press) or report of D-trials (left-button press). For this analysis, we used only trials in which the sample contained random motion (360° range), thus having no net direction. The use of the random motion sample removed the potential confound of relating firing rates to the real, rather than the perceived, sample direction indicated exclusively by the monkey's choice. Choice probability (CP) of 0.5 indicates no relationship between cell activity and monkey's choice; values >0.5 indicate greater activity on trials reported as same (right-button press), whereas values <0.5 indicate higher rates on trials reported as different (left-button press). The significance level of AROC values representing CP was established with a permutation test by randomly redistributing firing rates into two groups regardless of the actual trial type, upon which AROC was calculated. This was repeated 5,000 times, creating a distribution from which a P value was determined.
We examined the activity of 171 neurons in area MT of two monkeys with sufficient data for the analysis of test responses on S- and D-trials. Of these, 118 neurons were also recorded during the task with sample presented in a remote location, and 97 had sufficient data for the analysis of comparison effects. All of these neurons showed direction-selective (DS) responses to motion presented in their RFs (average AROC = 0.97). Only trials with test stimuli moving coherently in the preferred direction were used for all of the analyses. Responses of these neurons to motion in the sample and in the test placed remotely from the neuron's RF have been described previously (Zaksas and Pasternak 2005).
Comparison Effects on Trials With In-RF Sample
We previously reported that the responses of many MT cells during the test were affected by the direction of the preceding sample (Zaksas and Pasternak 2006). We used ROC analysis to examine the differences in responses to the test when preceded by either the same or different sample direction. Although we found some neurons with reliably stronger responses on S-trials, reduced responses during S-trials were more common. As a result, the average response during these trials was significantly lower—the phenomenon we had termed “match suppression”.
In this study, we re-examined these effects, focusing on the timing of response differences during the two types of trials, their sign, and the nature of modulation that these differences represent. We used ROC analysis to reveal the presence, the reliability, and the sign of the differences in test responses reflecting sample direction. While some neurons decreased their firing rates on S-trials, other cells responded more strongly during these trials, and these effects often emerged at different times in the response. The two example neurons, shown in Fig. 2, A and B, illustrate the two types of responses that we encountered. The cell in Fig. 2A responded less on S- than on D-trials, and this difference emerged ∼100 ms after test onset. On the other hand, the response of the example cell in Fig. 2B was stronger on S- than on D-trials—the difference that emerged later in the response.
We quantified the differences between responses on S- and D-trials for these and the rest of recorded neurons with ROC analysis (see materials and methods, Data analysis) by sliding a 100-ms window across spike trains in 10-ms increments throughout the duration of the test. Figure 3A shows color-coded AROC values for individual neurons. Cells were arranged according to the time at which ROC values first rose above 0.65 (S > D cells) or fell below 0.35 (D > S cells). It can be seen that D > S signals tended to occur earlier in the response than S > D and that in individual neurons, these differences were relatively transient. From these data, individual neurons were initially assigned into two groups: cells with at least five consecutive, 100-ms periods separated by 10 ms with values >0.65 were placed into the S > D group, whereas cells with at least five consecutive periods with values <0.35 were placed into the D > S group. We found that the two types of comparison effects were carried by different neurons, with 19% of cells (n = 32) showing stronger responses on S-trials and 36% (n = 61) firing more on D-trials.
We evaluated the significance of these effects by sliding 100-ms bins in 10-ms steps across spike trains recorded during the test. This analysis revealed a total of 516 epochs with D > S activity and 217 epochs with S > D activity (P < 0.05). These numbers are significantly greater than what was obtained from randomized distributions (D > S, P < 0.0005; S > D, P = 0.0035, bootstrap analysis; see materials and methods, Data analysis), confirming that the observed comparison effects were extremely unlikely to have occurred by chance. Neurons with significant comparison effects (P < 0.05) are indicated in Fig. 3A.
Temporal dynamics of comparison effects.
Figure 3A shows that the comparison signals were largely transient, and each type is carried by a separate subset of cells. The distribution of timing of maximal comparison effects calculated in 50-ms bins sliding by 10 ms for the two groups of cells is plotted in Fig. 3, B and C. The plot in Fig. 3B shows that the cells in the D > S group were not distributed unimodally (P = 0.0002, Hartigan's dip test), forming two groups, one with a maximal effect occurring early (119 ms ± 4) and the other late (406 ms ± 11) in the response. In contrast, comparison effects in the S > D group began appearing later in the response, with no indication of bimodality (323 ± 23 ms; Fig. 3C). Based on this observation, further analysis of comparison effects was carried out separately on the three groups of cells: cells with early D > S effect, cells with late D > S effect, and cells with S > D effect.
The comparison of average firing rates on S- and D-trials of the three groups of cells is shown in Fig. 3, D–F. The plots show the differences in firing rates during the two types of trials and the time-course of these differences for the three groups of cells. We quantified these differences by comparing the firing rates with ROC analysis. The results illustrate the strength and time-course of the effects. In the early D > S group (n = 34), the comparison effect was transient, appearing early in the response, becoming significant 20 ms after the test onset (P < 0.05, one-sample t-test), and reaching its maximum 100 ms later. Cells in the late D > S group (n = 27) reached significance 110 ms after the test onset (P < 0.05). In cells in the S > D group (n = 32), the difference between responses during the two types of trials appeared even later, reaching significance 190 ms after test onset (P < 0.05, one-sample t-test). The intensity plot in Fig. 3A highlights not only the early- and late-appearing comparison effects but also their transient nature. The average durations of these signals were 117 ± 5 ms for S > D cells, 94 ± 5 ms for early, and 93 ± 4 ms for late D > S; difference among groups was not significant (P = 0.10, one-way ANOVA).
The three groups of cells were identified on the basis of their activity during the test. To determine whether neurons within each group also shared other response characteristics, we evaluated their maximal firing rates in response to motion. This was done by analyzing their responses during the sample, where the rates are uncontaminated by the comparison effects. This analysis revealed no significant differences in rates among the three groups [S > D, 130.2 ± 13 spikes per second (sp/s); early D > S, 148.9 ± 11 sp/s; late D > S, 143.1 ± 12 sp/s; P = 0.35, one-way ANOVA].
We also examined whether the cells within each group shared any other characteristics relevant to our behavioral task. We were particularly interested in their activity during the early memory delay, as we previously reported that cells with stimulus-specific, early-delay activity were also characterized by unusually high maximal firing rates (Bisley et al. 2004). Thus we examined whether the three groups differed in their activity recorded during a 200–400 ms of the early-delay epoch (see Fig. 12 in Bisley et al. 2004). This analysis revealed significant differences among groups (P = 0.04, one-way ANOVA), with only the early D > S group exhibiting stronger stimulus-specific, early-delay activity dominated by the antipreferred direction than cells with no comparison activity (AROC: 0.32 ± 0.03 vs. 0.42 ± 0.02; P < 0.05, post hoc test). There were no differences between early-delay activity for any other pair-wise comparisons among the groups (S > D cells: 0.38 ± 0.03 and late D > S cells: 0.36 ± 0.03; P > 0.05, post hoc test). This result shows that the early D > S group carried reliable signals about the identity of sample direction, both early in the delay and during the comparison test, suggesting that they may play a distinct role in the circuitry underlying the ability to perform this memory for direction task.
Direction range in sample.
The comparison effects, shown in Fig. 3, A–F, were computed for trials with sample containing coherently moving dots (0° range). As the direction range of the sample was varied from trial to trial (see Fig. 1C), we were also able to examine the effect of direction range on the comparison signals during the test. The data in Fig. 3, G and H, compare the effects recorded early and late in the response for high and low direction ranges. They illustrate the near-absence of measurable effects of range on comparison effects both in the early and late portions of the response. The early D > S group showed maximal comparison effects early (Fig. 3G) in responses, and these effects were significant at both direction ranges (significant deviations from AROC = 0.5), whereas the effects in the other two groups emerged later for both 0° and 300° direction range (Fig. 3H).
In summary, this analysis revealed that in addition to the previously reported, lower responses on S-trials (Zaksas and Pasternak 2006), MT neurons carry later-occurring signals, reflecting sample direction, some showing stronger responses on D-trials and some stronger responses on S-trials. When averaged, the lower responses on S-trials dominated the activity early in the response. Later in the response, the effects of opposite sign canceled each other, providing an explanation for the earlier-reported, apparent transient match-suppression effect during the test (Zaksas and Pasternak 2006). The current analysis also revealed that in individual neurons, the comparison effect was largely transient, occurring in different neurons at different times.
Comparison Effects on Trials With Remote Sample
We examined whether comparison signals were also present when the sample was far removed from the neuron's RF. Since under these conditions, the information about sample direction was not directly accessible to MT neurons (see discussion, Top-down influences), signals reflecting sample identity would most likely arrive in MT via top-down inputs. Stimulus conditions and the behavioral performance during this version of the task are shown in Figs. 1 and 2. The monkeys performed nearly as well with sample and test presented in opposite hemifields as when the two stimuli appeared at the same location (Fig. 1C). Test responses of two example neurons recorded on trials with remote sample are shown in Fig. 2, C and D. As with sample in-RF, we identified two types of comparison effects: some cells showed stronger responses on D-trials, whereas other cells responded more on S-trials. Examples of these effects are shown in Fig. 2, C and D.
To identify and quantify these effects, we used the same ROC analysis as that applied to responses recorded on sample in-RF trials. The intensity plot in Fig. 4A shows these effects in individual neurons, arranged according to the AROC values and their timing. Cells with significant comparison effects (P < 0.05) are indicated. It shows that some neurons showed stronger responses on S-trials and some on D-trials. It also shows that these differences were relatively brief, occurring in different neurons at different times. To classify cells into S > D and D > S groups, we used the same criteria as those for sample in-RF condition. Based on these criteria, we identified 26% of cells (n = 25) with the S > D effect and 20% of cells (n = 19) with the D > S effect. Overall, the proportion of cells showing each type of comparison effect depended on the location of the sample (P = 0.020, χ2 test), the effect most likely due to the absence of the early D > S effect and the resulting reduction in the number of D > S group during the remote sample condition (P = 0.013, χ2 test).
We used a bootstrap analysis to evaluate the significance of these effects. It revealed 180 epochs with the S > D effect, the number significantly higher than expected by chance (P = 0.038). On the other hand, 129 epochs with a D > S effect were not significantly different from chance (P > 0.05, bootstrap analysis), casting some doubt as to whether these cells represent a functionally distinct group. On the other hand, since both the bootstrap analysis and categorization by threshold have identified a small number of cells (Fig. 4A) with D > S responses, we included these cells in the analysis. While some aspects of activity displayed by these neurons suggest that they may constitute a functionally distinct cell subgroup, the results of this bootstrap analysis and their relatively small numbers set the limit on the conclusions about their comparison effects.
Temporal dynamics of comparison effects.
The distributions of times of maximal comparison signals for the two groups of cells are plotted in Fig. 4, B and C. The effects for the small number of cells in the D > S group were unimodally distributed between 150 and 400 ms, with the average maximum time of 295 ± 30 ms (P = 0.45, Hartigan's dip test; Fig. 4B). It is noteworthy that the early D > S effect, prominent on trials with sample in-RF, was absent. The effects for the S > D group (Fig. 4C) began showing up early in the response, appearing in different neurons throughout the test. The average latency (287 ± 28 ms) of these effects was similar to that for D > S cells (P = 0.85, two-sample t-test). The distribution of effects for the S > D group was within the confines of unimodal distribution (P = 0.41, Hartigan's dip test), bearing some resemblance to the effects shown by the S > D group in the sample in-RF condition.
The differences in activity during the two types of trials, quantified with the ROC analysis (Fig. 4, D and E), highlight the similarity in the time-course of both types of effects, which emerged within the first 200 ms of the response and reached their maximum toward its end, although the effect appeared earlier in the S > D group of cells (S > D cells, 140 ms; D > S cells, 60 ms; one-sample t-test, P < 0.05). Similarly to sample in-RF condition, these effects in individual neurons were transient, and their durations were not significantly different for the two types of effects (S > D, 103 ± 6 ms; S < D, 96 ± 6 ms; P = 0.67, two-sample t-test). The comparison of maximal firing rates in response to the sample for the two groups revealed no significant differences between them (D > S, 171.3 ± 15 sp/s vs. S > D, 157.3 ± 13 sp/s; P = 0.53, two-sample t-test).
Direction range in sample.
The effect of direction range in the remote sample on the comparison effect is shown in Fig. 4, F and G. The effects in the S > D group changed little on trials with 300° range (P > 0.05, two-sample t-test), retaining a significant effect both early and late in the response (50–250 ms and 300–500 ms; P = 0.0005, one-sample t-test). The D > S group exhibited similar comparison effects at both direction ranges late in the response (300–500 ms) but showed a robust comparison effect only at a 300° range during 50–250 ms after the test onset and thus a significant effect of range on the comparison effect early in the response (P = 0.0004, two-sample t-test). This effect, not observed with S > D cells, may be a consequence of the later onset of the effects in these cells, given that they peaked later than the 50- to 250-ms window used to analyze the early-response component. On the whole, the data illustrate the limited dependence of comparison effect on direction range following the remote sample. An explanation for the limited effect of direction range may lie in the robust activity and the preserved direction selectivity in response to stimuli containing a broad range of directions (Bisley et al. 2004). Similarly, DS responses in the PFC, a likely source of top-down influences in MT, also remain robust at high direction ranges during the same behavioral task (Zaksas and Pasternak 2006). Thus the activity generated during the sample at both high and lower coherence is likely to provide sufficient signals for the comparison process required by the coarse direction discrimination task used here. This is supported by only modest decreases in performance on trials with sample containing a 300° range (Fig. 1C).
Comparison Signals: Response Suppression Or Enhancement?
Our data suggest that differences in responses to identical test stimuli on S- and D-trials represent modulation of test responses by the remembered sample direction. However, the direct comparison of responses during the two types of trials does not tell us whether these differences represent response enhancement, suppression, or both. We addressed this question by comparing test responses with responses to the same stimuli. We reasoned that activity during the sample represents purely sensory responses to visual motion at the start of the trial and provides a good metric for evaluating the nature of the observed modulation of responses during the test. For this analysis, we computed a response modulation index (MI) = (Rtest − Rsample)/(Rtest + Rsample), where R refers to the average response for each cell. In this analysis, an index of 0 indicates that firing rates during the test are identical to those recorded during the sample. Positive values reflect higher rates during the test, indicative of response enhancement, whereas negative values are indicative of response suppression.
Response modulation during S- and D-trials as a function of time in the response is plotted separately for the three groups of cells in Fig. 5, A–C. Early in the response (50–250 ms), the S > D group showed no significant modulation during the two types of trials (Fig. 5A; P > 0.15, one-sample t-test). However, during the last 200 ms, these cells showed strong response enhancement on S-trials (P = 0.00001, one-sample t-test) but also response suppression on D-trials (P = 0.004, one-sample t-test). This complementary pattern of modulation, consisting of enhancement on S-trials and suppression on D-trials, carried by the same group of cells, amplifies the differences between the two types of trials. While the late D > S group showed no response modulation early in the test (Fig. 5B; P > 0.05), during the last 200 ms, its responses were strongly enhanced on D-trials (P = 0.0003, one-sample t-test) and showed some suppression on S-trials (P = 0.047), amplifying the difference in activity between S- and D-trials. Finally, the early D > S group exhibited strong suppression early in the response on S-trials (Fig. 5C; P = 0.00001, one-sample t-test) and no modulation on D-trials (P = 0.19, one-sample t-test). These neurons no longer showed significant modulation later in the response (P > 0.5). On the whole, responses of the early D > S group were strongly dominated by early suppression when the direction of test matched that of the sample, i.e., on S-trials.
This analysis revealed the existence of three types of response modulation carried by different groups of cells. Shortly after test onset, S-trials were signaled by strong response suppression exhibited by the D > S group. Later in the response, two distinct groups of neurons showed response enhancement. The S > D group increased its response on S-trials, whereas the D > S group increased its response on D-trials. Both groups further amplified these comparison effects by also decreasing their responses during the other set of trials; i.e., neurons with response enhancement on S-trials also showed some suppression on D-trials, and conversely, neurons with response enhancement on D-trials also showed suppression on S-trials.
To assess the nature of modulation by the remote sample, we compared responses of individual neurons recorded during the test with their responses recorded during sample placed in the neuron's RF. This necessitated the comparison of activity recorded during separate blocks of trials. Because such comparisons may be subject to errors due to time-dependent drifts, we limited this analysis only to neurons with comparable baseline activity and response patterns under the two conditions (n = 76). We found that the S > D group (n = 20) showed significant response suppression on D-trials both early and late in the response (Fig. 5D; P < 0.005, one-sample t-test), whereas the S-trials showed no significant modulation compared with the sample response for both early (P = 0.14) and late (P = 0.22). For the D > S group (Fig. 5E; n = 16), there was no significant response modulation on D-trials throughout the response (P > 0.36, one-sample t-test). On S-trials, there was an indication of late suppression, but this effect also failed to reach significance (P = 0.18, one-sample t-test). Thus the differences between sample and test directions were signaled primarily by suppression in the S > D group. These results highlight that during S-trials, the comparison effects depended on sample location, since the early suppression and enhancement, prominent on trials with sample in-RF, were largely absent following the remote sample. In contrast, signals associated with D-trials appeared to depend less on sample location, since S > D cells signaled these trials by response suppression under both conditions.
Comparison of modulation between the two sample conditions.
The above analysis revealed both similarities and differences between the two sample conditions. We examined whether the apparent specialization for carrying specific comparison signals revealed in individual neurons on sample in-RF trials was also expressed on trials with remote sample. This was possible by comparing MIs computed for individual neurons under the two sample conditions. We identified a subset of neurons (n = 76) with a sufficient number of trials under the two conditions to allow a direct a cell-by-cell comparison of modulation by sample direction.
We separately computed MIs for S- and D-trials and analyzed the data in two independent epochs, 50–250 ms (Fig. 6A) and 300–500 ms (Fig. 6B), after the onset of test. Data points on the plots show neurons that behaved similarly under two conditions, i.e., both showed either enhancement (top right) or suppression (bottom left). Although many neurons clustered in these two quadrants, a number of cells appeared in quadrants indicative of inconsistency of the effects for these cells during the two conditions. Overall, we found relatively modest but significant positive correlation between MIs computed for the two sample conditions (r = 0.23, P = 0.04 for S-trials; r = 0.23, P = 0.04 for D-trials). Later in the response, the modulation of many neurons became stronger, with a number of cells remaining in suppression and enhancement, indicative of the similarity in the nature of their modulation during the two sample conditions. However, many cells continued to show inconsistent patterns of modulation (e.g., enhancement during remote sample and suppression during sample in-RF). Thus while the significant, positive correlation remained, the strength of the correlation changed little (S-trials, r = 0.30, P = 0.009; D-trials, r = 0.27, P = 0.02). A degree of consistency between response modulation with sample inside and outside of the neuron's RF revealed by this analysis suggests that some of the mechanisms underlying modulation of the test response by sample direction may not depend on the site of sample origin, supporting the idea of global representation of stimuli used in the discrimination task.
Relating Activity During the Test to Behavioral Choice
In our task, perceptual decision can be made only on the basis of the comparison with the direction of the preceding sample, i.e., during or after the presentation of the test. We examined whether the activity of neurons carrying comparison effects was also more predictive of choices made at the end of each trial. This was done by comparing activity during the test on trials ending with a same report (or a right-button press) with trials ending with a different report (left-button press) and calculating CP (Britten et al. 1996; Zaksas and Pasternak 2006). Only neurons with a sufficient number of the same and different reports (more than four trials of each) were used, and the analysis was performed only on responses recorded, following with sample containing random motion (360° range). In this analysis, the values >0.5 were indicative of higher activity associated with same reports, whereas values <0.5 indicated higher activity associated with different reports.
Because of the restrictions in the type and number of trials required by this analysis, we ended up with a relatively limited number of neurons within each of the three groups with comparison effects. Nevertheless, our results revealed a link between the comparison effects and the behavioral report. Neurons with stronger responses on S-trials (S > D group) exhibited significant CP with higher rates prior to same reports (Fig. 7A). This can be seen in consistently higher CP throughout most of the test (Fig. 7A) and in the significantly shifted distribution of CPs, particularly for the second one-half of the response (Fig. 7B; 300–500 ms; CP = 0.58 ± 0.03; P = 0.03, one-sample t-test). The early D > S cells showed significant CP much later, during the last 80 ms of the test (Fig. 7C; n = 26; CP = 0.57 ± 0.03; P = 0.02). It is not known whether this activity persisted until the button press, since we did not record post-test activity. The late D > S group also showed signs of CP dropping below 0.5 toward the end of the test. This effect, however, was not significant (Fig. 7D; n = 18; CP = 0.46 ± 0.05; P = 0.4), most likely because of the limited number of cells. Finally, neurons with no comparison effects (D = S) also showed a significant, choice-related activity, indicative of higher activity prior to same report (Fig. 7, E and F; n = 67; CP = 0.53 ± 0.01; P = 0.012).
The analysis of activity recorded on trials with remote sample revealed limited differences associated with same and different reports for either cell group with comparison effects (Fig. 7, G–I). The average CP for the two cell groups did not deviate significantly from 0.5; the distribution of CPs showed no significant shifts (Fig. 7, H and I, S > D, n = 19, CP = 0.52 ± 0.04, P = 0.43; D > S, n = 13, CP = 0.53 ± 0.04, P = 0.52, one-sample t-test). This negative result is in contrast to the relatively robust CP observed on trials with sample in-RF. It is unlikely that the failure to detect the effect was due to the small cell number, since significant CP was found for sample in-RF for a group with nearly the same cell number with sample in-RF. In contrast, neurons in the D = S group showed significant CP during the last 100 ms of the test (Fig. 7, J and K, n = 47, CP = 0.56 ± 0.02, P = 0.004). Thus following the remote sample, choice-related activity was carried largely by neurons not representing sensory comparisons.
In summary, this analysis, although based on a relatively small number of neurons, revealed weak but significant choice-related activity in MT during the comparison phase of the task. While in some cases, this activity appeared to be strongest in cells with comparison signals, it was also carried by cells not showing such signals and invariably appeared late in the response, suggesting its top-down origins.
We have shown that during the comparison phase of the task, responses of many MT neurons were modulated by the direction of the preceding sample. During both sample conditions, the same and different trials were signaled by distinct groups of cells. With sample in-RF, the early D > S group showed weaker responses on S-trials, the effect that occurred soon after the onset of test and was due to response suppression on trials with matching directions. The second group (S > D) responded more strongly on S-trials, the difference that occurred later in the response and was largely due to response enhancement. While this group was the most robust predictor of monkeys' choice, other groups, including cells with no comparison activity, were not devoid of choice-related activity when both sample and test stimuli were in the RF. Finally, the late D > S group showed stronger responses on D-trials, the effect due largely to late-occurring response enhancement and some suppression. Similar specialization was observed when the sample was remote, although the nature of these effects was different. The comparison effect, represented largely by the S > D cell group, was dominated by response suppression that was greater on D-trials than on S-trials. These neurons, however, carried no detectible choice-related activity.
Origins of Response Modulation During the Comparison Test
The structure of our task allowed a direct comparison between responses to the test during S- and D-trials and responses to the same stimulus presented during the sample. Since responses of MT neurons to the sample most likely represent purely stimulus-driven activity, they provided an opportunity to evaluate response modulation by influences other than the current stimulus. This allowed us to identify the mechanisms underlying the three types of comparison signals.
Early suppression on same trials.
Approximately one-third of neurons with comparison signals exhibited this effect on trials with sample in-RF. To explain this effect, we will first consider sensory adaptation, one of the well-established, local mechanisms operating in MT. Specifically, following exposure to the preferred direction, MT cells show reduced responses during subsequently presented, identical stimulus, an effect observed in passively fixating monkeys (Van Wezel and Britten 2002) and under anesthesia (Wallish and Movshon 2008). The absence of early suppression on trials with remote sample, when responses are very weak or absent (Zaksas and Pasternak 2005), is thus consistent with adaptation playing a role during sample in-RF condition. On the other hand, there are several arguments against adaptation explaining the entirety of this effect. First, the short sample duration followed by a lengthy delay provides suboptimal conditions for adaptation-induced response reduction, since it usually requires a more prolonged exposure to the adapting stimulus (Wallish and Movshon 2008). Furthermore, the transient nature of the observed effect is inconsistent with adaptation-induced response reduction, which is likely to persist throughout the duration of the test (Van Wezel and Britten 2002). Finally, a relatively small proportion of cells showing suppression on S-trials also argues against sensory adaptation playing a major role, since a larger proportion of cells would be affected by this ubiquitous cortical phenomenon.
Another mechanism that may underlie early response reduction may be “repetition suppression” observed in the IT cortex under a variety behavioral paradigms, including a match-to-sample task (Freedman et al. 2003; Miller et al. 1993a), passive fixation (Liu et al. 2009), repetition priming (McMahon and Olson 2007), and same/different task (Woloszyn and Sheinberg 2009). This response reduction, thought to represent a passive comparison mechanism (Miller and Desimone 1994), resembles the response suppression on S-trials observed here. The similarities include its relatively early onset and its presence in a distinct population of cells. Indeed, the cells exhibiting early suppression on the same trials showed no modulation following antipreferred sample expected from adaptation-based phenomena (Priebe and Lisberger 2002), suggesting their unique specialization for detecting stimulus similarity. In any case, the presence of repetition suppression does not exclude the possibility of adaptation contributing to the effect. It is indeed possible that they share similar neural mechanisms occurring over different time scales.
In a previous study, we examined the nature of cells carrying stimulus-related, early-delay activity and found that such neurons have unusually high firing rates (Bisley et al. 2004). Here, we examined early-delay activity in the three groups of cells identified on the basis of their responses during the test. Interestingly, only the early D > S group showed significantly stronger early-delay activity than the cells with no comparison effects. This observation suggests that these neurons may play a distinct role, not only during the comparison test but also during the memory delay.
Late response enhancement.
These late-occurring D > S signals dominated response enhancement on D-trials with weak suppression on S-trials, amplifying the difference between the two types of trials. The effects in the S > D group were largely in the form of late-response enhancement. Here, however, the enhancement signaled S-trials and was accompanied by weak suppression on D-trials. Thus the two groups of cells functioned in a complementary fashion, each specialized in signaling one of the two types of trials. These late-occurring signals dominated by enhancement are likely to represent a more active process, probably supplied by top-down influences (Miller et al. 1993b). The top-down origins of the S > D effect are supported by its prevalence on trials with remote sample (see below, Top-down influences). In contrast to the enhancement on S-trials during sample in-RF condition, these cells showed strong response suppression during D-trials. These opposite patterns of modulation suggest the operation of different comparison mechanisms during the two sample conditions, both likely top-down origins and in line with the recently proposed model of the circuitry underlying sensory comparisons (Engel and Wang 2011).
Responses of MT neurons to motion are characterized by the early transient phase, followed by a more sustained component thought to represent the effect of short-term adaptation of intracortical origins (Priebe et al. 2002). Since the transition to the sustained phase of the response occurs ∼80 ms after stimulus onset, the late-response enhancement observed here takes place largely during the sustained phase of the MT response. While our data shed no direct light on the relationship between the two phenomena, the transient nature of the comparison effects and their appearance at different points in the response suggest that they may be largely independent of the sustained component of the MT response that persists until stimulus offset.
Access to the information about sample direction.
We previously reported that MT neurons do not carry consistent information about sample direction throughout the memory delay (Zaksas and Pasternak 2006). Indeed, at the end of the delay, such signals are exceedingly rare. Nevertheless, soon after the onset of test, this information is used in the comparison signals. While late-occurring comparison signals recorded under both sample conditions could be explained by top-down influences, the early suppression is more likely to be local to MT. In the absence of consistent spiking activity representing this information during the delay, its source is unclear. One solution to this problem has been offered by the computational work of Mongillo et al. (2008), who proposed that the information about past spiking activity is maintained by the calciummediated synaptic facilitation in a network of cells with recurrent connections.
We introduced the remote sample condition to determine whether and how the information about sample direction originating in the opposite hemisphere can be used by MT neurons. We reasoned that stimuli presented in opposite and noncorresponding quadrants relative to MT RFs could be accessed only via top-down inputs. This reasoning was based on the fact that interhemispheric connections between MTs in the two hemispheres are confined to the mirror-symmetric locations (Krubitzer and Kaas 1990; Maunsell and van Essen 1983) and on the absence of known anatomical connections between representations of noncorresponding locations of the opposite hemifields [for discussion, see Zaksas and Pasternak (2005)]. Thus a given MT neuron is unlikely to have a direct access to both comparison stimuli appearing in opposite and noncorresponding field locations. Under these conditions, MT neurons would most likely have to rely on the information arriving from downstream areas with access to both hemispheres and large RFs. This information would either be in the form of comparison signals or activity representing the direction of the remote sample. There is no dearth of evidence for MT neurons being affected by top-down influences. Their activity is affected by spatial and feature-based attention (Cook and Maunsell 2002; Martinez-Trujillo and Treue 2004; Treue and Maunsell 1999), reflects time-dependent anticipation (Bisley et al. 2004), and represents top-down directional signals arriving from the opposite MT (Zaksas and Pasternak 2005). The present data demonstrate that the top-down modulation of activity in MT also involves the comparison phase of the delayed direction discrimination task.
PFC is commonly considered to be an important source of top-down influences recorded in sensory neurons (Miller and Cohen 2001). Indeed, PFC cells have been shown to faithfully represent sample and test directions, carry some stimulus-related signals during the delay, and exhibit comparison effects during our task (Hussar and Pasternak 2009; Zaksas and Pasternak 2006). Thus these neurons could provide MT with signals representing sample direction after it had appeared at a remote location, as well as with the modulation during the test. The late enhancement following in-RF sample may have originated in the PFC, and the possibility of a common source for the signals recorded under the two sample conditions is supported by significant correlation between them (see Fig. 6). Other potential sources of top-down signals arriving in MT during our task include parietal cortex, strongly interconnected with both MT and the PFC (Lewis and Van Essen 2000; Medalla and Barbas 2006), some with very large RFs (Ben Hamed et al. 2001), strong direction selectivity (Colby et al. 1993; Fanini and Assad 2009), and documented, active involvement in direction-discrimination tasks (Freedman and Assad 2009; Shadlen and Newsome 2001).
Behavioral Relevance of Comparison Signals
One of the limitations of examining the CP in our data was a relatively small number of neurons with comparison signals with a sufficient number of trials recorded with 360° sample. As a result, this analysis could be performed only on a subset of cells with comparison effects. A second limitation was that our recordings of neural activity were discontinued at the time of test offset, and we were unable to determine whether choice-related activity persisted until the monkey pressed a button to indicate choice. Despite these limitations, we found a reliable link between comparison effects and behavioral choice. Neurons in the S > D group showed particularly robust CPs that persisted until test offset. In this case, the sign of the choice-related activity (higher activity for same choices) was congruent with the comparison signal type (higher activity for S-trials). Furthermore, evolution of both memory and choice-related activity followed a similar time-course, suggesting a possible, functional link between this group of cells and the comparison/decision circuit. In this context, we were surprised by the very different patterns of CP exhibited by the other two cell groups. CP of the early D > S group showed higher rates prior to same reports during the last 100 ms of test, whereas during the same period, the late D > S group showed an opposite trend toward higher activity prior to a different report. Finally, we also observed significant CP in cells with no comparison effects during both sample conditions. Thus although our data show that cells with comparison signals can also carry choice-related activity, the presence of significant CP in cells with no comparison effects suggests that at the level of MT, the two types of activity may be independent.
This independence is consistent with the notion that decision-related signals recorded during our task are being formed outside of MT. Indeed, the presence of significant decision-related signals in the PFC during the same task (Zaksas and Pasternak 2006) points to these neurons as a potential source of choice-related activity observed here. This does not rule out choice-related signals relying on the information provided by sensory comparison signals, an association also observed across cortical areas in monkeys performing a vibration discrimination task (Jun et al. 2010).
Comparison With Other Studies
Our task differs from the frequently used working memory task involving comparisons between two stimuli, the match-to-sample task. In the most common version of this task, monkeys are presented with a sample, followed by a sequence of stimuli, and release a lever when one of these stimuli matches the sample (Miller et al. 1993b). Thus during this task, the animals actively search and report only the matching stimulus. In contrast, in our task, the monkeys reported not only the same (match) but also different (nonmatch) trials and were rewarded equally for reporting each. Thus the same trials (i.e., match) did not have a special significance to the animals. Recordings from IT cortex during the match-to-sample task revealed that the majority of neurons showed early suppression on match trials, and a small proportion of cells showed stronger responses later on such trials (Miller et al. 1993b). This pattern of results is in contrast to our finding of a large proportion of MT neurons with comparison effects showing stronger responses on different (nonmatch) trials. One explanation of these differences may lie in the difference between the match-to-sample task used by Miller et al. (1993b) and the same/different task used here. Indeed, in our study, the same and different trials were signaled by separate cell groups, reflecting the demands of the same/different task, just as “match enhancement” signals in IT reflected the demands of the match-to-sample task. The finding that the late-occurring comparison signals in MT reflect task demands is consistent with these signals being supplied by the PFC, the region known to adapt to the behavioral context (Hussar and Pasternak 2009).
The magnitudes of the comparison effects reported here are relatively modest compared with responses to sensory stimuli and are typical of those found for nonsensory modulations in visual cortex (Cook and Maunsell 2002; Martinez-Trujillo and Treue 2004; Miller et al. 1993b). Recent work by Cohen and Maunsell (2010) in the context of spatial attention has found that small signals, which may be considered inconsequential on a single-cell level, when carried by a larger number of neurons, are highly predictive of behavior. It is quite possible the same principle will apply here.
In summary, we have identified different types of comparison signals in MT, each with a distinct, underlying neuronal mechanism, involving both a spatially constrained bottom-up component, as well as spatially independent top-down modulation, which also evoke a representation of the preceding sample. The finding that the activity of many neurons, including cells with comparison signals, is predictive of perceptual decisions underscores active participation of MT in comparisons involving motion direction.
This work was supported by grants from the National Eye Institute: RO1 EY11749 (T. Pasternak) and P30 EY01319 (Center for Visual Science, University of Rochester), and the National Health and Medical Research Council, Australia: Overseas Postdoctoral Fellowship (L. L. Lui).
No conflicts of interest, financial or otherwise, are declared by the author(s).
Daniel Zaksas collected the data. We are grateful to James Bisley, Albert Compte, Greg DeAngelis, Cory Hussar, and Duje Tadin for comments on the manuscript. We also thank Marc Mancarella for technical assistance and Martin Gira for help with electronics.
- Copyright © 2011 the American Physiological Society