| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1Laboratory of Sensorimotor Research, National Eye Institute and 2Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland
Submitted 31 March 2004; accepted in final form 12 August 2004
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
, 2002; Bichot et al. 2001a, b; Murthy et al. 2001; Sato and Schall 2003; Sato et al. 2001, 2003; Schall and Hanes 1993; Schall et al. 1995, 2004; Thompson et al. 1996). The consistent finding across all of these studies is that after the presentation of a visual search array, but before gaze shifts, a selection process takes place by which most visually responsive neurons in FEF signal the location of the singleton target stimulus. Missing from these reports, however, is information about what happens on error trials, on those trials in which one of the distractors is chosen as the saccade goal.
Models of visual attention and saccade target selection posit that peaks of activity across a hypothetical visual salience map in the brain encode stimulus conspicuity with the highest peak winning out to guide attention and gaze shifts (e.g., Findlay and Walker 1999; Itti and Koch 2001). However, the term "visual salience map" is somewhat misleading because during active vision, the physical salience of objects is only partly responsible for attracting gaze. The viewer's knowledge contributes greatly to what attracts attention. For example, knowing the color of the object one is looking for makes all objects of that color stand out and therefore more likely to attract gaze (e.g., Bichot and Schall 1999a; Motter and Belky 1998). Normally, visual attention and gaze are guided by the combination of bottom-up intrinsic visual salience and by the top-down knowledge and expectations of the viewer (Yarbus 1967). Therefore in this report, the term "salience" is used to describe visual conspicuousness derived from both bottom-up and top-down influences.
Evidence is growing that, at least on correctly performed visual search trials, activity across the FEF reflects the combination of bottom-up and top-down influences and functions as a salience map for guiding saccades (reviewed in Thompson et al. 2001). According to the salience map hypothesis, activity on error trials should specify the next saccade and the magnitudes of activation representing the search stimuli should reflect the relative importance of each stimulus within the context of the task and predict the likelihood of making the saccade choice (Wolfe 1994). Alternatively, it is possible that FEF does not identify the goal of every saccade; instead FEF activity might identify the most physically salient object of the search array on both correct trials and error trials. Evidence for this possibility comes from previous work showing that FEF neurons select the singleton stimulus of a popout search array even when saccades are not made (Thompson et al. 1997). Further evidence comes from a study showing that FEF activity accurately tracks the jump of a singleton target in a double-step search task even when the saccade was not made to the singleton target but instead made to the distractor at the previous target location (Murthy et al. 2001). At the other extreme is the possibility that because the saccades are the same, the presaccadic activity will be the same on both correct and error trials and therefore not encode the salience of the stimulus. Evidence for this possibility comes from saccade countermanding studies that show that saccade-related activity in FEF rises to a specific and constant threshold on both correct and error trials before saccade initiation (Hanes and Schall 1996; Hanes et al. 1998).
To distinguish between these possibilities, we analyzed neuronal data collected from monkeys performing 2 different visual search tasks in which there were enough saccade choice errors to reliably evaluate the activation on error trials. The first task was an easy–hard search task in which the stimulus features were varied randomly from trial to trial. Easy search trials in which the singleton target was easily distinguished from the distractors were interleaved with hard search trials in which the singleton target was similar to the distractors. On hard search trials the monkeys made a significant number of erroneous saccades to one of the distractors. No target trials, in which monkeys were rewarded for maintaining fixation, were included with some of the recordings that resulted in some false alarm errors. Also, during the recordings that included no target trials, the monkeys occasionally maintained fixation when a target was present. The second task condition was a popout color search task in which the target and distractor features switched over trials. Unlike the bottom-up derived errors in the easy–hard task, in this task condition the errors are attributed to a top-down influence. In the feature-switching task the subjects become primed to make saccades to the stimulus with a specific feature, which leads to a higher percentage of errors on the first few trials after the switch (Maljkovic and Nakayama 1994).
Results were similar for both tasks and support the hypothesis that activity across FEF forms a salience map, a topographic representation of saccade probability. The selective activation was highest for the location that encoded the saccade goal, but the magnitude of selection before the saccade reflected the probability of choosing that item as the saccade goal. Some of these data previously appeared in abstract form (Thompson et al. 2002).
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
Data were collected from 4 macaque monkeys (F, M, L, O), Macaca mulatta and Macaca radiata, weighing 4–10 kg. The animals were cared for in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and the guidelines of the Vanderbilt Animal Care Committee. The surgical procedures were described previously (Schall et al. 1995).
Behavioral procedure
Using operant conditioning with positive reinforcement, all of the monkeys were trained to perform a singleton visual search task in which reward was contingent on shifting gaze to an oddball target. After fixation of a central spot for about 600 ms, the target was presented at one of 4 or 8 iso-eccentric locations equally spaced around the fixation spot. The remaining locations were occupied by the distractors. During the physiological recordings the stimuli were placed so that at least one stimulus always fell within the response field of the neuron. The monkeys were rewarded for making a saccade to the target within 500 ms after the search array was presented and fixating the target for 400 or 500 ms. If the monkey broke fixation before stimulus presentation, made a saccade to a location other than the target, made a saccade to the target but failed to fixate it for the prescribed period, or did not initiate a saccade within the prescribed period, the trial was immediately aborted, and the monkeys failed to receive the juice reward. This undermined an analysis of subsequent saccades but encouraged monkeys to find the target on the first saccade. On average monkeys ran about 800 search trials while recordings were made from each neuron. Two different variations of this basic search task were used to manipulate the monkeys' performance accuracy: easy–hard search and popout search with feature switching.
EASY-HARD SEARCH TASK.
In the easy–hard visual search task the singleton target was presented with 7 distractors that were either different from or similar to the target. Target–distractor similarity was adjusted so that the mean reaction time was 
30 ms longer in the hard search condition. The easy and hard search trials were randomly interleaved and on each trial the target appeared randomly at one of the 8 possible locations. The target and distractors were distinguished by either color or direction of motion (Fig. 1, A and B).
|
; Nagy and Sanchez 1990). For monkey F, the green was CIE (International Commission on Illumination) x = 283, y = 612, red was CIE x = 655, y = 327, and yellow-green was CIE x = 363, y = 552, all having luminance of 11.1 cd/m2. For monkey M, the green was CIE x = 281, y = 609, red was CIE x = 632, y = 338, and yellow-green was CIE x = 375, y = 538 with a luminance of 13.4 cd/m2.
Data were collected from 3 monkeys (L, M, and O) during the easy–hard motion search experiment. For motion search, each stimulus was a circular aperture of randomly positioned dots, a proportion of which translated coherently in a specified direction, whereas the remaining dots were replotted at random locations every 3 video frames (Fig. 1B). The apertures were 2.5° across and were presented at 10° eccentricity. This eccentricity allowed for at least one stimulus location to be inside the neuron's response field. The stochastic motion stimulus corresponds to that used in earlier studies (e.g., Britten et al. 1992) and was described previously (Sato et al. 2001, 2003). The direction of motion was either left or right, and the direction of motion of the target and distractors remained the same during each recording session and varied pseudorandomly across sessions. On easy motion search trials all of the dots in a given aperture moved in the same direction. On hard motion search trials 40–50% of the dots in each aperture moved in random directions.
The easy–hard color and motion visual search experiments with monkeys L, M, and O included 30–45% randomly interleaved catch trials, in which only distractors were presented (Sato et al. 2003). On these catch trials, the monkeys had to maintain fixation on the central spot for 1,500 ms to obtain the reward. For both color and motion search, there were 2 different sets of distractors used for the catch trials; one set (red or high motion coherence) was associated with easy search and the other set (yellow-green or low motion coherence) was associated with hard search.
POPOUT SEARCH WITH FEATURE-SWITCHING TASK. Data were collected from 2 monkeys (C and F) performing the popout search task with feature switching. In this experiment, the monkeys were rewarded for making a single saccade to a target among 3 distractors that differed from it in color (i.e., red target among green distractors or green target among red distractors) (Fig. 1C). The color of the target and distractors switched across trials with a probability of 50 or 33%, or in blocks of 10 trials; the 3 different switch probabilities were pseudorandomly intermixed within each recording session. An example sequence of popout search trials is shown in Fig. 1C. The 2nd trial in this sequence represents the 1st trial after a feature switch, and the next trial represents the 2nd trial after the feature switch; the last trial of the sequence represents another 1st trial after a feature switch.
MEMORY-GUIDED SACCADE TASK.
Monkeys were also trained on the memory-guided saccade task (Bruce and Goldberg 1985; Hikosaka and Wurtz 1983). This task served 2 purposes: to distinguish visual from movement activity for cell classification, and to map the spatial extent of each neuron's response field. In this task, the target was flashed alone for 80 ms, but the monkeys were required to maintain fixation on the central spot for another interval of random duration ranging from 400 to 1,000 ms. When the fixation spot disappeared, the monkeys were rewarded for making a saccade to the remembered location of the target. Once gaze shifted, the target reappeared to provide feedback and a fixation target for the monkeys. Neurons were classified as those exhibiting only visual responses (visual neurons), those exhibiting only movement related responses (movement neurons), and those exhibiting both visual and movement related responses (visuomovement neurons).
Data collection and analysis
Single units were recorded with insulated tungsten electrodes (FHC). The electrodes were introduced through guide tubes positioned in a 1-mm-spaced grid (Crist et al. 1988) and were positioned with a hydraulic drive (FHC). Action potentials were amplified, filtered, and discriminated using either an analog time–amplitude window discriminator (BAK) or computer-based window discriminator (Plexon). FEF recordings were done in the rostral bank of the arcuate sulcus, which was confirmed with magnetic resonance imaging or histology.
We used a method adapted from signal detection theory (Green and Swets 1966) to determine the time course of discrimination and to what degree the activity of each neuron discriminated the saccade goal on correct trials and on error trials. This method was previously described (Thompson et al. 1996).
First we identified the stimulus locations that were clearly inside and outside of the neuron's response field during the memory-guided saccade task. Often 2 or 3 of the stimulus locations were determined to be inside (or outside) the response field because the responses on correct trials were indistinguishable when the target was presented at those locations. Trials involving locations at the edge of a response field that produced weak responses to a target were excluded from all analyses. During the visual search task, the monkeys did not exhibit tendencies to preferentially make incorrect saccades to any of the stimulus locations so there was no evidence of saccade preference biases that could have potentially affected the results.
We generated a spike density function for each trial by convolving action potentials with a Gaussian filter (
= 10 ms). For the analysis of neural activity on correct trials, we compared the distributions of discharge rates when the saccade was made to the target in the response field to the distributions of discharge rates during trials when only distractors were in the response field and the saccade was made to the target outside the response field. For the analysis of error trials we compared the distributions of discharges rates when the saccade was to a distractor in the response field and the singleton target was outside the response field to the distributions of discharge rates when the saccade was to a distractor outside the response field and the singleton target was inside the response field. The comparison was made at 1-ms intervals starting at the time of search array presentation for each neuron. The separation of the 2 distributions of activity at each time interval was quantified by calculating receiver operating characteristics (ROC) curves (Thompson et al. 1996). Points on the ROC curve were generated by plotting the fraction of trials of one distribution with discharge rates greater than a criterion as a function of the fraction of trials of the other distribution with discharge rates greater than the same criterion. The entire ROC curve was generated by incrementing the criterion from zero to the maximum discharge rate observed on a single trial in steps of 1 spike/s. The area under the ROC curve represents measures of the separation of the 2 distributions of activity. An area under the ROC curve value of 0.5 signifies that the 2 distributions being compared are completely indistinguishable, whereas a minimum value of 0.0 or a maximum value of 1.0 signifies that the 2 distributions do not overlap at all. As a convention, ROC area values >0.5 indicated that the activation on the trials when the saccade was into the response field was greater than the activation on the trials when the saccade was to a location outside the response field. For correct trials, the saccade goal was the same as target location; however, on error trials, the saccade goal was not the same as the target location. For clarity we use the term "target" when referring to the singleton target of the search array and "saccade goal" when referring to the stimulus to which the monkeys shifted gaze.
The time course of the discrimination process was quantified by plotting the area under the ROC curve as a function of time. The time course of the selection process after the presentation of the search array is most evident when the data are aligned on the time of search array presentation. The magnitude of selection reached before saccade initiation is best measured when the data are aligned on the time of saccade initiation. To determine how well neurons discriminated the saccade goal before saccade initiation, we averaged the ROC area over the last 30 ms before saccade initiation. The 30-ms time duration was arbitrary; however, when other durations were used, the results did not differ qualitatively. Potential changes in baseline neuronal discrimination were investigated by calculating the average ROC area for the first 30 ms after search array presentation. This duration probed neural activity biases that may have been present immediately before the initial visual responses to the search array. The nonparametric Kruskal–Wallis test was used to identify significant variation in the average ROC area across trial conditions.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
MISLOCALIZED TARGET ERRORS.
Easy search trials were randomly interleaved with hard search trials (Fig. 1, A and B). Neural activity on correct trials recorded with this task was previously examined (Bichot et al. 2001; Sato et al. 2001). Data were collected with 3 variations of the easy–hard search task: color search without target absent trials, color search with target absent trials, and motion search with target absent trials (see METHODS). In this section we will compare the activity on target-present trials in which the monkey incorrectly shifted gaze to one of the distractors to the activity on correct trials.
The monkey's behavioral performance during the easy–hard search task revealed the difference in task difficulty on easy and hard trials. During easy search the target was very different from the distractors (e.g., a green target among red distractors) and the monkeys correctly performed the task on 95.4% of trials. During hard search the target was similar to the distractors (e.g., a green target among yellow-green distractors) and the monkeys correctly performed the task on only 76.1% of trials. The relative difficulty was also evident in the saccadic reaction times (Fig. 2). On correct trials, the average reaction time for easy search was 200.3 ms and for hard search was 264.0 ms. On error trials in which the monkey made a saccade to one of the distractor stimuli, the average reaction time was 215.2 ms for easy search and 251.8 ms for hard search.
|
5 trials for each of the 2 error conditions to be compared. For this analysis, the first error condition was when the target fell in the neuron's response field and the saccade was made to a distractor outside the neuron's response field, and the second error condition was when the target fell outside the neuron's response field and the saccade was made to a distractor in the neuron's response field. For hard search, enough data were collected from 102 neurons. There were too few errors on easy search trials to analyze for any of the neurons. The results obtained from the individual monkeys, from data collected with the color or motion search, and from data collected during recording sessions that included or did not include target absent trials were not different, and therefore the results are combined.
The activity of a visually responsive FEF neuron on easy correct trials, hard correct trials, and hard error trials is shown in Fig. 3 aligned on the time of search array presentation (Fig. 3A) and on the time of saccade initiation (Fig. 3B). This neuron exhibited the typical target-selection response on correct trials that was reported previously for FEF neurons (Bichot et al. 2001b; Sato et al. 2001; Schall et al. 1995; Thompson et al. 1996). The initial visual response did not discriminate the target from distractors, but over time the activity evolved to indicate whether the target was in its response field. The time course and magnitude of this discrimination process were determined using an ROC analysis (Thompson et al. 1996). The area under the ROC curve estimates the probability of an ideal observer to correctly choose to which of 2 distributions a sample belongs. It is also a convenient nonparametric method of obtaining a measure of the difference between 2 distributions normalized to values between 0 and 1, with 0.5 indicating completely overlapping distributions. Initially after the search array presentation but before the initial visual response of the neuron (0–50 ms), ROC areas were about 0.5, which indicates equivalent baseline activity. During the initial visual response beginning at about 50 ms the ROC areas remained at 0.5, which reflects the inability of the initial visual response to distinguish the target from distractors. At about 100 ms after search array presentation the ROC area begins to grow and it eventually reaches an asymptotic level before saccade initiation. This growth reflects the process by which the target of the search array is identified. The maximum level of the ROC area before saccade initiation is a measurement of how well the neuron was able to distinguish the target from the distractors. There are 2 main differences in the discrimination process between easy and hard correct trials. First, after search array presentation the discrimination process emerged earlier on easy trials than on hard trials. Second, the reliability of selection before saccade initiation, as indexed by the presaccadic ROC area, reached a higher level before saccade initiation during easy search than during hard search. Both of these findings were previously reported (Bichot et al. 2001b; Sato et al. 2001).
|
). However, this pattern was not observed in the other neurons in our sample (see pooled average in Fig. 5); thus overall there was not an initial selection for the singleton target of the search array on error trials.
|
|
). This was done because the results were indistinguishable across the different neuron types (see following text, Fig. 10). Aligning on the time of target presentation or the time of saccade initiation did not alter the results. However, the time course of the selection process after the presentation of the search array is most evident when the trials are aligned on the time of search array presentation (Fig. 5, A and B), and the presaccadic selection activity is most easily observed when the trials are aligned on the time of saccade initiation (Fig. 5, C and D). At the population level, selective activity emerged earliest on easy trials. The selection process on hard correct trials and hard error trials had a similar time course. Also, on error trials, neurons selected the saccade goal, not the singleton stimulus of the search array. However, the magnitudes of selection before saccade initiation across the 3 trial conditions were not equal. A Kruskal–Wallis test revealed that there was significant variation in ROC areas in the 30 ms before saccade initiation (P < 0.001) across the 3 trial conditions. The magnitude of selection before saccade initiation was highest for easy correct trials, followed by hard correct trials, and lowest for hard error trials (Fig. 5, B and D). Lower ROC areas are a result of both lower activity representing the saccade goal and higher activity representing the nonsaccade stimulus (Fig. 5, A and C).
|
). False alarm errors are defined as those catch trials on which the monkey made a saccade to one of the stimuli before the 1,500-ms time limit expired. Figure 6B shows the cumulative reaction time distributions for the data collected with catch trials. The reaction times of easy false alarms averaged 297.3 ms, and hard false alarms averaged 318.0 ms. Approximately 10% of the false alarm trials had reaction times >500 ms and were outliers from the main part of the distribution. On target present trials, the monkeys were required to make a saccade within 500 ms. Therefore to make a fair comparison across all trial types, only those trials with reaction times <500 ms were included in the neural activity analysis.
|
Figure 7 shows the pooled average activity on false alarm trials for the 28 neurons that met the trial number criteria for easy and hard false alarms. Also shown in Fig. 7A is the activity on correct catch trials (thin lines). The activity on correct hard catch trials was greater than the activity on correct easy catch trials after about 150 ms after stimulus presentation; this difference between correct easy and hard catch trials is attributed to the similarity of the distractors to monkeys' memory of the target feature and was the subject of a previous report (Sato et al. 2003). The presaccadic activity on false alarms identified which distractor stimulus was the saccade goal. The distractor that was the saccade goal on false alarm trials evoked greater responses than identical distractors on correct catch trials. In addition, distractors that were not the saccade goal on false alarm trials evoked weaker responses than identical distractors on correct catch trials. The time course of the selection of the saccade goal on false alarm trials most closely matched the time course of selection process on the target-present trials that shared the same distractor feature; the saccade goal was selected earlier on easy false alarms than on hard false alarms (Fig. 7B). Before the saccade (Fig. 7, C and D), the level of discrimination was slightly higher on hard false alarms than on easy false alarms. Although the difference in the magnitude of presaccadic selection between the 2 false alarm trial groups did not reach statistical significance (Kruskal–Wallis, P = 0.18), there was significant variation in the magnitude of presaccadic selection across the 5 trial conditions that ended with a saccade (P = 0.002). In the next section we will relate the variation in the magnitude of presaccadic selection to the probability of making each saccade choice.
|
). In the present analysis, the ROC area corresponds to the probability of an observer correctly assigning the activation on a single trial to the group of trials it belongs to when the decision rule is that higher values correspond to the location of the saccade goal and lower values correspond to one of the other stimulus locations. Our hypothesis is that the population activity across the retinotopic map within FEF represents the integration of bottom-up and top-down salience and the peak on this map guides the saccade in a probabilistic fashion. This hypothesis predicts that when 2 activity peaks are similar, there is a greater likelihood that because of variability, the usually lower activation will exceed the usually higher activation and will result in more errors. If this hypothesis is correct, then the ROC area reached before saccade initiation will be correlated with the probability of making a saccade to the location with the highest activation. As in previous studies using "antineuron" analysis techniques (e.g., Britten et al. 1992; Thompson et al. 1996), this analysis assumes that pooling multiple trials from single neurons estimates the population response on single trials. Our previous work has shown that this assumption is reasonable for these data (Bichot et al. 2001b). The probability of the monkeys generating a saccade varied across the different stimulus–response conditions. There were enough trials to compare presaccadic activity for 21 neurons. All neurons were recorded in separate recording sessions. For these 21 recording sessions the monkeys, on average, made a correct saccade to the target on 94% of easy search trials and 75% of hard search trials. On 20% of hard search trials the monkeys made an incorrect saccade to one of the distractors. False alarm saccades were made on 19% of easy catch trials and on 43% of hard catch trials. These percentages varied somewhat across recording sessions and provided the behavioral measurement of saccade probability for each stimulus–response condition (see Fig. 8B).
|
However, it is possible that differences in saccade metrics between the different trial conditions contributed to the differences in neural activity and therefore to ROC area. To test for this possibility, we performed a multiple regression analysis, regressing ROC area against probability of saccade choice, average saccade peak velocity, average saccade amplitude, and average saccade latency. Because the eccentricity of the stimuli varied across sessions according to response field eccentricity, it was necessary to normalize the velocity, amplitude, and saccade latency measurements as a percentage of the mean values from each session. The result of the multiple regression analysis was that only saccade probability accounted significantly (P < 0.05) for the variation in ROC area [y-intercept = 0.76, slope = 0.15; F(1,103) = 24.9, P < 0.001].
The only behavioral measure that accounted for the variation in presaccadic ROC area was the monkeys' probability of generating the behavior. We cannot know whether the relationship continues to be linear as the probability of saccade choice approaches zero. Nevertheless, the correlation does suggest that, when given the option of not making a saccade, the difference in activation between 2 similar activity peaks in FEF must reach some threshold before a saccade is made. An analysis of activity on trials in which a singleton target was present but a saccade was not made tests this hypothesis.
MISSES.
Previous work has shown that when activity in FEF movement neurons reaches a threshold, a saccade is generated. When the growth of movement activity is suppressed before the threshold is met, no saccade is made (Hanes and Schall 1996, Hanes et al. 1998). Our data now suggest this may also be true for FEF activity related to deciding which object to look at. The hypothesis that a threshold of selection must be reached before an object is chosen as the saccade goal was examined by looking at the activity on miss trials. A miss was defined as those target-present trials on which the monkey maintained fixation for the entire time allowed (500 ms), at which time the visual stimulus was turned off and the trial was aborted. For easy search, misses occurred on just 0.2% of trials; there were not enough data to analyze easy search misses for any of the neurons. For hard search, misses occurred a little more frequently, on 3.7% of trials. There were enough data to analyze the activity on misses on hard search trials for 10 neurons. For the data collected from these 10 neurons the miss rate was 6.7%.
Figure 9 compares the pooled average activation for the trials associated with the hard search condition for the 10 neurons with enough trials to analyze in all 4 possible stimulus–response conditions: correct trials, mislocalized saccade trials, false alarms, and misses. On miss trials, there was a selective response that identified the location of the singleton stimulus. Because no saccade was made on miss trials, it is not possible to compare the magnitude of selection on miss trials to the values plotted in Fig. 8B. However, it is evident in Fig. 9 that on miss trials the magnitude of selection never reached the level that was reached for the trials that resulted in a saccade. Although the data are limited, this result supports the hypothesis that when given a choice to make a saccade or not in the presence of multiple competing stimuli, there is a selection threshold that must be reached in FEF before a saccade goal is identified and a saccade is made.
|
), the presaccadic movement activation in FEF was different across different stimulus–response combinations in this task, even though the saccades were not different. Popout visual search with feature switching
In the 2nd experiment, 2 monkeys performed a popout color search task in which the stimulus colors that defined the target and distractors switched unpredictably. The target and distractors colors were easily distinguishable, red and green. On each trial, the monkey was rewarded for making a saccade to the singleton target. However, the target and distractor colors switched unpredictably every few trials (Fig. 1C), and this manipulation resulted in a saccade to a distractor on many trials. The greatest effect on saccade choice was on the 1st trial after the target and distractors switched colors. The monkeys made an incorrect saccade to a distractor on 34% of the 1st trials after the feature switch. On the 2nd trial the error rate decreased to 20% and by the 5th trial after the switch the error rate stabilized at about 9%. A complete description of the behavior and neural activity on correct trials during this task was previously published (Bichot and Schall 2002). Because there were relatively few error trials after the 2nd trial after the switch, it was necessary to combine the results from the 2nd trial and all subsequent trials until the next feature switch into one group.
Reaction times reflected the differences in difficulty across trials. Figure 11 shows the cumulative reaction time distributions. The longest reaction times were on correct trials immediately after the feature switch (average = 292.1 ms). The shortest reaction times were on the easy correct trials, trials that occurred on the 2nd trial or later after the feature switch (243.3 ms). Reaction times on error trials were intermediate to the correct trials, and the reaction time distributions did not differ significantly for the errors that occurred on the first trial or on the later trials after the feature switch (1st trial errors = 254.7 ms; 2+ trial errors = 252.2 ms).
|
The distribution of baseline ROC areas and presaccadic ROC areas on error trials from individual neurons is shown in Fig. 12. The results were the same as in the easy–hard visual search task. There was no evidence of a pretrial bias that could influence saccade choice in the 30 ms immediately after the presentation of the search array. During this time the ROC areas averaged 0.51 on the first trial after the switch, and 0.50 on the later trials. Neither distribution was significantly different from 0.5 [1st trial: t-test, t(29) = 0.347, P = 0.7; 2+ trials: t(29) = 0.141, P = 0.9]. In the 30 ms before saccade initiation, nearly all of the neurons had ROC areas >0.5 [1st trials: t(29) = 8.25, P < 0.001; 2+ trials: t(29) = 6.47, P < 0.001]. Before errant saccades the ROC areas averaged 0.71 on the 1st trials after the switch and 0.75 on the later trials. As in the easy–hard visual search task, FEF neurons selected the saccade goal, not the oddball of the search array.
|
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
; Malkovic and Nakayama 1994; McPeek et al. 1999). Nevertheless, the results from the 2 different tasks were remarkably similar. FEF neurons exhibited selection for the saccade goal, not the oddball stimulus in the search array, and the magnitude of the selection before saccades was proportional to the probability of making each choice. These results are consistent with the view that FEF activity forms a salience map: an integration of bottom-up and top-down influences in a topographic map with multiple peaks of activity with the highest activity peak specifying the goal of the next saccade (reviewed in Thompson et al. 2001). Relationship to saccade production
The present results strengthen the argument that FEF is involved in perceptual decision processes leading to a saccade decision rather than only preparing a saccade to a goal chosen elsewhere in the brain, such as in LIP (e.g., Roitman and Shadlen 2002). The evidence for this is that the magnitude of activity depends on both the visual similarity of objects in the visual scene (easy–hard search task), and on top-down knowledge based on previous trials (feature-switching search task). Also, the magnitude of presaccadic selection was correlated with the probability of the monkeys' decision and not with the metrics of the saccades. This is consistent with previous work that showed that visually responsive neurons in FEF are more related to visual processing than to saccade production (Hanes et al. 1998; Murthy et al. 2001; Sato and Schall 2003; Thompson et al. 1996, 1997).
The same pattern of activity in movement neurons as visual neurons corroborates theories of information processing that propose that information flows continuously from visual selection processes to movement preparation processes (Bichot et al. 2001a; Eriksen and Schultz 1979). This was surprising because, although the saccades made on error trials were to visual stimuli located at the same positions as on correct trials, the presaccadic activation of movement neurons reached different levels across the different stimulus–response conditions (Fig. 10). This result was not expected because previous work has shown that the activity of movement neurons rises to a constant threshold before saccade initiation (Hanes and Schall 1996). There are 2 possible explanations for the difference in results that are still consistent with the rise-to-threshold hypothesis. First, a threshold was reached earlier than in the time measured in this study. In the easy–hard task movement neurons exhibited a similar level of activity across the different trial conditions at about 60 ms before saccade initiation at which time the activity diverged. However, Hanes and Schall (1996) also measured movement activity within the 30 ms before saccade initiation. An alternative explanation is that the saccade initiation threshold was different across the different trial conditions. In the previous study (Hanes and Schall 1996), the visual target stimulus was always presented alone and was the only possible saccade target. In this study the visual target was presented within the context of a search array and a decision of which stimulus to look at was required. It is possible that a threshold was reached, but the threshold was different for the different target–distractor combinations or it is possible that lower thresholds favored error trials. Resolving this apparent inconsistency will require further investigation.
A visual salience map in FEF
When viewing a scene, our attention and gaze are directed to conspicuous objects that stand out from the background. Visual conspicuity occurs when an object has a unique feature (e.g., color, motion) that sets it apart from the rest of the image. When an object is visually conspicuous, it captures attention through a bottom-up process. Bottom-up refers to the automatic, preattentive processing that occurs in a massively parallel manner across the entire visual field and is based exclusively on the properties of the image. In addition, an object will stand out when it matches the viewer's expectations, such as when searching for something familiar or for an object with a specific feature (e.g., Bichot and Schall 1999a; Motter and Belky 1998). In this case attention is guided through a top-down process. Top-down refers to selection based on cognitive factors such as the goals and knowledge. Usually, attention and gaze are guided by a combination of bottom-up and top-down influences (Yarbus 1967).
Models of attention and saccade target selection posit the existence of a 2-dimensional topographic "salience map" of the visual world in the brain that controls the deployment of covert attention and saccadic eye movements (Cave and Wolfe 1990; Findlay and Walker 1999; Itti and Koch 2001; Olshausen et al. 1993; Treisman 1988; Wolfe 1994). To effectively guide visual attention and eye movements in a complex world, it is essential that the activity on this map represent bottom-up physical salience regardless of what visual feature renders the salience, and top-down knowledge and expectations of the viewer. Therefore because of top-down influences, the most physically salient object may not necessarily be represented by the highest activity. A winner-take-all competition within this "salience map" gives rise to the most behaviorally relevant location, and specifies the goal for the next eye movement (reviewed in Itti and Koch 2001).
Behaviorally, an object's "salience" is indexed by the probability that it becomes the goal of the next saccade. During each fixation, there are multiple peaks of activity across the theoretical salience map. The behavioral relevance of any one object in the world depends not only on the magnitude of its representation on the map, but also on the other activity peaks across the map that represent competing objects. An object is more behaviorally relevant when its peak on the map is much greater than the other peaks and is less behaviorally relevant when the peaks are similar in magnitude. Therefore in a saccade choice task, the relative differences between the activity peaks on the salience map would be indexed by saccade probability.
We previously proposed that FEF serves as a salience map (reviewed in Thompson and Bichot 2004; Thompson et al. 2001). This view was based on the results of several different experiments that probed the visual selection process in FEF during visual search. Unlike in other areas of the visual system (e.g., Ogawa and Komatsu 2004), FEF neurons do not exhibit feature selectivity (Mohler et al. 1973; Schall et al. 1995); instead they exhibit selective activation that represents the spatial location of behaviorally relevant stimuli, whether that relevance is derived from bottom-up or from top-down factors (Bichot and Schall 1999b, 2002
