Visually guided movements can be inaccurate, especially if unexpected events occur while the movement is programmed. Often errors of gaze are corrected before external feedback can be processed. Evidence is presented from macaque monkey frontal eye field (FEF), a cortical area that selects visual targets, allocates attention, and programs saccadic eye movements, for a neural mechanism that can correct saccade errors before visual afferent or performance monitoring signals can register the error. Macaques performed visual search for a color singleton that unpredictably changed position in a circular array as in classic double-step experiments. Consequently, some saccades were directed in error to the original target location. These were followed frequently by unrewarded, corrective saccades to the final target location. We previously showed that visually responsive neurons represent the new target location even if gaze shifted errantly to the original target location. Now we show that the latency of corrective saccades is predicted by the timing of movement-related activity in the FEF. Preceding rapid corrective saccades, the movement-related activity of all neurons began before explicit error signals arise in the medial frontal cortex. The movement-related activity of many neurons began before visual feedback of the error was registered and that of a few neurons began before the error saccade was completed. Thus movement-related activity leading to rapid corrective saccades can be guided by an internal representation of the environment updated with a forward model of the error.
When confronted with unexpected events that render current actions inappropriate, primates can interrupt planning and program new responses. However, errors occur if the compensation is too slow, so the capacity to quickly correct errors is necessary for achieving goals. Evidence indicates that under certain circumstances errors can be corrected sooner than sensory feedback would permit (e.g., Bernstein et al. 1995; Cooke and Diggles 1984; Rabbit 1966). This ability is of interest because it involves an act of control to redirect overt movement. Visually guided saccades provide a useful domain in which to investigate such rapid error correction because, although saccades are regarded as ballistic movements under only intermittent control, many studies have provided evidence of gaze errors being corrected very rapidly, sometimes resulting in trajectories that are curved toward the target after initial errors (e.g., Becker and Jürgens 1979; Findlay and Harris 1984; McPeek and Keller 2001; McPeek et al. 2003; Minken et al. 1993; Port and Wurtz 2003; Van Gisbergen et al. 1987).
A critical node in the network controlling the programming of saccadic eye movements is the frontal eye field (FEF), a sensorimotor area that contributes to the selection of targets and preparation of saccadic eye movements (reviewed by Schall 2002; Schall et al. 2003; Thompson et al. 2001). In keeping with its sensorimotor function, a variety of cell types were previously described (e.g., Bruce and Goldberg 1985; Schall 1991) that specify where and when a saccade will be made. Although visual neurons appear to identify targets for saccades (e.g., Bichot and Schall 1999; Thompson et al. 1996), movement cells generate signals sufficient to control whether and when a saccade will be produced (Hanes and Schall 1996; Hanes et al. 1998). Even though much has been learned about how FEF contributes to the production of correct saccades, no studies have examined the role of FEF in producing saccades that correct errors. However, previous work identified neurons in the supplementary eye field and anterior cingulate cortex that signal when saccade errors are produced (Ito et al. 2003; Stuphorn et al. 2000).
When monkeys produce accurate corrective saccades with latencies less than the latency of visual feedback, these movements must be produced using a vector that is updated for the change of eye position produced by the error saccade. This problem has received much empirical and theoretical interest (e.g., Hallett and Lightstone 1976; Sparks and Mays 1983). One mechanism that can account for fast on-line error correction involves a comparison of the spatial location of the goal with the current eye displacement, for which a specific role for postsaccadic neurons in FEF was hypothesized (Goldberg and Bruce 1990). In this framework, error correction can begin only after the erroneous saccade has occurred. Alternatively, fast on-line error correction may involve a comparison of the spatial location of the goal with the anticipated eye displacement. Consequently, programming the corrective saccade can begin before the occurrence of the error and in parallel with the erroneous saccade.
Evidence in support of this latter hypothesis is derived from the finding of a characteristic modulation of some visual neurons in the FEF, lateral intraparietal cortex (LIP), and the superior colliculus (SC) that has been described as remapping (Duhamel et al. 1992; Umeno and Goldberg 1997, 2001; Walker et al. 1995; reviewed by Colby and Goldberg 1999). These neurons respond to stimuli that will be brought into their receptive fields by saccades, even before the saccades actually take place. More recently, McPeek and Keller (2002a) reported that the activity of visually responsive neurons in SC represent the location of the salient object in a search array if monkeys shift gaze to another location before producing a corrective saccade to the overlooked target; they suggested that the sustained representation by these visual neurons remapped the location of the target in the reference frame of the errant saccade to afford rapid error correction.
If remapping the target location enables the brain to rapidly and accurately correct saccade errors (Vaziri et al. 2006), then several implications of this hypothesis can be tested. First, parallel programming for rapid error correction occurs if the remapping is established early enough. Second, neurons that represent the target location as well as neurons associated with saccade programming will exhibit activation before afferent visual signals occur. Third, the timing of activation of neurons that represent the target location as well as the timing of activation of neurons associated with saccade programming must predict the time of initiation of the corrective saccade.
To investigate how remapped visual information before a saccade can be used rapidly to correct errors and improve visual search performance, we trained monkeys to perform a task that combines color singleton search with the classic double-step perturbation in which the target unpredictably changes location before the saccade is initiated. Monkeys were reinforced only for shifting gaze to the final target location, so the corrective saccades after errors were unreinforced. Reinforcing only the correct saccade as opposed to reinforcing after the second saccade results in faster corrective saccades (Ray et al. 2004). As a result, this search-step task yielded a reliably high fraction of errors, most of which were followed by corrective saccades. Thus the search-step task affords an ideal opportunity to elucidate the neural mechanisms responsible for generation of rapid corrective saccades.
Data were collected from three adult monkeys (two Macaca mulatta and one Macaca radiata) weighing 7–12 kg. The animals were cared for in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and the guidelines of the Vanderbilt Animal Care and Use Committee. Surgical and recording methods were previously described elsewhere (Schall et al. 1995). Experiments were under computer control using TEMPO/VIDEOSYNC software (Reflective Computing) that displayed visual stimuli (Sony Trinitron 500-PS monitor), delivered juice reward, and sampled eye position (250 Hz) and unit activity (1 kHz). A perturbation technique similar to the stop-signal (countermanding) task (e.g., Hanes and Schall 1995; Logan and Cowan 1984; Osman et al. 1990) was used to investigate gaze control during visual search. Our search-step task combines color singleton search with the classic double-step task (Becker and Jürgens 1979; Lisberger et al. 1975; Westheimer 1954; Wheeless et al. 1966) (Fig. 1A). The target and distractors were isoluminant red (CIE: x = 0.632, y = 0.340) or green (CIE: x = 0.279, y = 0.615) squares with luminance = 9.95 cd/m2 on a 2.0-cd/m2 background. On no-step trials monkeys were rewarded for making a saccade to a color singleton target among distractors. On random target-step trials the target and a distractor swapped locations through an isoluminant color change. In other words, the target stepped to a new location in the search array with a variable delay after presentation of the search array but before any saccade was produced. On these target-step trials monkeys were reinforced only if they canceled the partially programmed saccade to the original target location and directed gaze instead to the new target location. A correct gaze shift on target-step trials will be referred to as compensated saccade (others have called these “final angle responses”). However, monkeys often failed to compensate for the target step and instead made a saccade to the original target location; these errant noncompensated saccades (others have called these “initial angle responses”) were not rewarded. When monkeys produced errant noncompensated saccades to the original target location, they subsequently produced a corrective saccade to the new target location. These corrective saccades were never reinforced.
The probability of successfully compensating varied as a function of the delay between the initial presentation of the search array and the target stepping to a new location in the array. This target-step delay was adjusted using a staircase procedure to obtain an approximately equal number of correct (compensated) and error (noncompensated) step trials during each experimental session. Target-step delays typically ranged from 30 to 140 ms (Fig. 1B). For double-step trials, the target was presented alone and then stepped to one of the possible locations.
In the data analyzed for this study all sessions had 50% step trials. A restricted set of targets steps was used to increase the yield of data during the neurophysiological sessions. Targets could step to and from the three array positions centered around and opposite to the neuron's response field, yielding 18 possible combinations of target steps. Thus targets stepped into or out of response fields but never stepped within a response field. Target steps were randomized and were interleaved with no-step trials in which target position was randomized and equiprobable across all locations. Behavioral data indicated that the monkeys could predict neither the occurrence of a target-step trial nor the location of the stepped target.
Analysis of data for this study required matching the direction and amplitude of correct and corrective saccades. Therefore only those combinations of target steps were chosen that resulted in subsequent corrective saccades falling into the response field of the cell in question. This was accomplished as illustrated in Fig. 2. For example, the no-step activity of a neuron with the movement field toward 9 o'clock was compared with the activity on step trials in which the error was directed to 2 or 4 o'clock, followed by a horizontal corrective saccade toward 9 o'clock. If the movement field were larger, the number of permissible configurations increased. Given the circular geometry of the search array, the amplitude of the corrective saccades between two positions in the array must be longer than the amplitude of the saccade from the center to one of the positions. The longest corrective saccade across the array (being a diameter) will have twice the amplitude of the saccade from the center (being a radius). Also, a corrective saccade from, for example, the upper-right to the upper-left position (Fig. 2, top) will have an amplitude √2 times that of the saccade from the center. Insofar as possible, the comparisons equated saccade amplitude. Furthermore, these differences in saccade amplitude were much less than the size of the response fields of FEF neurons. Nevertheless, we report below the maximal difference in the neural activity for correct and corrective saccades that would be expected based on such amplitude differences.
Saccades were detected using criteria based on velocity and change of eye position. Eye position was digitized at 250 Hz and smoothed by a boxcar filter of 12-ms binwidth. The algorithm first searched for intervals in which radial eye velocity exceeded a criterion of 30°/s. Then, saccade initiation and termination were defined as the beginning and end of the monotonic change in eye position lasting ≥12 ms before and after the high-velocity gaze shift. Based on the 250-Hz sampling rate this method is accurate to within 4 ms because it identified the digitized eye-position sample at which the gaze shift began. The effectiveness of the algorithm can be evaluated in Fig. 6.
Single neurons were recorded from the FEF using standard electrophysiological recording techniques. Spike-density functions were constructed by convolving spike trains with a Gaussian distribution (SD = 10 ms). The time at which significant differential activity began across two conditions was defined as the instant when the difference between the two spike-density functions first exceeded 2 SDs beyond the mean difference in activity measured in the 800-ms interval preceding array presentation provided the difference ultimately reached 5 SDs and was maintained >2 SDs for ≥50 ms. A similar analysis was also carried out convolving spike trains with a function that resembled a postsynaptic potential (PSP) to generate the spike-density function. The time constant of the growth phase and the time constant of the decay phase of the function used for convolution were 1 and 20 ms, respectively.
In addition, we also used a receiver operating characteristic (ROC) analysis, based on signal detection theory (Green and Swets 1966), to further validate the results. Two sets of trials corresponding to correct no-step trials and corrective step trials were compared. We considered only those neurons that provided at least four trials in both sets. Spike trains from original sets of trials were bootstrapped to construct 5,000 simulated spike trains in each set for reliable comparison. The spike-density function corresponding to each trial was constructed by convolving with a Gaussian distribution function (SD = 10 ms).
Comparisons were conducted in nonoverlapping 1-ms bins starting 100 ms before the end of saccade to 200 ms after the end of the saccade. Points on the ROC curve were generated by plotting the fraction of step trials when the monkey made the erroneous saccade away from the response field followed by the corrective saccade into the response field with activity greater than a criterion as a function of the fraction of no-step trials when the monkey made the correct saccade away from the response field with activity greater than that of the criterion. The entire ROC curve was generated by incrementing the criterion from zero to the maximum discharge rate observed on a single trial in steps of 1 spike/s. The area under the ROC curve provides a quantitative measure of the separation between two distributions of activity. An area under the ROC curve value of 0.5 signifies that the two distributions being compared are completely overlapping (i.e., indistinguishable), whereas a maximum value of 1.0 signifies that the two distributions do not overlap at all (i.e., perfectly distinguishable). To describe the growth in the area under the ROC curve over time the data were fit with a cumulative Weibull distribution function. The time of differential activity was determined from the growth of the ROC area over time and was defined as the time when the ROC area reached a value of 0.75 (Thompson et al. 1996). To verify the efficiency of this method we repeated the same analysis constructing the spike-density function corresponding to each trial by convolving the spike train with a combination of exponentially increasing and decreasing functions that resembled a PSP. The time constant of growth phase and the time constant of decay phase of the function used for convolution were 1 and 20 ms, respectively. In 17 cases (seven with PSP filter and ten with Gaussian filter) the fits were not satisfactory, so the discrimination time was simply the time of the first ROC value that exceeded 0.75 and remained ≥0.75 for 6 ms. For seven cells for which ROCs were obtained with four or five trials, we verified the accuracy of the estimate of the discrimination time by repeating the bootstrap procedure three times for each cell. We found that the maximum error in the estimation was 3 ms.
Our application of the double-step task during visual search is designed to probe how the oculomotor system responds to new visual information before saccades are initiated. This motivation is somewhat distinct from that of certain other previous uses of the double-step task, such as investigating coordinate transformations by requiring subjects to execute a sequence of saccades to the successive target locations (e.g., Hallett and Lightstone 1976; Umeno and Goldberg 1997, 2001). The reward contingencies of our task required monkeys to shift gaze directly to the final location. The reward contingencies of these other double-step saccade studies required human and monkeys to shift gaze to each target location in succession. In fact, we previously studied human performance under both instructions and found that corrective saccades have shorter latencies than those of correct saccades and parallel programming of saccades is facilitated when the second saccade was correcting an error instead of finishing a sequence (Ray et al. 2004).
Visually evoked activity was distinguished from movement-related activity through established criteria (Hikosaka and Wurtz 1983). In the conventional memory-guided saccade task visual neurons exhibited increased spike rate after stimulus presentation but no buildup of activity preceding gaze shifts (Fig. 5A). In contrast, movement-related neurons exhibited a pronounced increase of discharge rate before and during the memory-guided saccade (Fig. 5B). Some movement-related neurons had very weak or absent visual responses; these were classified as movement neurons. The movement-related neurons with pronounced visual responses were classified as visuomovement neurons.
To quantify the relative magnitude of visual and movement activity in the memory-guided saccade task, a visual-movement index (VMI) was calculated for each neuron. Visual activity (VA) was defined as the mean firing rate above the spontaneous activity of the neuron in a time window 50 to 200 ms after stimulus onset. Movement activity (MA) was defined as the mean firing rate above the same spontaneous activity of a time window 100 to 50 ms before saccade onset. The spontaneous activity was measured as the mean firing rate in a span of 800 to 400 ms before the stimulus onset. VMI was calculated as VMI = (MA − VA)/(VA ± MA). If movement activity was less than spontaneous activity, VMI was normalized to −1; similarly, if visual activity was less than spontaneous activity, VMI was normalized to +1. Therefore neurons with comparatively higher movement activity yield positive VMI and neurons with greater visual activity yield negative VMI.
Behavioral evidence for parallel programming
Evidence for parallel programming of successive saccades was obtained from an analysis of the interval between the error and corrective saccade relative to the target-step delay (Fig. 3). The logic of this analysis applies to saccade programming (e.g., Becker and Jürgens 1979) and, more generally, to the characterization of the psychological refractory period (e.g., Pashler 1994). The key measure is the interval between the target step and the beginning of the noncompensated error saccade; this will be denoted as reprocessing time (RPT; identical to D of Becker and Jürgens 1979). The label of reprocessing time for this interval is intended to convey the simple idea that until the saccade is initiated, the visual system can reprocess the search array that has changed; once gaze shifts, visual processing is disrupted. The premise of this analysis is that if saccade programming is strictly serial—that is, if programming the corrective saccade to the new target location can begin only some time after the errant saccade to the old target location has been executed—then the intersaccadic interval (ISI) cannot decrease with increasing reprocessing time (Fig. 3A, left, dotted lines in Fig. 3, B and C). However, if the corrective saccade can be programmed before the visual consequences of the errant saccade have been encoded or even before the errant saccade has been executed, then the ISI will be inversely related to reprocessing time (Fig. 3A, right, solid lines in Fig. 3, B and C).
Figure 3D plots ISI as a function of reprocessing time from a typical search-step session. This scatterplot appears to be composed of two distributions: shorter ISIs that decrease with increasing RPT (black) and the longest ISIs that do not change with RPT (grey). To quantify whether ISI varied significantly with reprocessing time, only ISIs <95th percentile latency of saccades produced in no-step trials were analyzed (no-step distribution shown on right in Fig. 3D); this criterion removed ISIs that were longer than a typical saccade latency. In other words, the saccades with latencies long enough to be initiated after complete visual processing of the new image are obviously produced through serial processing. For the session illustrated, 75% of all the corrective saccades were produced with latencies <95th percentile latency (272 ms) of saccades produced in no-step trials. For the corrective saccades with the shortest latencies, the ISI was negatively correlated with reprocessing time (slope = −0.79, r2 = 0.23, n = 321, P < 0.001). This was observed in 43 of the 44 data collection sessions in which visuomovement and movement cells were recorded (Fig. 3E; slope median = −0.79, min = −1.58, max = −0.51, r2 median = 0.15, min = 0.001, max = 0.48; data from one session showed positive slope).
We also collected data in 14 double-step sessions in which the target was presented with no distractors. On a random 50% of trials the target stepped to a new location before the saccade was initiated. We found the same negative correlation between ISI and RPT (Fig. 4; slope median = −0.68, min = −1.32, max = 0.12, r2 median = 0.10, min = 0.001, max = 0.43). Thus the monkeys exhibited the same pattern of parallel saccade programming here as in the search-step task. Having established that corrective saccades were produced rapidly after errant saccades, we can now consider the neural basis of this rapid error correction.
The goal of this experiment was to determine whether the timing of corrective saccade production could be explained by the timing of activity in FEF contributing to programming the saccade after the error saccade. If monkeys could produce corrective saccades earlier than visual feedback about the error could be encoded, then the visual and movement-related neurons should modulate earlier than a typical ISI after execution of the error (Fig. 3C, solid line). If programming corrective saccades can begin only after visual processing of the new image is finished, then the movement-related neurons should increase their activity later and at a time that does not vary with reprocessing time.
Cell classification: contrasting visually evoked and movement-related activity
Every neurophysiological investigation of FEF in macaque monkeys has agreed that the FEF contains diverse types of neurons with most exhibiting visual responses and some exhibiting modulation associated with saccade production (Bruce and Goldberg 1985; DiCarlo and Maunsell 2005; Hanes et al. 1998; Helminski and Segraves 2003; Schall 1991; Segraves and Goldberg 1987; Sommer and Wurtz 2001). In addition to their patterns of modulation, the relative magnitude of visual and movement activity in the memory-guided saccade task was quantified using a visual-movement index (VMI), which was calculated for each neuron as VMI = (MA − VA)/(VA ± MA). Neurons with comparatively higher movement activity yield positive VMI and neurons with greater visual activity yield negative VMI. Accordingly, the average VMIs (±SE) for movement and visual neurons were 0.4 (±0.09) and −0.06 (±0.09), respectively, which were significantly different from each other [t = 3.5, P < 0.001, degree of freedom (df) = 32], whereas the average VMI of visuomovement cells was −0.09 (±0.11). This report contrasts the pattern of activity of 15 visual neurons (13 from monkey C and two from monkey F) with that of 20 movement (15 from C, three from F, and two from L) and 24 visuomovement (21 from F and three from L) neurons.
Neurons with and without movement-related activity also exhibited different patterns of modulation when corrective saccades were produced. Compared with when a distractor remains in the receptive field, when the target steps into the receptive field, visual neurons select the new location of the target (Fig. 5C) (Murthy et al. 2001). For the representative visual neuron, target selection occurred 193 ms after array presentation and persisted until the corrective saccade was produced. Thus visual neurons in FEF maintained a correct representation of the changing image in spite of an initial errant gaze shift. This activity was previously described as a remapping signal when observed in the superior colliculus (McPeek and Keller 2002b). However, although the visual representation of the location of the stimulus is a necessary preliminary to error correction, the movement-related activity immediately preceding the corrective saccade is a more proximal basis of saccade preparation. We now show that neurons with movement-related activity producing a saccade to the final target location became active very soon after the error (Fig. 5D).
Analysis of saccade-related activity
To compare saccade-related activity specific to the programming of corrective saccades we ensured that comparisons were made between only correct and corrective saccades with similar directions and amplitudes (Fig. 2). Figure 6 illustrates the pattern of activity of a typical FEF movement-related neuron; behavioral data associated with this neuron are shown in Figs. 1, 3, and 4. Before addressing the question of how rapid saccade corrections occur, we determined whether the timing or magnitude of movement-related activity preceding corrective saccades (Fig. 6B) was different from that preceding correct saccades (Fig. 6A). Neural activation during correct no-step responses into the movement field was compared with the subset of errant step trials in which the corrective saccades most closely matched the direction and amplitude of correct saccades made into the movement field during no-step trials (Fig. 6C, inset). To ensure the most valid comparison between corrective and correct activity, trials were also matched for saccade latency.
Previous work showed that saccades are initiated when movement-related activity in FEF reaches a threshold that does not vary with saccade latency (Hanes and Schall 1996). Therefore the average spike rate 10–20 ms before initiation of correct and corrective saccades was compared. Although for the neuron shown in Fig. 6C the magnitude of activity for correct saccades was slightly less than that for corrective saccades, across the sample correct and corrective saccades were initiated when the level of movement-related activity in FEF reached a single discharge rate threshold. The mean (±SE) difference between threshold discharge rates before correct and corrective saccades was 7.2 ± 3.7 spikes/s, which was not significantly different from 0 (t = 1.93, P = 0.06, df = 42, power = 0.35).
It is well known that the timing and magnitude of movement-related activity vary with the location of a saccade relative to the most sensitive point of a neuron's movement field. Across the sample of movement-related neurons the mean amplitude of corrective saccades was 13.3° and that of correct saccades was 9.73°, with the mean absolute value of the difference in amplitudes across types of saccades being 3.55°. This difference in amplitude corresponds to that expected from the geometry of the circular search array (9.73° × √2 = 13.8°). Thus the amplitude of corrective saccades tended to exceed the amplitude of correct saccades, so some difference in the timing or magnitude of activity associated with corrective and correct saccades into a neuron's movement field may arise from variability in the location of the saccade endpoints relative to the optimal location in the movement field. Consideration of the size of FEF movement fields mitigates this concern. The movement fields of FEF neurons can be characterized as Gaussian in direction and log-Gaussian in amplitude (Bruce and Goldberg 1985). Although we did not obtain amplitude tuning for the neurons in this sample, maps of saccade direction tuning obtained in this laboratory (Schall et al. 1995, 2004) quantitatively match the original report of Bruce and Goldberg (1985). Given the typical amplitude tuning of FEF movement neurons at the eccentricity of the neurons sampled (about 10°), a 3.55° difference in saccade amplitude would translate into no more than a 3–9% difference in discharge rate. Therefore any difference larger than roughly 10% in magnitude and timing of activity associated with correct and corrective saccades cannot be explained entirely by differences in saccade amplitude.
Analysis of activity before corrective saccades
To determine whether the activity of FEF movement-related neurons can account for the short latency of corrective saccades, the activity associated with corrective saccades into the movement field after noncompensated errors outside of the movement field (Fig. 6, B, C, E, and G) was compared with the activity associated with correct no-step saccades directed to the same location outside of the movement field (Fig. 6, D and G). Trials were matched for saccade latency and direction so that any difference in activity between these trials must arise from the process of producing the corrective saccade.
To match the direction with the correct saccade to the response field the equivalent corrective saccade was first translated to the origin of the coordinate of the display screen and then rotated clockwise by the angle subtended by the corresponding response field to the positive x-axis. If the endpoint of the saccade did not fall within ±5° the trial was discarded. A subset of correct saccades in no-step trials whose latency fell within the span of latency of noncompensated saccades of same direction in the step trials was examined.
As illustrated in Fig. 6G, the activity of this neuron producing the corrective saccade began 29 ms before the end of the error saccade. To validate this result we also determined the time of differential activity by performing ROC analyses, based on the theory of signal detection (Green and Swets 1966; Thompson et al. 1996). Based on this analysis the activity of this neuron producing the corrective saccade began 6 ms before the end of the error saccade (Fig. 6F). Qualitatively similar modulation was observed in double-step trials (Fig. 7). Because there were no distractors, these double-step data rule out the possibility that the early movement-related activity observed in the search-step trials was just the response to the distractor appearing in the response field before the target step.
To determine whether the modulation was specific to the programming of the corrective saccade to a particular location or represented a more general, nonspatial error signal, the activity associated with corrective movements was contrasted between saccades into, beside, and away from the movement field (Fig. 8). This comparison confirms the spatially selective nature of the activity preceding corrective saccades. The same result was obtained across the population. The average spike density from 100 ms before the error to 100 ms after the error for corrections directed toward and away the response field were contrasted. The discharge rate was significantly higher (paired t-test, t = 8.0, df = 56, P < 0.001) when the correction was into (means ± SE = 50.6 ± 4.1 spikes/s) the neuron's movement field compared with when it was away (means ± SE = 21.8 ± 2.6 spikes/s) from the movement field.
Figure 9 plots the times when the sample of visual, visuomovement, and movement neurons became active before corrective saccades relative to initial array presentation and relative to the end of the error saccade. For visual neurons with no movement-related modulation, the start of this activation marked the earliest time when the new location of the stepped target was encoded (Fig. 5C). For neurons with movement-related activity, the start of this activation marked the beginning of programming of the corrective saccade (Fig. 5D). The new target location was selected by visual neurons 227 (median 214) ± 71 ms after presentation of the search array, by visuomovement neurons 252 (median 262) ± 46 ms, and by movement neurons 249 (median 242) ± 39 ms after array presentation. Activation times for visual, visuomovement, and movement neurons were not significantly different (one-way ANOVA, F = 0.96, df = 53, P = 0.39).
The time of modulation measured relative to the time of the error saccade provides the main evidence demonstrating the early programming of the corrective saccade. As illustrated in Fig. 9, neurons with only visual responses changed discharge rate on average 21 (median: −33) ± 72 ms before termination of the noncompensated error. Visuomovement neurons changed discharge rate on average 17 (median 16.5) ± 46 ms after termination of the noncompensated error. Movement neurons changed discharge rate on average 7 (median 1) ± 39 ms after termination of the noncompensated error. These times were not significantly different from one another (one-way ANOVA, F = 1.97, df = 52, P = 0.15). The critical new result is the time when movement-related activity leading to the corrective saccade began. Given the typical 50-ms duration for 10° saccades, the movement-related activity of some neurons leading to the corrective saccade began before the error saccade was initiated; significant modulation was exhibited by 41% of movement-related neurons before the error saccade was completed. Another 39% of movement-related neurons became active after the error saccade but before the earliest afferent delay measured in FEF (40 ms; Schmolesky et al. 1998), ruling out visual feedback for guidance of the rapid corrective saccade. Furthermore, almost all of the movement-related neurons became active before the modal value of the latency of explicit error signals observed in the supplementary eye fields and the anterior cingulate cortex during the closely related stop-signal task (Ito et al. 2003; Stuphorn et al. 2000). Thus FEF movement-related activity leading to corrective saccades usually began before errors could be detected through visual or monitoring feedback.
We wanted to ensure that the conclusions drawn from this analysis were not susceptible to particularities of the measurement procedure. Therefore we repeated the analysis using different methods and criteria. First, we determined the time when the modulation preceding corrective saccades occurred based on an ROC analysis (Thompson et al. 1996). The results of this ROC approach revealed qualitatively indistinguishable results using either the Gaussian or the PSP convolution kernels. The area under the ROC curve increased with the time measured from the end of erroneous saccade, indicating that the corrective activity evolved over time with increasing discriminability from the activity corresponding to the correct saccade. To estimate the time course of this correction process we fit the plot of area under the ROC with a Weibull function. The instant in time when the fit reached 0.75 was considered as the beginning of the modulation (Fig. 6F). Figure 10 shows the result from this ROC analysis across the population of neurons. Using a Gaussian (or PSP) convolution function, we found 32% (10%) of movement neurons, 15% (5%) of visuomovement neurons, and 67% (40%) of visual neurons exhibited significant modulation before the erroneous saccade was completed and another 37% (53%) of movement neurons, 25% (27%) of visuomovement neurons, and 33% (30%) of visual neurons became active after the error saccade but before the earliest visual efferent delay to FEF of 40 ms. The foregoing analysis was accomplished using measurements from average spike-density functions. Because the choice of the kernel used for the spike-density function can affect measurements of timing of modulation, a different analysis was also done that directly detects the moment of modulation according to spike times. A Poisson spike-train analysis was applied to measure the beginning of activation on single trials as described and demonstrated previously (Hanes et al. 1995; Pouget et al. 2005; Schmolesky et al. 1998; Thompson et al. 1996). How rapidly the correction began was determined by taking the median of the earliest periods of beginning of activation for each neuron across the trials (Fig. 6E, arrow). By this measure, neurons with only visual responses changed their discharge rates on average (±SE) 39.7 ± 20.2 ms after termination of the noncompensated error. Visuomovement neurons changed discharge rate on average 59.9 ± 12.5 ms after termination of the noncompensated error. Movement neurons changed discharge rate on average 63.9 ± 14.2 ms after termination of the noncompensated error. These times were not significantly different from one another (F = 0.6, df = 56, P = 0.55). According to this trial-by-trial measure, modulation in activity leading to the corrective saccade began before error saccade termination in 20% of movement neurons, before the earliest afferent delay measured in FEF (40 ms; Schmolesky et al. 1998) for another 10% of movement neurons and before the common latency of explicit error signals observed in the supplementary eye fields and the anterior cingulate cortex during the stop-signal task (Ito et al. 2003; Stuphorn et al. 2000) for another 35% of movement neurons. A one-way ANOVA indicated that there was no difference in the time of the beginning of corrective activity of movement-related neurons (movement and visuomovement) measured by all three techniques (Fig. 10) (F = 0.97, df = 116, P = 0.38). Thus the general conclusion that activity in FEF producing corrective saccades begins earlier than error or visual feedback and sometimes even before the error is completed does not depend on the use of a particular form of analysis.
Relation of neural activity to reprocessing time
A further test of the role of movement-related activity to saccade generation is afforded by the tendency of the latency of the corrective saccade relative to the error saccade (ISI) to decrease with the latency of the error saccade relative to the target step (reprocessing time) (Figs. 3 and 4). Specifically, if movement-related activity in FEF contributes to programming the corrective saccade, that activity must begin earlier after longer reprocessing times. In contrast, if the onset of the movement-related activity arose from error feedback of some kind, then it should depend on the timing of the errant saccade alone and not reprocessing time. To evaluate these alternatives, corrective saccades were divided into those associated with the shortest reprocessing times (less than the mean reprocessing time) and those associated with longest reprocessing times (greater than the mean reprocessing time). The result of this analysis is shown in Fig. 11 for the movement neuron illustrated in Fig. 6. Figure 11A shows that corrective activity began earlier before errors associated with longer reprocessing times (and shorter ISIs) and began later before errors after shorter reprocessing times (and longer ISIs). To quantify this relationship, the average start time of corrective activity was related to the average reprocessing time from the groups of shorter and longer reprocessing time trials. For this neuron the two were inversely related (Fig. 11B; slope = −0.89), consistent with the hypothesis that the timing of the corrective saccade is dictated by the timing of movement-related activity.
This analysis was performed for movement-related neurons that provided sufficient and reliable data for both long and short reprocessing times during the search-step task. This sample exhibited a significant inverse relationship between the beginning of corrective activity and reprocessing time; the average slope of −0.72 was significantly <0 (one-tailed t-test, t = −5.1, df = 35, P < 0.001) with 81% (29/36) of neurons exhibiting negative slopes (Fig. 12). Furthermore, the mean time of corrective neural activity was inversely related to the mean reprocessing time in each search-step session (slope = −0.8, r2 = 0.16; F = 7.9, P = 0.007, df = 43). Only seven double-step sessions provided enough data to contrast activity on trials with short and long reprocessing times. This inverse relationship was observed in five of seven double-step sessions, but the population trend was not significant.
The relation between movement-related activity and the timing of corrective activity was further tested by considering only those corrective saccades that occurred at the fastest ISIs (<5th percentile of the latencies of single correct saccades) because these were most likely to result from parallel processing. In addition, we performed the same analyses for those trials producing the longest ISIs (>95th percentile of latencies of single correct saccade; grey points in Fig. 3) because these were most likely to result from serial processing. As expected, we found that the time of differential activation for the cell shown in Fig. 6 occurred at 73 ms after the error for trials with longer ISIs as opposed to 29 ms before the error for trials associated with shorter ISIs. Consistent with the hypothesis that the second corrective saccade is programmed in parallel with the first erroneous saccade we observed that the differential activity was significantly sooner for the faster corrective saccades across the population of movement-related cells (paired t-test, t = 2.7, df = 9, P < 0.025). This provides additional evidence that the latency of the corrective saccades depends on the timing of the movement-related activity in FEF.
We show for the first time that visual and movement-related neurons in FEF are active before saccades made to correct quickly errant gaze shifts. Corrective saccades can be produced quickly because visual neurons establish and maintain a representation of the location of the salient target even when gaze does not initially shift there (McPeek and Keller 2002b; Murthy et al. 2001). This study provides new information about the next process in error correction: the production of the corrective saccade. We show now that the movement activity preceding corrective saccades can begin before the error can be detected by afferent visual processing or error monitoring and, furthermore, that the movement-related activity specifies when corrective saccades are initiated. These observations provide the most direct evidence to date for a neural mechanism of rapid error correction.
The interpretation of the results hinges on the distinction between visually evoked and movement-related activity. We do not take for granted the complexity of identifying neural activity with cognitive processes (Schall 2004), but we do not see how some such identification can be avoided. Numerous investigators described visual, visuomovement, and movement neurons in FEF (Bruce and Goldberg 1985; DiCarlo and Maunsell 2005; Helminski and Segraves 2003; Schall 1991; Segraves and Goldberg 1987; Sommer and Wurtz 2001; Umeno and Goldberg 1997, 2001) and SC (Horwitz and Newsome 1999; Mays and Sparks 1980; McPeek and Keller 2002b) and identified visual neurons in FEF and SC with visual processing but not saccade programming (Horwitz and Newsome 1999; Horwitz et al. 2004; McPeek and Keller 2002a; Sato and Schall 2003; Sato et al. 2001; Thompson et al. 1996, 1997, 2001). Data from the stop-signal task show that saccade-related but not visual activity in FEF and SC can be identified with saccade programming that specifies whether and when saccades will be initiated (Hanes et al. 1998; Paré and Hanes 2003). Further evidence for distinguishing neuron types is differential anatomical connectivity. It is well known, for example, that neurons in different layers of the cortex entertain different afferents and efferent targets and, consequently, have more or less distinct functional properties. Combined recording and electrical stimulation studies have demonstrated that neurons in FEF with movement-related activity project to the SC (Segraves and Goldberg 1987; Sommer and Wurtz 2001) and brain stem (Segraves 1992). Therefore such neurons correspond to pyramidal cells in layer 5 (e.g., Fries 1984). It is less clear whether neurons with exclusive visual responses project to the SC; one study obtained negative evidence (Segraves and Goldberg 1987) and another, positive evidence (Sommer and Wurtz 2001). Certainly, neurons in the supragranular layers of FEF that do not project to the SC subserve visual target selection (Thompson et al. 1996). Thus the weight of the evidence seems to us to support the distinction between visually responsive and movement-related neurons. Ultimately, the adaptability and arbitrariness of macaque and human behavior cannot be explained without a distinction between sensory and motor processes.
Updating and remapping
The original evidence for the remapping hypothesis was that some visually responsive neurons in visuomotor structures, including FEF (Umeno and Goldberg 1997), respond to a stimulus that is not currently in the response field but will be after an upcoming saccade. The present data extend this observation in several critical ways. First, Umeno and Goldberg (1997) reported that 31% of visuomovement and 0% of movement cells showed predictive receptive field shifts. However, we found that 80% of visual neurons selected the new target on error trials and 100% of movement-related neurons demonstrated significant activity before the corrective saccade. Thus assuming equal sampling and classification criteria across studies, the extent of modulation we observed exceeds what has been reported for remapping and likely represents a different phenomenon or a more potent expression of the same phenomenon. Therefore we do believe it is useful to maintain a distinction between the process of “remapping” visual representations (presumably for perceptual stability) and the process of producing accurate saccades. Such a distinction is warranted because of the disparities that can occur between perceptual localization and saccade endpoints (Matin and Matin 1972).
It should also be noted that the task conditions we used are qualitatively different from those used in the original studies describing remapping. Our search- and double-step tasks challenged monkeys to suppress one saccade to make the correct saccade in step trials. In contrast, the remapping studies either do not permit the monkey to make a second saccade or the two targets appear well before the first saccade is initiated. As a result, in the remapping studies monkeys never made saccades that were optimal for the movement cell, so it may not be surprising that none of the movement cells was activated. In testing with human subjects we found that performance of double-step saccades depends on instructions; second (corrective) saccades to the final target location occur significantly more rapidly if subjects are instructed to cancel the initial saccade and redirect gaze to the final target location as opposed to being instructed to follow the successive targets (Ray et al. 2004).
Second, the original results suggest that a remapping signal would be synchronized on the error saccade that remaps the image, but we found that the onset of movement-related activity during search-step trials was inversely related to reprocessing time, the interval between the target step and the error saccade. Thus whereas the activity of visually responsive neurons in FEF may be described as spatial remapping, the activity of the movement-related neurons invites an interpretation in terms of saccade programming. It could be argued that the activity of movement neurons is simply driven by visual neurons. However, overwhelming evidence from cognitive psychology (e.g., Kornblum at al. 1990) as well as several experiments from our laboratory (Juan et al. 2004; Sato and Schall 2003; Sato et al. 2001) demonstrate conclusively that visual activity does not directly and immediately drive movement activity. In the final analysis, the process of remapping to maintain perceptual stability and the process of rapid error correction are more complementary than exclusive. In whatever manner the brain updates its representation of visual coordinates according to changing eye position, the new results of this study demonstrate that it must occur by the time that FEF movement-related activity begins.
Rapid error correction
The inference of parallel programming of saccades was made based on unusually brief intersaccadic intervals observed when subjects correct errors in the context of rapidly changing or alternative saccade endpoints (e.g., Becker and Jürgens 1979; Goossens and Van Opstal 1997; Hooge and Erkelens 1996; McPeek and Keller 2002a; McPeek et al. 2000; Theeuwes et al. 1999; Viviani and Swensson 1982). ISIs ≲150 ms are generally regarded as evidence for parallel programming because the second saccade is produced before visual input can be processed. The present data included some but not a large number of such brief ISIs, probably resulting from particular features of this search-step task. In particular, the isoluminant color change in the search-step task is a weaker stimulus than the transient stimuli used in most double-step saccade studies. Also, the corrective saccade was never rewarded, so there was no urgency for the monkeys to redirect gaze. Finally, adjusting the target-step delay in a staircase procedure prevented monkeys from succeeding on many trials. In another study, we found that strategic control can be exerted on the production of the sequence of saccades in this task (Ray et al. 2004). Therefore we believe the monkeys deliberately slowed saccade production to maximize reward earnings.
Nevertheless, equally conclusive evidence for parallel programming is an inverse relationship between the initiation of the first saccade and the reprocessing time (Becker and Jürgens 1979). This inverse relationship was observed in nearly all of the search-step and double-step data-acquisition sessions. Therefore despite longer ISIs, the behavioral data are entirely consistent with the criteria previously used for rapid, parallel saccade programming.
Error and conflict detection in particular and executive control in general have been very active areas of inquiry in cognitive neuroscience for the last decade (e.g., Botvinick et al. 2004). However, in spite of a great deal of work on error detection, relatively little is known about the neural mechanisms of error correction. Recently, several papers described the pattern of neural activity in the superior colliculus concomitant with saccades having curved trajectories (McPeek and Keller 2003; Port and Wurtz 2003; Walton et al. 2005). The present results extend these earlier studies in at least four ways. Activity associated with the first (error) and second (corrective) saccades could be more clearly distinguished because we looked specifically at saccades requiring activation in the hemisphere opposite that which produced the first saccade. Second, we monitored both visually evoked and movement-related activity preceding the corrective saccade being made into the response field. Third, we demonstrate for the first time the temporal correlation between the movement-related activity and the initiation of the corrective saccade. Fourth, our results demonstrate in the frontal lobe the kinds of dynamic signals contributing to gaze control that heretofore were reported only in the superior colliculus.
As mentioned, previous studies showed how the endpoint of the corrective saccade is maintained in the activity of visually responsive neurons in FEF and SC (McPeek and Keller 2002b; Murthy et al. 2001). However, the results of these studies cannot be said to constitute direct evidence for concurrent saccade programming because neither analyzed the movement-related activity producing the corrective saccade. This was the goal of the present report.
We measured when movement-related activity began before the corrective saccade into the movement field of the neurons. The most compelling evidence for concurrent saccade programming would be whether the movement-related activity preceding the corrective saccade began before the error saccade was initiated; a few neurons did this. Equally compelling evidence would be whether the movement-related activity preceding the corrective saccade began before the error saccade was terminated, which more neurons did. Effectively all of the movement-related neurons programming the corrective saccade became active within 100 ms after the error saccade was terminated. The mean visual latency in FEF is about 70 ms (Pouget et al. 2005; Schmolesky et al. 1998; Thompson et al. 1996) and the mean latency of error signals in the medial frontal lobe is 110–180 ms (Ito et al. 2003; Stuphorn et al. 2000). Thus our primary new observation is that activity producing corrective saccades can begin rarely before the errant saccade is accomplished, often before postsaccadic visual input about the error can be registered, and almost always before the brain registers that an error was made through endogenous error monitoring. This constitutes the first clear physiological evidence for how corrective saccades can be produced faster than serial processing would permit. Further compelling evidence that this activity contributed to concurrent saccade programming was the systematic relationship between the time of the neural activation and the reprocessing time.
This work was supported by Robin and Richard Patton through the E. Bronson Ingram Chair in Neuroscience and National Institutes of Health Grants RO1-EY-08890, F32-EY-14502, T32-EY-08126, P30-EY-08126, and P30-HD-015052.
We thank L. Boucher, M. Leslie, M. Mebane, P. Pouget, V. Stuphorn, and G. Woodman for comments; S. Hoffman for assistance with TEMPO; G. Fox and C. Wiley for manuscript preparation; and M. Feurtado for help with animal care.
↵* These authors contributed equally to this work.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2007 by the American Physiological Society