The localization of peri-saccadically flashed objects shows two types of errors: first, a uniform shift in saccade direction, and second, a compression of visual space around the saccade target. Whereas the uniform shift occurs when the experiment is performed in complete darkness compression occurs when additional visual references are available. Thus peri-saccadic mislocalization contains motor and visual components. To distinguish between both factors we compared peri-saccadic localization errors during pro- and anti-saccades. In the case of anti-saccades, the visual cue that elicits the saccade and the actual eye movement are in opposite directions. We asked whether peri-saccadic compression can be observed with anti-saccades, and if so, whether the compression is directed toward the visual cue or follows the actual eye movement. In blocked trials, subjects performed saccades either toward a visual cue (pro-saccade) or to the mirrored position opposite to a visual cue (anti-saccade). Peri-saccadically, we flashed a thin vertical bar at one of four possible locations. Subjects had to indicate the perceived position of the bar with a mouse pointer about 500 ms after the saccade. Experiments were performed in complete darkness and with visual references. Peri-saccadic mislocalizations occurred during anti-saccades. The mislocalizations were very similar for pro- and anti-saccades in magnitude and direction. For both, pro- and anti-saccades, mislocalizations were directed toward the actual eye movement and not the visual cue.
About three times per second, saccadic eye movements direct our gaze rapidly to a new direction in the environment. Every saccade causes a fast and massive displacement of the image on the retinae. Yet, we normally neither perceive motion of the retinal image nor a shift in the position of the retinal image. Rather, the world appears stable, and we are not even aware that we performed a saccade. This phenomenon traditionally referred to as space constancy or visual stability allows us to visually explore our environment without misattributing self-induced image motion to motion of the world. How does the visual system achieve stability despite the drastic retinal changes that occur during saccades?
Researchers have discussed both extraretinal and visual factors as responsible for the maintenance of visual stability. For instance, von Holst and Mittelstaedt (1950) and Sperry (1950) suggested that extraretinal information, such as an efference copy or a corollary discharge, is used to maintain visual stability. On the other hand, Gibson (1966) argued that the visual information in the retinal image is entirely sufficient to maintain visual stability. More recently, another theory for a visual origin of transsaccadic stability has been proposed (Currie et al. 2000; Deubel et al. 1996, 1998; McConkie and Currie 1996). In contrast to Gibson's theory, which argued that the entire visual image is exploited, this theory ascribes a special role to the saccade target. It suggests that only information about position and properties of the saccade target are stored across a saccade and that this information is used to re-calibrate the visual scene after the saccade.
To probe the mechanisms of visual stability, researchers have exploited several perceptual disturbances that occur at the time of saccades (for a review, see Ross et al. 2001). Two presumably related, but different, effects can be observed. The first is a suppression of visual sensitivity (saccadic suppression), which raises thresholds for detection of stimuli during a saccade. The second is a perceptual distortion of visual space for objects that are briefly presented before, during, or after a saccade (spatial distortion). Presumably, these distortions arise from processes that transform the presaccadic spatial representation of the scene into the postsaccadic spatial representation. An analysis of the mislocalizations therefore may give insight into these processes. Specifically, the parameter dependency of the mislocalization allows us to investigate what kind of information the visual system uses to achieve a stable percept of the environment. The peri-saccadic mislocalization can be further divided into two different effects. The first effect is characterized by a mislocalization pattern that is independent of the spatial position at which the object was flashed. Usually, these mislocalizations start about 100–50 ms before saccade onset, reach a maximum when the eyes start to move, and are directed along the axis of the saccade. Before and during the first half of the saccade, the error is in the direction of the saccade. Later, during the saccade, the error reverses direction, and mislocalizations are opposite to saccade direction (e.g., Dassonville et al. 1995; Honda 1991; Schlag and Schlag-Rey 1995). Because the mislocalization at any given moment in time is uniform in direction and magnitude at all presentation positions, we call this type of mislocalization a uniform shift. A second type of error consists of a nonuniform mislocalization across the visual field, i.e., a mislocalization that depends on the spatial position of the flashed object (e.g., Bischof and Kramer 1968; Honda 1995; Matin and Pearce 1965). It resembles a compression of visual space around the saccade target (Morrone et al. 1997; Ross et al. 1997). Previously, we showed that the pattern of mislocalization depends on the presence or absence of visual references (Lappe et al. 2000): a uniform shift occurred when the experiments were done in complete darkness; a compression of visual space occurred when additional visual references were available. Because in complete darkness the only available information is extraretinal, the uniform shift is believed to rely on extraretinal signals. Compression of visual space, on the other hand, seems related to visual processes.
Although these findings clearly show that both visual and eye movement factors contribute to peri-saccadic mislocalizations, it is not obvious how visual and eye movement factors work together during peri-saccadic localization. Presaccadic visual factors comprise the visual scene and the visual cue that elicits the saccade. Postsaccadic visual factors include the new view of the visual scene after the saccade and the view of the saccade target or, if the saccade target is absent as in our experiments, visual references toward its position. Extraretinal factors comprise the direction and amplitude of the saccade and the landing point of the eyes. To separate visual from extraretinal information, Dassonville et al. (1995) and Miller and colleagues (Bockisch and Miller 1999; Miller 1996) asked subjects to perform saccades to an auditory saccade target in complete darkness and localize a briefly presented stimulus. In these experiments all visual factors were avoided, and subjects could use only extraretinal information to localize the flashed stimulus. The results of all three studies showed a uniform mislocalization of the flashed stimulus without compression.
The observation that compression only occurs when visual references are available after the saccade suggests a relationship to the saccade target theory (Currie et al. 2000; Deubel et al. 1996, 1998; McConkie and Currie 1996). In this view, compression may occur if the relative distance of the flash to the saccade target is underestimated and if that distance is used to reconstruct the flash's location from the saccade target's position after the saccade. Visual references would then be needed to inform the system about the position of the saccade target. Under this assumption, one may ask whether the position of the flash is encoded relative to the visual image of the saccade target or relative to the intended motor goal of the saccade. For normal saccades, these two parameters are closely related because the visual cue that elicits the saccade is identical to the point to which the eye moves. To differentiate between both factors we will exploit a paradigm that was first described by Hallett (1978) and is now known as the “anti-saccade task.” In this task, subjects are asked to make a saccade in the direction opposite to where a visual stimulus appeared. This task requires suppressing a reflexive saccade to the visual stimulus and initiating a voluntary eye movement to an unmarked location in the direction opposite to the visual cue (Hallett 1978). The anti-saccade paradigm therefore separates the saccade goal and the landing point of the eyes from the visual cue that elicits the saccade. Previous studies mainly used this paradigm to investigate physiological aspects of the cognitive control of saccades (e.g., Everling et al. 1999; Gottlieb and Goldberg 1999; Schlag-Rey et al. 1997) or to investigate differences in saccade parameters such as latency, velocity, or duration of the saccade (e.g., Hallet 1978; Smit et al. 1987).
Although the neuronal correlates of anti-saccade generation are still discussed, electrophysiological studies in animals and brain imaging studies in humans (PET) have revealed that prefrontal, frontal, and parietal cortical areas are involved in the correct performance of anti-saccades. The prefrontal and frontal areas include the dorsolateral prefrontal cortex (DLPC), the frontal eye field (FEF), and the supplementary eye field (SEF). All these areas are also important in the generation of pro-saccades. The mechanism by which the visual system computes the internally defined saccade target is still discussed, but it is clear that the generation of normal pro-saccades is at least in parts different to the generation of correct anti-saccades (for review, see Everling and Fischer 1998). Schlag-Rey et al. (1997) found neurons in the SEF that discharge significantly more before anti-saccades than before pro-saccades. They suggested that the behavior of these neurons reflects a mechanism by which the command of a voluntary anti-saccade can repel the command of a reflexive pro-saccade.
In this study, we take advantage of the anti-saccade paradigm to investigate the respective roles of the visual cue and the motor goal of the saccade in peri-saccadic mislocalization. Specifically we ask whether peri-saccadic mislocalizations also occur during anti-saccades, and if so, whether these mislocalizations follow the initial visual cue that elicits the saccade or the goal of the actual eye movement. An abstract of these results has been presented elsewhere (Awater and Lappe 2001).
Visual stimuli and experimental set-up
All experiments were performed in a dark room (luminance ≤ 0.1 cd/m2). Observers sat 60 cm in front of a large projection screen (type Cineplex, 120 × 120 cm, Dataframe). The head was immobilized by a head/chin rest. Stimuli were generated with a frame rate of 120 Hz by a Silicon Graphics computer (Indigo 2) and were back projected with a CRT video projector (Electrohome ECP 4100) onto the large projection screen. The experiments were done in two experimental blocks. In the first block, we tested only saccades in a fixed direction (rightward saccades). In the second block, saccade direction was varied either to the left or to the right of the fixation point.
The experiments were done in series of 100 single trials. All subjects performed for each condition at least three series on different days. For every subject and condition, between 170 and 400 responses were collected.
Six subjects (the 2 authors and 4 students), 23–37 yr of age, participated in the study. While subjects HA and ML were included in both blocks of experiments, the other four subjects participated only in one block of experiments. Data from four subjects were collected for each block. All subjects had normal vision and were experienced in psychophysical experiments.
Each trial started with the appearance of a fixation point (FP, 0.35 × 0.35°, luminance 20 cd/m2) at the center of the screen (Fig. 1A). After a variable time (1,700–1,870 ms), the fixation point disappeared and a visual cue, with the same size and luminance as the fixation point, was briefly presented for 50 ms either 7° to the left or 7° to the right of the fixation point (Fig. 1B). In blocks of trials, participants were instructed to perform as quickly as possible a saccade to the visual cue (pro-saccade) or to the mirrored position of the visual cue (anti-saccade; Fig. 1C). Around saccade onset, a thin vertical bar (0.35 × 90°, luminance 20 cd/m2) was flashed for one video frame (8 ms) at one of four possible positions arranged symmetrically around the fixation point [–10.5° (cross), –3.5° (circle), 3.5° (square), 10.5° (triangle)]. For a peak saccade velocity of 300° /s, the bar's image on the retina would be smeared to maximally 2.5°. About 500 ms after the saccade, a mouse pointer appeared in the middle of the screen, 2° above the center of the screen. The participants were asked to indicate the perceived position of the flashed bar by moving the mouse pointer to this position and pressing the left mouse button (Fig. 1D). At this time, subjects were not required to keep fixation. After participants had pressed the left mouse button, a new trial started with a new appearance of the fixation point. With this general procedure we tested three different conditions.
darkness. In this condition, all stimuli were presented on a dark background. Because the fixation point disappeared together with the presentation of the saccade target and saccade latencies varied between 160 and 380 ms, neither the fixation point nor the visual cue of the saccade target was visible at saccade onset or at the time when the subjects indicated the perceived position of the bar.
ruler on. In the second condition, visual references were provided by a horizontal ruler that was present during the whole trial (Fig. 1, C and D). The ruler was a white horizontal line (luminance 20 cd/m2) with short vertical lines at 7° intervals, each labeled with a number. One of the vertical lines fell on the fixation point and another on the saccade target.
postbar. In the third condition, the ruler was absent at the beginning of the trial but was switched on together with the bar. We were concerned that when the ruler was visible before the saccade, subjects might program pro-saccades to a ruler position rather than anti-saccades in response to the visual cue. The resulting saccades might thus not be true anti-saccades. In the “postbar” condition, the ruler was not visible at the time of the presentation of the visual cue and could therefore not serve as a visual references for the programming of the anti-saccades. Rather, the ruler was switched on after the presentation of the visual cue, when the programming of the saccade has already been done. In previous experiments, this condition resulted in compression that was as strong as when the ruler was visible at all times (Lappe et al. 2000).
Eye movement measurement
We recorded horizontal eye position with an Ober2 infrared eye tracker at a sample rate of 200 Hz. The goggles of the eye tracker restricted the binocular visual field to 30 × 20°. Saccade onset for each saccade was determined off-line by a velocity criterion (10% of maximum speed). To ensure correct saccade direction for both pro- and anti-saccades, the experimenter visually checked each saccade for incorrect pro- and anti-saccades. Furthermore, saccade amplitude and the timing of the saccade was visually controlled. Trials in which the saccade did not meet the requirements of the task were discarded.
For each trial, we determined the perceived position of the flashed bar and the time of the flash relative to saccade onset. To compare mislocalizations across subjects and conditions, we defined two index measures. The first describes the overall amount of mislocalization in saccade direction (shift index). For any point in time, it is defined as the mean error over the four perceived positions of the bar. The second index describes the strength of the compression (compression index). For any point in time, it is defined as the standard deviation across the four perceived positions of the bar. Both indices are normalized to their respective average values 100 ms before and after the saccade. This means for each bar position we first calculated control values as the mean perceived bar position when the bar was presented either 100 ms before or 100 ms after the saccade. In some cases, there were no measurements for the time range 100 ms after the saccade. In these cases, the control values were calculated only by the measurements 100 ms before the saccade. The shift index was defined as the mean of the differences of the perceived positions in the time range from 100 ms before to 100 ms after the saccade minus the control positions. The compression index was calculated as the ratio of the SDs of these differences and the SD of the control values. A value of 1.0 of the compression index indicates no compression, and a value of 0.0 means that all four positions would be seen in a single place.
Before analyzing the perceptual localization errors that occur at the time of pro- and anti-saccades, it is important to ensure that our subjects performed true anti-saccades. Especially in the condition ruler on, in which the ruler was present during the whole trial, the ruler provided a visual landmark at the position opposite to the visual cue (i.e., the landing point of the eyes in the case of anti-saccades). Thus instead of performing an anti-saccade, subjects could have simply use the disappearance of the fixation point as a trigger for a pro-saccade to the landmark. To check whether our subjects performed real anti-saccades, we compared their latencies in the pro- and the anti-saccade conditions. Many studies of anti-saccade performance in humans (Doma and Hallett 1988, 1989; Fischer and Weber 1992; Hallett 1978; Hallett and Doma 1980; Smit et al. 1987) and nonhuman primates (Amador et al. 1998; Everling and Munoz 2000; Everling et al. 1999; Gottlieb and Goldberg 1999; Schlag-Rey et al. 1997) have consistently reported that anti-saccades have a longer latency than pro-saccades. Figure 2 shows mean saccade latencies for our subjects in all conditions. For each subject and each condition, saccade latencies for anti-saccades were significantly longer than for pro-saccades (Mann-Whitney rank sum test: P ≤ 0.001).
Does the peri-saccadic mislocalization follow the initial visual cue or the goal of the actual eye movement?
Figure 3 shows the peri-saccadic mislocalizations for single subjects in the three conditions “darkness” (top row), “ruler on” (middle row), and “postbar” (bottom row) for pro- and anti-saccades (left and right columns). The x-axes show the time of the flash relative to saccade onset, and the y-axes show the apparent position of the bar. Each dot indicates a single measurement. Lines show running averages through the data, obtained with a Gaussian filter of 33 ms width. The gray arrows indicate the size and direction of the actual eye movement. It is obvious that the pattern of mislocalization is very similar for pro- and anti-saccades. For both saccade types, mislocalizations start about 50 ms before saccade onset, reach a maximum value at saccade onset, and return to the starting level at the end of the saccade. Furthermore, and more importantly, in all conditions, the localization errors follow the actual eye movement and not the visual cue that elicited the saccade. In the condition darkness, all four curves show a uniform mislocalization first in and later against the direction of the saccade. In the conditions ruler on and postbar, the mislocalizations are less uniform across the four flash positions. The most peripheral bar position (topmost curves) shows no mislocalization in saccade direction. Less peripheral bar positions (bottom curves) show a mislocalization into but never against saccade direction. This pattern of results is similar to the earlier described compression in that the errors vary with position in the visual field. The compression can be seen when one compares the distances of the perceived bars from each other. At saccade onset the four curves are closer together than before or after the saccade. The compression is less pronounced than in earlier experiments, however, because the position beyond the saccade target (top most curve) shows no mislocalization against saccade direction.
Does the peri-saccadic mislocalization depend on randomized saccade directions?
In the first block of experiments, subjects performed only rightward saccades. This means that, for pro-saccades, the visual cue appeared at +7°, whereas for the anti-saccades, the visual cue appeared at –7°, but in both cases, the actual eye movement was performed in the same direction. In the first block of experiments, the subjects could therefore anticipate the saccade direction and pay more attention to the known position of the following saccade target. In the second block of experiments, we randomly intermixed saccade directions. In this case, an anticipatory shift of attention was not possible because the visual cue could appear either to the left or to the right. If peri-saccadic mislocalization is independent of whether the saccade was always performed in a fixed predictable direction or varied across trials (unpredictable direction), the results in the two blocks of experiments should be the same. Figure 4 shows that this is the case. The figure plots the responses of one subject in the condition ruler on when the saccade was performed in a predictable (left column) and in an unpredictable direction (right column). The pattern of mislocalization is similar in the two cases. This is true for pro- and anti-saccades. Thus localization errors do not depend on the predictability of the direction of saccade direction.
Compression of visual space
To compare and quantify the effects of shift and compression, we calculated shift and compression indices for each condition (see methods). Figure 5 shows the shift (top) and the compression index (bottom) for the three conditions averaged across subjects. The dotted curves indicate pro-saccades. The solid curves indicate anti-saccades. A positive value of the shift index indicates a uniform shift in the direction of the saccade and a negative value a uniform shift in the direction opposite to the saccade. The figure clearly shows that the results for pro- and anti-saccades are very similar. However, the shift indices differ between the three conditions. In the ruler on and postbar conditions, the shift returns to zero at the end of the saccade, whereas in the condition darkness, a reversion and an overshoot in the direction opposite to the saccade at the end of the eye movement can be seen. Such a reversal of the direction of uniform shift has been described earlier and ascribed to a sluggish extraretinal signal that cannot follow the speed of the eye movements (Dassonville et al. 1992; Honda 1989, 1990). In contrast, the shift in the two conditions ruler on and postbar shows no reversal. In these two conditions, the shift starts about 70 ms before saccade onset, reaches a maximum at the beginning of the saccade, and returns to baseline shortly after saccade offset.
A difference between the three conditions can also be seen in the compression index (Fig. 5, bottom). The compression in the conditions ruler on and postbar (middle and right column) was stronger than in the condition darkness (left column). Again, this was the case for both pro- and anti-saccades. We ran a Kruskal-Wallis one-way ANOVA of the average compression values in the last 50 ms prior to saccade onset of each condition. This time window was chosen because during the last 50 ms prior to saccade onset strong localization errors occur while the eyes stay at the fixation point and therefore no influences from the actual eye movement (such as retinal smear) occur. The ANOVA revealed that the compression in the conditions ruler on and postbar was significantly different from the compression in the condition darkness. Conditions ruler on and postbar did not show significant differences (all pairwise multiple compression procedure; Dunn's method: P ≤ 0.05). This was the case for both pro- and anti-saccades.
The present experiments were performed to investigate whether peri-saccadic localization errors, specifically peri-saccadic compression, are directed toward a visual cue that elicits a saccade or toward the goal of the eye movement. We used the anti-saccade paradigm to separate the visual cue from the motor command of the eyes. The results showed almost identical peri-saccadic mislocalizations during pro- and anti-saccades that are directed toward the actual eye movement and not the visual cue. Furthermore, and consistent with our previous findings (Lappe et al. 2000), the pattern of mislocalization varies with respect to the presence or absence of visual references: peri-saccadic compression only occurs when visual references are available immediately after the saccade. Strength of compression as well as magnitude of uniform shift was identical for pro- and anti-saccades. From these findings, we conclude that the initial visual cue is not important for peri-saccadic localization errors.
Strength of compression
Although compression is clearly visible in the results when one compares the distances of the bars from each other in the compression index, the compression in the current experiments is weaker than in previous studies (Lappe et al. 2000; Morrone et al. 1997; Ross et al. 1997) and is not accompanied by mislocalizations against the saccade direction for a flash position beyond the saccade target. Several differences between the present experiment and the older experiments may account for this. One reason could be the small distance between the saccade target and the bars in the present experiment. For the bar position beyond the saccade target, this distance is only 3.5°. In a previous study, we presented the bar at two positions beyond the target (3.6° and 7.2°) and found that the nearer bar position often did not show a strong mislocalization in the opposite direction to the saccade (Lappe et al. 2000). In the present experiments, we may therefore not necessarily expected a strong mislocalization against the direction of the saccade at that bar position beyond the target. A further reason for the weak compression could be the small saccade amplitude. In the present experiments, saccade amplitude was 7°. Other studies that found compression used saccade amplitudes of 12.8° (Lappe et al. 2000) or 10–20° (Morrone et al. 1997; Ross et al. 1997). On the other hand, differences in saccade direction (fixed or randomized) cannot explain the weak compression. In our experiments (Fig. 4) as well as in the study of Bockisch and Miller (1999), the results for fixed and randomized conditions were very similar.
Contribution of visual and eye movement parameters to peri-saccadic localization
Our results show that localization errors at the time of saccadic eye movements follow the actual eye movement and not the visual cue. This is the case for pro- and anti-saccades. At first glance, this seems to suggest that the localization errors originate mainly from motor factors rather than from visual factors. However, as in our previous study, peri-saccadic localization errors were different in darkness and with visual references. Thus visual factors are also involved. The uniform shift observed in darkness is believed to rely on the inaccurate information of the motor command, i.e., the extraretinal signal. However, if only a uniform extraretinal signal contributes to visual perception, one would expect an equal shift regardless of whether the experiments were performed in darkness or with visual references. This is not the case, and therefore we must conclude that also visual factors contribute to the localization process. How can the observation that the mislocalization follows the eye movement be reconciled with the observation that visual factors induce the compression?
Visual factors that may be involved in compression include the saccade target as well as supplementary visual references. Supplementary visual references induce compression only when they are available after the saccade. The presence of references prior to the saccade does not lead to compression (Lappe et al. 2000). The time immediately after the saccade and the presence of visual references at this time are also important factors in the “reference object theory” (Deubel et al. 1984 1996, 1998) and the “saccade target theory” (Currie et al. 2000; McConkie and Currie 1996) of space constancy. These theories propose that the visual system assumes the world as being stable unless evidence to the contrary is registered immediately after the saccade. The position of the saccade target is stored prior to saccade onset. After the saccade, the visual system searches for the saccade target within a spatio-temporal window. When the saccade target is found, the assumption of a stable world is maintained. If the saccade target cannot be found, the assumption of a stable world is broken, and the visual system has to re-analyze the visual scene. According to these theories, the saccade target plays a crucial role in the perception of space around saccadic eye movements.
When an object is briefly flashed before the saccade, its position after the saccade cannot be estimated from postsaccadic vision but must be reconstructed from presaccadic information. Compression may occur if the flash's location after the saccade is reconstructed from relative position information with respect to the saccade target and if this relative distance measure is compressed. In this case, compression would be seen when the saccade target's position can be used for postsaccadic reconstruction of the scene, i.e., when information about the saccade target's position is available from visual references. Compression would not be seen when the saccade target's position is not visually specified after the saccade as in the darkness condition. These considerations leave open, however, whether the compressed distance measure is relative to the visual image of the saccade target or relative to the motor goal of the saccade. Because the mislocalization follows the eye movement rather than the visual cue in the anti-saccade task, we suggest that the compression is relative to the intended motor goal of the saccade. The anti-saccade task does not provide a visual saccade target that can be stored and that matches with the landing point of the eyes. Therefore a relative localization mechanism between the saccade target and the flashed bar as it is proposed in the reference object theory is unlikely. Recently, Deubel (2003) and Deubel et al. (2002) showed that the proposed re-calibration process must not necessarily be related to the saccadic target but may also use other objects within the spatial window around the saccade target as references. In the anti-saccade condition, the visual system may use the intended landing point of the eyes, i.e., the goal of the saccade, in the absence of a visual target. The visual system could then try to arrange this position with the space provided by external references. In darkness, such references are not available, and the system could use only the uniform but inaccurate information of the extraretinal signal. This would result in a uniform shift. In the conditions ruler on and postbar, external references are available after the saccade and therefore the visual system has the possibility to re-calculate the goal of the saccade from external references.
We thank K. Georg, M. Sommer, T. Schnetgoeke, and L. Michels for help.
Financial support from the Human Frontier Science Program is gratefully acknowledged.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2004 by the American Physiological Society