When we move forward the visual images on our retinas expand. Humans rely on the focus, or center, of this expansion to estimate their direction of self-motion or heading and, as long as the eyes are still, the retinal focus corresponds to the heading. However, smooth pursuit eye movements add visual motion to the expanding retinal image and displace the focus of expansion. In spite of this, humans accurately judge their heading during pursuit eye movements even though the retinal focus no longer corresponds to the heading. Recent studies in macaque suggest that correction for pursuit may occur in the dorsal aspect of the medial superior temporal area (MSTd); neurons in this area are tuned to the retinal position of the focus and they modify their tuning to partially compensate for the focus shift caused by pursuit. However, the question remains whether these neurons shift focus tuning more at faster pursuit speeds, to compensate for the larger focus shifts created by faster pursuit. To investigate this question, we recorded from 40 MSTd neurons while monkeys made pursuit eye movements at a range of speeds across simulated self- or object motion displays. We found that most MSTd neurons modify their focus tuning more at faster pursuit speeds, consistent with the idea that they encode heading and other motion parameters regardless of pursuit speed. Across the population, the median rate of compensation increase with pursuit speed was 51% as great as required for perfect compensation. We recorded from the same neurons in a simulated pursuit condition, in which gaze was fixed but the entire display counter-rotated to produce the same retinal image as during real pursuit. This condition yielded the result that retinal cues contribute to pursuit compensation; the rate of compensation increase was 30% of that required for accurate encoding of heading. The difference between these two conditions was significant (P < 0.05), indicating that extraretinal cues also contribute significantly. We found a systematic antialignment between preferred pursuit and preferred visual motion directions. Neurons may use this antialignment to combine retinal and extraretinal compensatory cues. These results indicate that many MSTd neurons compensate for pursuit velocity, pursuit direction as previously reported and pursuit speed, and further implicate MSTd as a critical stage in the computation of egomotion.
How do we know which way we are moving as we walk or drive? This seemingly simple question is actually rather complicated. We rely heavily on vision to guide us, but the task of the visual system is complicated by the fact that our eyes move in the head and the head moves on the body. Yet somehow our nervous system effortlessly and accurately guides us through even extremely complex environments. Gibson offered an important insight as to how we may solve this problem by identifying a visual cue that corresponds to the direction of self-motion (Gibson 1950; Warren 1995).
Gibson noted that when we move forward the retinal image expands. The center, or focus, of this expansion (FOE) corresponds to the instantaneous direction of translation, or heading, when the eyes are still. In this condition humans are able to use the FOE to accurately estimate their heading (Warren and Hannon 1988). However, when we smoothly rotate our eyes, as we commonly do while walking or driving, the FOE on the retina is displaced from the true heading as shown in Fig.1 A. Moreover, the faster we rotate our eyes the larger the FOE displacement (Fig. 1,A–C). Relying on the retinal FOE position alone would cause us to misperceive our heading, especially when we rotate the eyes quickly. Clearly we know which way we are headed even while making pursuit eye movements. How is this possible?
Banks and colleagues determined that extraretinal cues present during pursuit eye movements enable accurate self-motion judgments (Royden et al. 1992, 1994). They also found that observers judged self-motion about as accurately during fast pursuit as during slow pursuit, implying that the nervous system is able to compensate for the larger displacements of the FOE that occur during fast pursuit. If this perceptual ability is mediated by neurons in cortical area MSTd (dorsal aspect of the medial superior temporal area), then MSTd neurons should modify their tuning for the retinal location of the FOE more during fast pursuit than during slow pursuit. The results we report below indicate that they do. We also found that, contrary to what one would predict from human psychophysics, MSTd neurons modify their retinal FOE tuning based on purely retinal motion signals during approach to a frontoparallel wall.
We recently reported that many neurons in macaque extrastriate cortical area MSTd use pursuit signals to compensate, at least in part, for the displacement of the FOE caused by pursuit eye movements (Andersen et al. 1996; Bradley et al. 1996). We also reported that MSTd neurons use pursuit signals to compensate, at least in part, for the displacement of the center of rotation of rotary patterns caused by pursuit eye movements (Andersen et al. 1996; Bradley et al. 1996). Figure 1, D—F, illustrates how the center of rotation shifts orthogonally to the direction of pursuit, with faster pursuit causing greater shifts. Finally, we also found that during passive head rotation, which similarly displaces the FOE (or center of rotations) if the eyes remain fixed in the head, signals of vestibular origin drive pursuit compensation (Shenoy et al. 1999).
To ask whether neurons modify their tuning for the retinal focus location more at faster pursuit speeds, we recorded from MSTd neurons as monkeys pursued across computer displays simulating approach to a vertical wall. Monkeys either fixated a stationary point or pursued a target moving at three different speeds (2.58, 5.05, and 9.22°/s) while viewing a display simulating 1 of 11 different headings (6° steps) toward the screen along the axis of preferred-null direction pursuit for each neuron. We measured tuning curves—the neural responses to the 11 heading directions—during both fixation and pursuit. We determined the amount of compensation for FOE displacement by comparing the alignment of these tuning curves. We found, in individual neurons and across the population, that MSTd compensates more during fast pursuit than during slow pursuit as predicted for a cortical area reporting heading and other higher-order motion parameters regardless of pursuit speed. We obtained similar results from the neurons that preferred contraction or rotary patterns, as opposed to expanding patterns, by presenting contracting or rotating patterns and observing that these neurons also compensate more during fast pursuit than during slow pursuit.
We also asked how retinal and extraretinal signals contribute to pursuit compensation. We measured tuning curves for the location of the FOE, or center of rotation, in a simulated pursuit condition, in which the animal fixated while the optic-flow stimuli were swept across the display. Importantly, the visual image on the retina was the same in the real and simulated pursuit conditions since in the real pursuit condition the optic-flow display was fixed on the computer screen and was swept across the retina by a pursuit eye movement while in the simulated pursuit condition the eyes remained fixated but the entire optic-flow display was swept across the computer screen in the direction opposite to the real pursuit eye movement. Thus the visual stimulus was the same in real and simulated pursuit conditions, the only difference being that no extraretinal pursuit-related signals were present in the simulated pursuit condition because the eyes were stationary. This allowed us to measure the amount of retinally driven compensation at each simulated pursuit speed. We found that both individual MSTd neurons and the population as a whole compensate more during fast than during slow simulated pursuit, but the extent of compensation was significantly less during simulated than during real pursuit. These results will be discussed in relation to psychophysical studies in human subjects. A brief report of this material has appeared previously (Shenoy et al. 1998).
Experiments were conducted in two adult male Rhesus monkeys (Macaca mulatta). These monkeys (DAL and FTZ) also participated in a previous self-motion study (Shenoy et al. 1999). All protocols were approved by the Caltech Institutional Animal Care and Use Committee.
We have previously reported our surgical and behavioral-training techniques (Shenoy et al. 1999). In brief, we implanted screws in the skull and constructed a methylmethacrylate fixture for immobilizing the head. We also implanted a wire coil between the conjuctiva and the sclera for the measurement of eye position. Behavioral training on oculomotor tasks began no sooner than 1 wk after surgery. Monkeys received juice rewards for correct performance during both behavioral-training and experimental sessions. Adequate performance levels, typically well above 90% on all tasks, were reached after a few weeks of training. We then performed a second sterile surgical procedure to open a craniotomy (5 mm posterior, 17 mm lateral; left hemisphere in DAL, right hemisphere in FTZ) and to implant a cylinder (dorsoventral orientation) for chronic access to cortical area MSTd.
We have also previously reported our recording technique and procedure for identifying MSTd (Shenoy et al. 1999). In brief, we advanced standard microelectrodes dorsoventrally and recorded extracellular action potentials. MSTd was identified based on anatomical location (e.g., depth below dura, position relative to gray and white matter boundaries, and position relative to area MT) and response properties (e.g., large receptive fields including parts of the contra- and ipsilateral visual field and selectivity for optic-flow patterns such as expansion patterns). Tuning for optic-flow pattern type was determined by visual inspection and by recording with visual stimulation in at least one location in the receptive field (RF). Only optic-flow–tuned neurons were tested in all experiments and are included in our data base.
All experiments were conducted in a sound-insulated room, which was totally dark except for the visual stimuli. We generated expanding random-dot optic-flow fields by simulating forward translation at 16.5 cm/s toward a frontoparallel wall held 38.1 cm distant. Dots were white (approximately 10 candela/m2) on a completely black background and were spatially antialiased (3 × 3 pixels, using coverage factors for a circle), allowing dots to move smoothly at any speed. Each dot was assigned a random age (0–287 ms) and moved at constant velocity for the remainder of its 300-ms lifetime or until it crossed the stimulus boundary, in which case it was extinguished and reborn at a random location. Dot speeds were proportional to the eccentricity from the FOE, reaching 9.22°/s at 24° eccentricity; as in our earlier experiments, they did not evolve as a function of time. Displays were viewed binocularly.
To determine a neuron's preferred optic-flow pattern, we displayed eight patterns from “spiral space” and eight patterns from “laminar space” (Graziano et al. 1994). The spiral space patterns include expansions, rotations, contractions, and spirals; they are all constructed by simply rotating the motion vectors in the expansion pattern stimulus through a given (counterclockwise) angle: 0° for expansion, 45° for counterclockwise-expanding spiral, 90° for counterclockwise rotation, 135° for counterclockwise-contracting spiral, 180° for contraction, 225° for clockwise-contracting spiral, 270° for clockwise rotation, and 315° for clockwise-expanding spiral. The laminar stimuli, on the other hand, consist of uniform, unidirectionally moving dots drifting (4.33°/s) in one of eight directions that were evenly spaced at 45° intervals.
To simulate different headings, we displayed 11 different optic-flow patterns with varying focus positions. Focus positions varied from −30° to 30° in 6° increments along an axis parallel to the neuron's preferred-null pursuit axis (see Data analysis). Our procedure for varying the FOE position is conceptually identical to combining radial motion with different speeds of laminar motion, as done by Duffy and Wurtz (1997b). Recall that the FOE shifts in the direction of pursuit for expansion patterns (Fig. 1,A–C), in the direction opposite to pursuit for contraction patterns, and the center of rotation shifts in a direction orthogonal to the direction of pursuit for rotational patterns (Fig. 1,D—F) (Andersen et al. 1996; Bradley et al. 1996; Shenoy et al. 1999). Therefore, for the few neurons that preferred rotating patterns (see Data analysis), we varied the center of rotation from −30° to 30° in 6° increments along an axis orthogonal to the neuron's preferred-null pursuit axis.
We displayed optic-flow stimuli (400 dots) on a 20° × 20° region of a computer monitor (800 × 600 pixels, 75 frames/s). This was the largest possible stimulus area due to monitor size (50° × 38° at 38.1 cm) and the required movement of the stimulus across the screen in the simulated pursuit condition. Visual stimuli were presented either at a fixed location on the monitor (Preferred optic-flow experiment) or drifting across the monitor (simulated pursuit condition in the Pursuit compensation experiment). Pursuit targets moved at one of three speeds, 2.58, 5.05, or 9.22°/s in the preferred pursuit direction experiment and in the real pursuit condition of the pursuit compensation experiment. Fixation targets remained stationary on the display in the preferred optic-flow experiment and the fixed and simulated pursuit conditions of the pursuit compensation experiment. Pursuit and fixation targets were larger than the optic-flow stimulus dots (5 × 5 pixels, antialiased).
Visual stimulus design
We selected the optic-flow and pursuit-speed parameters described above to shift the FOE (or center of rotation) on the retina by prescribed amounts during pursuit across the optic-flow stimuli. The same FOE (or center of rotation) shifts were also achieved in the simulated pursuit condition by drifting the entire optic-flow stimulus across the monitor to produce the same net visual motion across the retina. Importantly, to investigate how pursuit speed influences visual responses, we needed to find visual-stimulus and behavioral parameters such that the range of pursuit speeds produces easily measurable focus (center of rotation) shifts and neural-response changes. For simplicity, we will only describe the design of the expansion stimuli; recall that contraction and rotation stimuli are created by rotating the visual motion vectors in the expansion stimuli through various angles.
When a subject approaches a wall, the visual image on the retina expands. This expansion is described by Equation 1where θ (rads) is the visual angle to a point on the wall and dθ/dt (rads/s) is the rate at which this angle increases. The point is located x (cm) from the center, the distance from the observer to the wall is z (cm), and the approach speed is Tz (cm/s). An equivalent governing equation is Equation 2where dx/dt (cm/s) is the radial speed and the ratio of Tz to z (1/s) is the speed gradient.
Except for the distance to the wall, which is the distance to the computer monitor (z = 38.1 cm; viewed as closely as possible to maximize visual area), all other parameters are free. We constrained the pursuit speed to the range between 0 and 10°/s, because faster pursuit for a few seconds requires a larger monitor and is more difficult for monkeys to perform. We selected three pursuit speeds in addition to 0°/s (fixed gaze); while testing more pursuit speeds is desirable, three provides sufficient data and keeps the total experimental time within reasonable limits (e.g., less than 2 h per neuron). We also constrained the shifts of the FOE or origin of rotation at each of the three pursuit speeds to be multiples of ΔH (deg), which is the angle separating simulated focus locations. This assures that retinal alignment of neural tuning curves will be possible by shifting the tuning curves by integer multiples of ΔH, for all pursuit speeds, which is helpful for cross-correlation analysis (see Data analysis). Finally, there should be a single forward approach speed. With z = 38.1 cm,Tz is constrained to fall between 10 and 20 cm/s to avoid approach speed regimes where pursuit speed has little effect on focus shift (Tz > 20 cm/s) or has a huge effect on focus shift (Tz < 10 cm/s).
We simultaneously solved Eq. 1 (or equivalentlyEq. 2 ) for each of three pursuit speeds, subject to the constraints, to arrive at a reasonable (but not unique) set of parameters. This solution has the added benefit of using the approach speed, the pursuit speed, and the focus shift used in our most recent study of pursuit compensation (Shenoy et al. 1999); this overlap should facilitate comparisons of the results. As mentioned above, the parameters are as follows: 16.5 cm/s approach speed (Tz /z = 16.5 cm/s/38.1 cm = 0.4331 s−1); 2.58, 5.05, and 9.22°/s pursuit speeds; and 6, 12, and 24° focus shifts. ΔH is 6° and the focus shifts correspond to 1, 2, and 4 ΔH. The slowest pursuit speed is slow enough to directly check for compensation in the presence of even a weak pursuit signal. The fastest pursuit speed was used in our previous study, yielding an average compensatory shift of approximately 21° (approximately 88% of perfect compensation), which is large enough to expect measurable shifts even at slower pursuit speeds. We used 11 focus locations spanning a wide range: −30, −24, … , −6, 0, +6, … , +24, +30° (by comparison ±32° range, Shenoy et al. 1999; ±40°, Bradley et al. 1996). In this study we are attempting to measure smaller predicted focus displacements (6, 12, and 24°) than previously (24°, Shenoy et al. 1999; 30°, Bradley et al. 1996), but the increased sampling resolution (ΔH =6°) and sample number (11 instead of 9 focus locations) enhances measurement resolution.
Monkeys were trained to fixate and to make smooth pursuit eye movements. One or both of these behaviors were employed in three sequential, blocked experiments: the preferred optic-flow, preferred pursuit direction, and pursuit compensation experiments.
The preferred optic-flow experiment measured the response of each neuron to spiral space and laminar space visual motion patterns. Trials consisted of acquiring and fixating (±2.5° stationary eye box) a stationary target. Monkeys acquired the target within 0.5 s of target onset, after which they were required to maintain fixation for an additional 1.2 s (1.7 s total trial time). We displayed optic-flow stimuli throughout this 1.2-s fixation period. For each trial we displayed one of the 16 optic-flow patterns (from spiral or laminar space) in a pseudorandom fashion (stimuli randomly drawn without replacement and blocked by repetition number). Optic-flow stimuli were centered at 0°,0°; +10°,+10°; −10°,+10°; −10°,−10°; or +10°,−10° (horizontal, vertical pairs; + indicates either contralateral or up) with respect to the point of fixation (0°,0°) to position the stimulus nearer the center of the neuron's receptive field. RFs were mapped roughly by hand-positioning optic-flow patterns.
The preferred pursuit direction experiment measured the response of each neuron to smooth pursuit eye movements in different directions and at different speeds. Trials consisted of pursuing (±4.0° moving eye box) a target moving in one of eight directions (0° is right, 45° is up-right, … , 315° is down-right) at one of three speeds (2.58, 5.05, 9.22°/s). Monkeys acquired the moving target within 0.8 s of target onset, after which they were required to continue pursuing for an additional 1.2 s (2.0 s total trial time). Pursuit directions and speeds were presented in pseudorandom order, and we inspected eye-position traces on-line and off-line to verify pursuit performance. Pursuit trajectories were centered on the same monitor location (0°,0°; gaze straight ahead) to equalize all gaze angles on average. Note that no optic-flow stimulus was presented during this task.
The pursuit compensation experiment measured the response of each neuron to a range of simulated headings while fixating or pursuing a target. Trials consisted of fixating (±4.0° stationary eye box) a stationary target or pursuing (±4.0° moving eye box) a moving target in one of three conditions: fixed gaze, real pursuit, or simulated pursuit. We inspected eye-position traces on-line and off-line (Fig. 7) to verify fixation and pursuit performance. In all conditions, monkeys acquired the stationary (or moving) target within 0.8 s of target onset, after which they were required to maintain fixation (or continue pursuing) for 1.2 s (2.0 s total trial time). In all three conditions we displayed optic-flow stimuli throughout this 1.2-s fixation (or pursuit) period. For each trial we displayed 1 of 11 optic-flow patterns, with varying focus locations, in pseudorandom order. Optic-flow stimuli were centered at 0°,0°; +10°,+10°; −10°,+10°; −10°,−10°; or +10°,−10° with respect to the point of fixation, and fixation was within ±5°, ±5° of screen center (0°,0°) to position the stimulus nearer the center of the neuron's RF.
Fixed-gaze condition trials presented the optic-flow stimuli while monkeys fixated. The optic-flow stimuli were displayed at a fixed location on the computer screen. Real pursuit condition trials presented optic-flow stimuli while monkeys pursued in the neuron's preferred pursuit direction (determined in the preferred pursuit direction experiment, see Data analysis) at 2.58, 5.05, or 9.22°/s. The optic-flow stimuli were displayed at a fixed location on the screen; the eyes pursued across this fixed stimulus display. Simulated pursuit condition trials presented the optic-flow stimuli while monkeys fixated, but the optic-flow stimuli drifted in the direction opposite the preferred pursuit direction at 2.58, 5.05, or 9.22°/s. Such counter-rotation creates a retinal stimulus identical to that in the real pursuit condition (i.e., drift of the entire stimulus across the retina in the direction opposite the preferred pursuit direction). The important difference between the two conditions is that the eyes rotate during real pursuit but not during simulated pursuit.
Real pursuit trajectories were centered on the point of fixation used in the other two conditions; this was done to equalize the average gaze angles. Stimulus counter-rotation trajectories in the simulated pursuit condition were centered on the stimulus location used in the other two conditions. These trajectories were equalized for the last 1.0 s of the 1.2-s fixation/pursuit period, as this is the 1.0-s period of neural data analyzed (see Data analysis).
Horizontal and vertical eye positions (<1° resolution) were sampled every millisecond, and action potential event times were stored for off-line analysis with microsecond resolution. Neurons from two monkeys were recorded. Data trends are similar in both monkeys so the data were pooled for population analyses.
We analyzed the preferred optic-flow experiment data to determine each neuron's preferred spiral space and laminar space optic-flow patterns. We calculated the average neural response (mean of 3 trial replicates) to the stimuli during the last 1.0 s of the 1.2 s stimulus presentation. Neglecting the first 0.2 s effectively discards the phasic stimulus response and emphasizes the tonic stimulus response. We then estimated the preferred spiral space and laminar space optic-flow patterns as the angle of the response-weighted vector sum (Geesaman and Andersen 1996; Shenoy et al. 1999). The preferred optic-flow pattern's angle (ϕ) is given by tan(ϕ) = S/C, where S is the sum of Fi sinϕiand C is the sum ofFi cosϕi over all eight spiral space or laminar space optic-flow patterns (i = 1, 2, … , 8).Fi and ϕicorrespond to the average firing rate and the angle in spiral space (or laminar space) of the optic-flow patterns, respectively. For example, a neuron might have a preferred direction in spiral space of 8°, corresponding to an expansion pattern with a slight counterclockwise rotational component. The same neuron might have a preferred direction in laminar space of 47°, corresponding to laminar motion up and to the right. These are the preferred optic-flow patterns in the sense that the difference between the neuron's response to the preferred pattern and its opposite is greater than the difference in response to other patterns and their opposites. Finally, we assessed tuning significance using circular statistics (Geesaman and Andersen 1996; Zar 1996). The Rayleigh test checks for single-mode tuning and uses the length of the response-normalized response-weighted vector sum ( /ΣFi ) and the total number of trials. These measures and tests were also used to calculate population-average preferred directions and directional biases. A χ2 test, with Yates' correction for continuity, was used to test for an expansion pattern (spiral space) or contralateral direction (laminar space) bias across the population (Zar 1996).
We analyzed the preferred pursuit direction experiment data to determine each neuron's preferred direction of pursuit at each of the three pursuit speeds, 2.58, 5.05, or 9.22°/s. We calculated the average neural response (mean of 3 trial replicates) during the last 1.0 s of the 1.2 s pursuit period. For each pursuit speed we estimated the preferred direction as the angle of the response-weighted vector sum by the same methods used for estimating the preferred optic-flow pattern. In this case the preferred pursuit direction (ϕ) is given by tan(ϕ) = S/C, whereS is the sum ofFi sinϕi andC is the sum ofFi cosϕi over all eight pursuit directions (i = 1, 2, … , 8).Fi and ϕicorrespond to the average firing rate and the pursuit angle, respectively. For example, a preferred pursuit direction of 184° is very close to leftward pursuit. This is the preferred pursuit direction in the sense that the neuron's response difference is greatest between pursuit in the preferred and opposite directions. The Rayleigh test was again used to assess pursuit-tuning significance in single neurons and across the population. A χ2 test, with Yates' correction, was used to test for an ipsilateral direction bias across the population.
We analyzed the pursuit compensation experiment data to determine the influence of real pursuit and simulated pursuit on the responses to the various optic-flow displays measured in the fixed-gaze condition. We constructed seven focus tuning curves for each neuron: one for the fixed-gaze condition, one for each of the three pursuit speeds in the real pursuit condition, and one for each of the three simulated pursuit speeds in the simulated pursuit condition. Tuning curves use the average neural response (mean of 3 trial replicates) during the last 1.0 s of the 1.2 s stimulus presentation.
To understand the effect of real and simulated pursuit at each of the three speeds on the visual responses, we compared tuning curves from these conditions with the fixed gaze tuning curve. In general, we found that the shapes of the tuning curves were similar and were often either sigmoidal or Gaussian. The primary difference between the curves was a horizontal offset, corresponding to shifts along the axis of the independent variable (focus location). We quantified this shift by cross-correlating each of the tuning curves with the fixed gaze tuning curve; the shift was taken to be the value of the cross-correlation offset parameter that yielded the highest correlation.
Cross-correlation is well suited for the type of data we analyzed (well-sampled tuning curves) and the question we asked (quantify horizontal shift). We have discussed previously the numerous merits of cross-correlation for such analysis, as well as the few limitations (Shenoy et al. 1999). Cross-correlation reduces to correlation at each horizontal shift Equation 3The mean response vectors for the two tuning curves arex and y, indexed by focus location, and the averages of the two response vectors are and . At any given horizontal shift a limited range of focus locations overlap; the focus location index spans this range. The correlation coefficient, rc , has a range of ±1, where perfectly correlated (anticorrelated) tuning curves have a correlation coefficient of +1 (−1). Totally uncorrelated tuning curves have a correlation coefficient of zero. To interpolate the tuning curves between the 6° spaced measurements, we smoothed the tuning curves with a three-point moving average (twice; uniform weights) followed by a spline interpolation (1° sampling). While all results are qualitatively similar without smoothing and interpolation, this method helps to detect small shifts.
We defined compensation, or compensatory shift, to be the difference between the theoretically determined FOE location on the retina during pursuit (e.g., 24° for 9.22°/s pursuit) and the empirically determined shift (e.g., 8°) as measured by cross-correlation. This example has a compensation value of 16° (24° − 8° = 16° or 16°/24° = 66.6%). We assessed the degree of compensation across the population by calculating the first, second (median), and third quartiles of the compensation distribution, as well as the mean. We used the Wilcoxon nonparametric t-test to assess the significance of these shifts away from zero, or the difference in shifts measured in two conditions (paired t-test). The Mann-Whitney nonparametric t-test was used to test if two distribution means are significantly different (nonpairedt-test).
To quantify the degree to which individual neurons change their compensation as a function of real or simulated pursuit speed, we regressed a line to the compensation versus pursuit speed data. We included the 0° compensation at 0°/s pursuit speed data point (autocorrelating the fixed gaze tuning curve yields 0° compensation by definition). We fit a linear model (slope and intercept) to the data because: 1) the theoretical FOE shift, and therefore the magnitude of the required compensatory shift in a neuron's tuning curve, is approximately linear as a function of pursuit speed; and2) the psychophysical literature on self-motion perception in humans suggests that error (equal to 1 – compensation) is a linear function of pursuit speed (Crowell et al. 1998a;Royden et al. 1994). We divided the slope of each neuron's regression line (M measured) by the slope of the line corresponding to perfect compensation (M perfect ≈ 24°/9.22°/s = 2.6° per °/s) to calculate a compensation index (CI):CI = 100 × (M measured/M perfect). The CI expresses each neuron's ability to compensate across pursuit speeds as a proportion of the increase required theoretically for perfect compensation. This CI is directly analogous to the CI used in our previous human psychophysical work (Crowell et al. 1998a) and facilitates the comparison of physiological and psychophysical performance.
We recorded and analyzed data from 40 neurons in two monkeys, 30 from monkey DAL and 10 from monkey FTZ, in the preferred optic-flow, preferred pursuit direction, and pursuit compensation experiments.
Preferred optic-flow experiment
Figure 2 shows an MSTd neuron that responded vigorously to certain spiral space optic-flow patterns (patterns with an expansive component in this case) but not to others. We estimated the preferred spiral space pattern (see Data analysis) and noted this preferred pattern for use again in the pursuit compensation experiment. Figure3 A plots the preferred spiral space pattern for all the neurons in our population. While we found a wide range of preferred patterns, the distribution was not uniform (P < 0.01, Rayleigh test); instead, significantly more neurons (27/40) preferred patterns containing an expanding component than a contracting component (P < 0.05, χ2test). In fact, the mean population angle was 20 ± 36° (95% confidence interval), which includes the pure expansion pattern (0°).
Figure 4 illustrates tuning properties quite similar to those in Fig. 2, but this figure plots a neuron's response to eight directions of laminar motion. This neuron clearly preferred rightward visual motion (349° or, equivalently, −11°). We estimated the preferred laminar space direction for each neuron and noted these directions for later analysis. Figure 3 B plots the preferred laminar space directions for our entire population. Again we found a wide range of preferred directions, but the distribution was not significantly nonuniform (P > 0.05, Rayleigh test) even though there was a trend to favor (25/40 neurons) contralateral directions (P > 0.05, χ2test). The mean population angle was 140°, reflecting this slight bias toward contralateral directions.
Preferred pursuit direction experiment
Figure 5 illustrates that an MSTd neuron can respond vigorously during pursuit in some directions, leftward-oriented directions in this case, but not in other pursuit directions. We estimated the preferred direction of pursuit for each of the three pursuit speeds separately (see Data analysis). We noted the preferred pursuit directions, as well as their opposites (null directions), for use in the pursuit compensation experiment. To quantify how similar these three preferred directions are in each neuron, we calculated the range, defined to be the smallest arc that contains all three preferred pursuit directions, for each neuron in our population. We found that most neurons had a direction range within one quadrant (<90° range): the population distribution has a 16.5° first quartile, 39.7° median, and 90.4° third quartile.
Figure 3 C plots the preferred pursuit direction, for 5.05°/s pursuit, for each neuron in the population. This pursuit speed is of particular interest as it is the closest to the speed of the visual stimuli (4.33°/s) in the laminar space optic-flow experiment (see following text). Again we found a wide range of preferred pursuit directions, and the distribution does not differ significantly from uniform (P > 0.05, Rayleigh test). The mean population angle is 359° (or, equivalently, −1°), suggesting a trend in favor of ipsilateral pursuit directions (26/40 neurons); however, the distribution does not significantly favor ipsilateral pursuit directions over contralateral pursuit (P > 0.05, χ2 test). Finally, the preferred direction distributions at 2.58 and 9.22°/s have mean population angles of 76 and 5°, respectively. Neither distribution significantly differs from uniform (P > 0.05, Rayleigh test) or favors ipsilateral over contralateral pursuit (P > 0.05, χ2 test). However, the 2.58°/s distribution does significantly favor upward over downward pursuit (P < 0.05, χ2test).
We also observed that pursuing faster in the preferred direction tends to increase the neural discharge rate. To quantify this effect we regressed a line to each neuron's measured discharge rates as a function of pursuit speed in the preferred direction. We also used the background firing rate to represent the response at 0°/s and normalized all responses to the background firing rate. The distribution of pursuit slopes, across the population of neurons, had a median of 16.3% response increase per degree/second of pursuit speed increase (6.0% per °/s first quartile, 32.4% per °/s third quartile). This trend to increase response with pursuit speed was significant across the population (distribution greater than zero,P < 0.001, Wilcoxon t-test).
Pursuit compensation experiment
Having measured the spiral space optic-flow tuning and the pursuit direction tuning for each neuron, we next measured how each neuron responds to different focus locations in three conditions (fixed gaze, real pursuit, and simulated pursuit) and at three pursuit speeds (2.58, 5.05, and 9.22°/s).
We generated 11 different spiral space optic-flow displays based on the cardinal direction in spiral space (i.e., expansion, contraction, clockwise, or counterclockwise rotation) that was closest to a neuron's preferred spiral space pattern. For example, if a given neuron preferred a spiral space pattern of 15° (expansion with a slight counterclockwise rotation), we rounded this to 0° (a pure expansion pattern). We then generated the 11 optic-flow displays by positioning the FOE at 11 locations along an axis parallel to the preferred-null pursuit axis (see Visual stimuli). As another example, if the preferred spiral space pattern was 130° (counterclockwise rotation with some contraction), we rounded this to 90° (a pure counterclockwise rotation), and generated the 11 optic-flow displays by shifting the center of rotation along an axis orthogonal to the preferred-null pursuit axis. Recall that the center of rotation shifts orthogonally to the direction of pursuit.
We similarly rounded the preferred pursuit direction to the closest of the eight pursuit directions used in the preferred pursuit direction experiment. For example, if the estimated preferred pursuit direction was 55°, we rounded this to 45°, which corresponds to pursuit up and to the right. These rounded directions were used both as the pursuit direction and as the axis along which the stimuli varied in the pursuit compensation experiment. When preferred pursuit directions varied significantly across the three pursuit speeds, we rounded to the pursuit direction closest to the middle of the range.
Pursuit compensation experiment: basic response and eye movements
Figure 6 shows how one neuron in the pursuit compensation experiment responded as a function of the simulated heading direction (i.e., tuning curve of the location of the FOE on the screen), as well as the eye-movement traces for each trial. The tuning for heading direction in each behavioral condition can be seen directly from the peristimulus time histograms, which also illustrate the strong responses that were typical in this experiment. We will return to the neural tuning curves below, but first we investigate the fixation and pursuit performance in all pursuit compensation experiments.
To provide more information about eye movements than can be gleaned from visual inspection of one experiment's eye-movement records (e.g., Fig. 6), we analyzed eye-movement records from all trials in all pursuit compensation experiments (approximately 8,500 trials total). On each trial we extracted the 1.2 s of eye movement records corresponding to the time in the trial where either fixation or pursuit of a target moving at 2.58, 5.05, or 9.22°/s was required.
We first used a regression analysis to determine the slope of each trial's horizontal and vertical eye traces, which are recorded as separate channels. This yielded estimates of horizontal and vertical eye movement speeds. Second, we combined these horizontal and vertical speed estimates to yield the net eye movement speed, irrespective of eye movement direction. Next we grouped these eye-movement speed estimates according to the trial condition from which they originated: fixation, slow simulated pursuit, medium simulated pursuit, fast simulated pursuit, slow real pursuit, medium real pursuit, or fast real pursuit. Finally, as shown in Fig.7 A, we constructed histograms of the eye-movement speeds for each group of trials.
In fixation trials, and in all simulated pursuit trials, monkeys fixate a stationary target and therefore should have a 0°/s eye-movement speed. As shown in Fig. 7 A, eye-movement speed distributions in these four conditions are quite close to zero and are extremely similar. This suggests that fixation is quite stable and is as stable during simulated pursuit trials (where the stimulus frame moves across the screen) as in fixed-gaze trials (where the stimulus frame does not move). While this analysis is not very sensitive to saccades per se, it is sensitive to a change in fixation position during the trial that would result from a saccade or slow drift eye position.
In pursuit trials monkeys must track a target that moves at 2.58, 5.05, or 9.22°/s across the screen. While it is unlikely that monkeys pursue precisely at these speeds, the critical question is whether they pursue close to as fast as these targets. If monkeys pursue considerably slower than these targets then our theoretical calculations of FOE shifts, and consequently our estimate of FOE compensation, will be off. Figure 7 A plots histograms of the number of trials at each pursuit speed (0.2°/s binwidth) for the three pursuit target speeds. The median values of the slow, medium, and fast pursuit speed distributions are 2.75, 5.36, and 9.75°/s, respectively. These speed distribution median values are actually slightly larger than the target speeds and correspond to pursuit gains (i.e., eye speed divided by target speed) of 1.07, 1.06, and 1.06, respectively. Thus, on average, we are slightly underestimating the FOE compensation during pursuit because, on average, the eye is moving slightly faster than the target, resulting in a slightly greater actual FOE shift on the retina than we took into account.
In addition to confirming mean eye velocity during fixation and pursuit conditions (Fig. 7 A), we also computed instantaneous eye velocity by differentiating the eye position traces. We differentiated the eye position traces after applying a digital low-pass filter (0–20 Hz passband; 40 dB suppression for 40 Hz and above) designed to reduce noise at higher frequencies and to help saccades stand out. Figure7 B plots the unfiltered horizontal eye position and horizontal eye velocity traces for all 231 trials associated with one pursuit compensation experiment. We show only the horizontal eye traces here since real and simulated pursuit was along the horizontal axis in this experiment, and thus vertical eye movements were small. Consistent with the mean eye velocity analysis presented in Fig. 7 A, Fig. 7 B shows that the eye position smoothly ramps when the visual stimulus is displayed (0–1,200 ms) in real pursuit trials (column 1) and the eye position remains constant when the visual stimulus is displayed in simulated pursuit and fixation trials (column 3). The instantaneous eye velocity traces (columns 2 and 4) reveal that, following the onset of the visual stimulus (0 ms), there is a slight increase in eye velocity variability. This increased variability arises from microsaccades and, importantly, appears to be relatively small and brief for any given trial. Note that 33 trials are plotted in each panel in Fig. 7 B.
To compare the variability during the 0–1,200 ms data collection period in the real and simulated pursuit conditions to the variability in the fixation condition we 1) computed the SD of the eye velocity trace for each trial in each condition (e.g., fast simulated pursuit), 2) averaged the SDs of the eye velocity across all trials (e.g., 33) in each condition (e.g., fast simulated pursuit),3) repeated this for each of the 40 pursuit compensation experiments performed with the two monkeys and, finally, 4) compared the distribution of average eye velocity SDs in real and simulated pursuit conditions to that in the fixation condition. Specifically, we asked if the distribution of average eye velocity SDs in a given condition (e.g., fast simulated pursuit) was significantly different from the distribution of average eye velocity SDs in the fixation condition. We found that the fast, medium, and slow simulated pursuit distributions were not significantly different from the distribution from the fixation condition (P > 0.05,t-test). This indicates that the drifting visual motion stimulus present in simulated pursuit conditions did not significantly increase eye velocity (approximately 0°/s) compared with the fixation condition, which has a stationary visual motion stimulus. We also found that the slow pursuit condition distribution was not significantly different from the fixation distribution (P > 0.05) but that the medium and fast pursuit condition distributions are significantly different from the fixation distribution (P < 0.05). This indicates that slow pursuit eye movements also had low eye velocity variability, comparable to the variability in the fixation condition, but that in the faster two pursuit conditions the eye velocity variability was slightly elevated. Overall the mean eye velocity and instantaneous eye velocity analyses confirm that the two monkeys were adequately performing the behavioral tasks.
Pursuit compensation experiment: basic compensation effect
Returning to the neural response in the pursuit compensation experiment, Fig. 8 A plots tuning curves for the location of the FOE on the screen for the same neuron shown in Fig. 6. This neuron's preferred spiral space pattern was expansion (0°) and its preferred pursuit direction was down and to the right (315°). The simulated headings (FOEs) therefore ranged from up and to the left (−30° on plot's abscissa), through straight ahead (0° on abscissa), to down and right (30° on abscissa). The fixed gaze tuning curve (thick solid line) is peaked at 6° up and leftward (−6° on abscissa) and falls off rapidly for more peripheral headings. Recall that, when the eyes are still, the focus position on the retina corresponds to the true heading.
What should we expect the tuning curve to look like in the real pursuit condition? Since pursuit down and to the right shifts the retinal FOE in the same direction, we would expect the tuning curve to shift toward more up-left headings (left along abscissa in plot) if the neuron simply reports the focus position on the retina (see Fig. 1,A–C). In other words, if the neuron were to remain most sensitive to the 6° up and left focus position on the retina, then it should be most sensitive to a heading even more up and to the left during pursuit, because the pursuit would displace the retinal FOE back to 6°. On the other hand, if the neuron reports the true heading, as opposed to the retinal position of the FOE, then we should expect the tuning curve during pursuit to be aligned with the fixed gaze tuning curve (i.e., no shift).
In the case of the neuron represented in Fig. 8 A, we observed a clear shift of the tuning curve in the real pursuit condition (Fig. 8 A, thin solid line) with respect to the fixed gaze tuning curve. But how much did this tuning curve shift? Is the real pursuit tuning curve shifted by the amount predicted (24° at 9.22°/s pursuit) for a neuron reporting the position of the retinal FOE? Or is the shift less substantial, indicating that the neuron is reporting something closer to the true heading?
To answer this question we cross-correlated the fixed gaze and real pursuit tuning curves to produce a cross-correlogram (thin solid line) as shown in Fig. 8 B. The peak correlation coefficient occurs at an offset of −8°, which means that the real pursuit tuning curve is shifted 8° up and to the left with respect to the fixed gaze tuning curve. The fact that this 8° shift is less than the 24° predicted by the displacement of the retinal FOE indicates that this neuron has partially compensated for the effects of the pursuit eye movement; this is the basic compensation effect reported byBradley et al. (1996) and Shenoy et al. (1999). Equivalently stated, this neuron compensates for two-thirds (16°/24°) of the retinal focus displacement at this pursuit speed.
Another important question is whether this neuron compensates as much for simulated pursuit (when only visual/retinal cues are available) as for real pursuit (when both visual and nonvisual/extraretinal cues are available). Figure 8 A shows the tuning curve in the simulated pursuit condition (thin dashed line); recall that the retinal stimulus is the same as in the real pursuit condition but there are no efference copy or proprioceptive cues specifying eye movement. This tuning curve is displaced farther to the left (toward headings up and to the left) than the real pursuit curve; the peak of the cross-correlation curve is at −16° (Fig. 8 B, thin dashed line). During simulated pursuit this neuron compensates for only one-third (8°/24°) of the retinal focus displacement, substantially less than during real pursuit. Importantly, this neuron is able to compensate for some of the retinal FOE displacement using visual cues alone, which is consistent with a previous physiological report (Shenoy et al. 1999) but differs from human psychophysical findings (see discussion).
Pursuit compensation experiment: compensation patterns in single neurons
The central question of this study is whether MSTd neurons modify their retinal focus tuning curves more at faster pursuit speeds. To answer this question we have plotted the same neuron's tuning curves for pursuit speeds of 5.05 and 2.58°/s in Fig. 8 C andE, respectively. By inspecting these tuning curves, and the associated cross-correlograms (Fig. 8, D and F), it is clear that these tuning curves are less spread apart than the tuning curves for the fastest pursuit speed (Fig. 8 A, 9.22°/s). However, some of this reduction is expected, because the retinal focus displacements are smaller at slower pursuit speeds: 12° for 5.05°/s pursuit and 6° for 2.58°/s pursuit.
To more readily compare a neuron's behavior across pursuit speed, we plot the compensatory shift or compensation (Fig.9 A) as a function of pursuit speed in the real pursuit and simulated pursuit conditions. The compensation is equal to the difference between the calculated displacement of the retinal focus at each pursuit speed and the observed shift of the neuron's tuning curve relative to the fixed gaze tuning curve. Consider, for example, real pursuit at 9.22°/s, which displaces the retinal focus by 24°. Since the 9.22°/s real pursuit tuning curve shift for this neuron is only 8°, the compensation is 24° − 8° = 16° and is plotted as such in Fig. 9 A. The values for the other pursuit speeds (solid circles) and for the simulated pursuit condition (open circles) are plotted similarly. Finally, the two heavy black lines represent perfect (steep) and zero (horizontal) compensation, respectively.
Figure 9 A concisely represents this neuron's pattern of compensation across pursuit speeds, for both the real and simulated pursuit conditions. Two important features of this graph are representative of the population of neurons. First, compensation was greater at faster pursuit speeds. To quantify this trend, we regressed lines (Fig. 9 A, gray lines) through the real and simulated pursuit data. Both slopes were positive, indicating increasing compensation with increasing pursuit speed. Second, compensation increased more quickly with pursuit speed during real than during simulated pursuit. We converted the slopes of these two lines to CI, as defined in Data analysis, by dividing by the slope of the perfect compensation line. This neuron's CI during real pursuit is 66.9%, while its simulated pursuit CI is only 34.8%.
A third property of the graph in Fig. 9 A that does not reflect a universal property of the population is that this neuron's compensatory shift is roughly proportional to pursuit speed (in other words, its CI is roughly constant across pursuit speed). While a few neurons exhibited this pattern, a counterexample can be seen in Fig.9 B, which plots the compensation pattern for another neuron. This neuron appears to compensate substantially only at the fastest pursuit speed, and then only in the real pursuit condition.
Pursuit compensation experiment: compensation trends in the population
To determine whether the two properties mentioned above—greater compensation at faster pursuit speeds and greater CIs during real than simulated pursuit—are present across the population of neurons, we formed separate population histograms of the compensation indices for the real and simulated pursuit conditions.
Figure 10 A shows that most neurons in both conditions have positive CIs, indicating that the compensatory tuning curve shift increases with pursuit speed. The median CIs for real and simulated pursuit are 50.7 and 29.6%, respectively, and both distribution means are significantly greater than zero (P < 0.001, Wilcoxon t-test). Therefore it is generally the case that MSTd neurons modify their focus tuning curves more at higher pursuit speeds.
The second question is whether compensation indices were greater across the population during real than simulated pursuit. Figure10 B is a histogram of the cell-by-cell difference between the CIs for real and simulated pursuit. The median CI difference is 20.8% and is significantly greater than zero (P < 0.05, Wilcoxon paired t-test), indicating that in most neurons the compensation index was greater during real than simulated pursuit. Direct comparison of the distributions of the two CIs (Fig.10 A), without forming the cell-by-cell difference, yields the same conclusion: the distribution of compensation indices from the real pursuit condition is significantly greater than the simulated pursuit distribution (P < 0.05, Mann-Whitney test).
We can also calculate and plot the population mediancompensatory shift during real and simulated pursuit as a function of pursuit speed (Fig. 11 A). Error bars correspond to first and third quartiles. This graph, together with Fig. 11 B, makes the same two points expounded in Fig. 10 in a very straightforward manner. All six compensation distributions in Fig. 11 A are significantly different from, and greater than, zero (P < 0.05, Wilcoxont-test) and real pursuit distribution means are significantly different from, and greater than, simulated pursuit distribution means at all three pursuit speeds considered separately (P < 0.05, Wilcoxon paired t-test). To quantify the rate of compensation growth as a function of pursuit speed, we regressed lines to all real and simulated pursuit condition data, considered separately, as shown in Fig. 11 B. Symbols indicate distribution means and regression lines are shown in gray. The real pursuit compensation index is 55.3% (38.0–72.6%, 95% confidence interval) and the simulated pursuit compensation index is 42.3% (24.2–59.9%, 95% confidence interval). Both slopes are significantly greater than zero, and the real pursuit slope is steeper than the simulated pursuit slope.
We wondered how neurons are able to modify their retinal focus tuning curves more at faster pursuit speeds and how retinal and extraretinal cues are integrated in neural pursuit compensation. To address the first question, we tested the idea that the increase in pursuit activity with increasing pursuit speed that we observed in the preferred pursuit direction experiment could drive increasing compensatory tuning curve shifts. However, we did not find a significant correlation between the rate of increase of pursuit activity and the increase of compensation during real pursuit (P > 0.05, Spearman). We also attempted to relate changes in pursuit-related activity to the portion of the compensatory shift attributable to extraretinal signals alone; however, we failed to find a significant correlation between the rates of pursuit activity increase and the difference in the increase in compensation between real and simulated pursuit (P > 0.05, Spearman). Another possibility is that increasing compensatory shifts are driven by the faster retinal motion that accompanies faster real and simulated pursuit. We did not measure neural responses to different speeds of visual motion because of the already demanding number of experimental trials, so we could not investigate this possibility.
To address how neurons integrate retinal and extraretinal cues for pursuit compensation, we investigated the relationship between single neurons' preferred-laminar motion directions and preferred pursuit directions. When a human or monkey makes a leftward eye movement, for example, this creates rightward laminar motion on the retina. Therefore a neuron that integrates retinal and extraretinal cues specifying leftward pursuit should prefer leftward pursuit (independently of any visual responses) and rightward laminar motion (independently of any pursuit-related activity). Figure 12shows that this is exactly what we found across the population of neurons. For each neuron we calculated the angular difference between the preferred pursuit direction at a pursuit speed of 5.05°/s and the preferred direction of laminar flow at a speed of 4.33°/s. The population histogram of these differences (Fig. 12) has a strong, statistically significant peak around 180° (P < 0.01, nonparametric angular–angular correlation), suggesting that MSTd neurons generally respond best to the combination of retinal and extraretinal signals created by pursuit eye movements.
Compensation and spiral space preferred patterns
Finally, we wanted to examine whether neurons preferring different spiral space motion patterns compensated to similar extents. Our previous studies of compensation in MSTd have found this to be the case and, therefore, we have grouped all neurons together for analysis thus far. First, our population consists of neurons preferring expansion (18/40), counterclockwise rotation (10/40), contraction (5/40), and clockwise rotation (7/40). We did not select neurons based on preferred optic-flow pattern; instead we attempted to record from a random sample to investigate how MSTd as a whole participates in compensating for the visual effects of pursuit.
Second, we analyzed neurons of the same category (i.e., expansion, counterclockwise rotation, contraction, and clockwise rotation neurons) to determine whether category has a substantial influence on compensation. We examined the compensation during real pursuit at all three speeds, compensation during simulated pursuit at all three speeds, and the slope with which compensation increases with pursuit speed for both real pursuit and simulated pursuit. The results of these analyses are summarized in Table 1. We did not find a clear difference in the extent of compensation when comparing the compensation measures for expansion, counterclockwise rotation, contraction, and clockwise rotation neurons. Although the distribution means appear different, the SDs are quite large. We have found previously that, even with considerably more MSTd neurons (139), and thus more neurons in each category (41% expansion, 33% contraction, and 27% rotation), the compensation at a given pursuit speed is very similar for each category (18 ± 3° expansion, 15 ± 3° contraction, and 17 ± 4° rotation) (Bradley et al. 1996). For these reasons we combined data across these preferred optic-flow categories for all previous analyses.
In the present study we asked if MSTd neurons are able to contend with the increase in retinal FOE displacement that accompanies faster pursuit (Fig. 1). After characterizing the basic visual and pursuit-related responses of each neuron, we measured tuning curves for the direction of heading during pursuit (Figs. 6 and 8). We found that, both in individual neurons (Fig. 9) and across the population (Figs. 10and 11), MSTd cells modify their tuning for the retinal focus positionmore at faster pursuit speeds than at slower pursuit speeds. This is what one would expect of a cortical area that reports heading and other parameters of self- or object motion (e.g., center of rotation) regardless of pursuit speed. Moreover, we found that retinal cues alone are capable of driving increased levels of pursuit compensation at faster pursuit speeds, although not as much as retinal and extraretinal cues in combination. This finding suggests that, under certain conditions, MSTd neurons compensate for pursuit eye movementsmore effectively than human observers do (see below). Finally, we observed a systematic anticorrelation between neurons' preferred directions of pursuit and laminar retinal motion, as would be expected for neurons that combine retinal and extraretinal signals to drive pursuit compensation.
Below we discuss the results and offer further suggestions about how MSTd may participate in the computation and representation of self-motion, but first we briefly review some of the response properties that implicate MSTd as playing an important role in self-motion processing.
Cortical area MSTd and self-motion processing
In this study we focused on how MSTd responds to visual motion patterns and pursuit eye movements, and then employs these response properties to contend with visual motion processing during pursuit eye movements. We recorded from MSTd neurons because MSTd has a wide array of fundamental response properties that are consistent with a role in self-motion processing. Although it is beyond the scope of the current report to review this literature, we briefly list some of the receptive field specializations and extraretinal inputs that implicate MSTd in self-motion processing: 1) large receptive fields,2) selectivity for particular optic-flow patterns (e.g., expanding or unidirectional motion) (Duffy and Wurtz 1991a,b; Graziano et al. 1994; Komatsu and Wurtz 1988a,b; Lagae et al. 1994;Lappe et al. 1996; Orban et al. 1992;Raiguel et al. 1997; Saito et al. 1986;Sakata et al. 1985, 1994; Tanaka et al. 1986,1989a,b; Tanaka and Saito 1989),3) optic-flow selectivity position invariance (i.e., the preferred type of flow pattern does not change across the receptive field) (Duffy and Wurtz 1991b; Graziano et al. 1994; Lagae et al. 1994; Orban et al. 1992), 4) optic-flow selectivity form/cue invariance (i.e., the preferred type of flow does not depend on the image elements carrying the motion signal) (Geesaman and Andersen 1996), 5) optic-flow selectivity stimulus-size invariance (Graziano et al. 1994), 6) tuning for the retinal position of the FOE (Duffy and Wurtz 1995), 7) tuning for the rate of expansion (Duffy and Wurtz 1997a), 8) selectivity for binocular disparity (Roy et al. 1992;Roy and Wurtz 1990), 9) direction- and speed-tuned extraretinal smooth pursuit signals (Bradley et al. 1996; Erickson and Thier 1991; Kawano et al. 1984, 1994; Newsome et al. 1988),10) tuning for the position of the eye in the head (Bremmer et al. 1997; Squatrito and Maioli 1996,1997), and finally, 11) microstimulation of expansion-selective columns biases behavioral estimates of the direction of motion or self-motion (Britten 1998;Britten and van Wezel 1998; Celebrini and Newsome 1995; Geesaman et al. 1997).
Preferred optic-flow experiment
We found that MSTd neurons are typically selective for both spiral space patterns (Fig. 2) and laminar space directions (Fig. 4). This is consistent with previous reports, as is our observation that neurons tend to respond best to spiral space patterns containing an expansion component (Fig. 3 A) (Duffy and Wurtz 1991a,b;Graziano et al. 1994). We found that slightly more than half of the population (25/40) prefers contralateral laminar motions, but this trend was not statistically significant (Fig. 3 B). A previous study has also reported a contralateral visual motion bias in MSTd (Komatsu and Wurtz 1988a).
Preferred pursuit direction experiment
We found that MSTd neurons are selective for the direction of pursuit (Fig. 5), as has been reported previously (Bradley et al. 1996; Komatsu and Wurtz 1998a; Shenoy et al. 1999). The distribution of preferred directions was not distinguishable from uniform (Fig. 3 C) except at the slowest pursuit speed, at which we found a bias toward upward pursuit. This result is in general agreement with a previous report that there is a weak preference for contralateral pursuit directions (Komatsu and Wurtz 1988a). Of particular interest for this study of speed effects is that the preferred pursuit directions of individual neurons are quite similar across pursuit speeds (approximately 75% have a range of 90° or less).
Finally, we observed that neural discharge rates tend to increase with the speed of pursuit. Komatsu and Wurtz (1988b) have reported that MSTd neurons also increase their discharge rate in response to faster laminar motion provided the stimulus size is within a certain range (see following text). We speculate that increased neural firing rates during faster pursuit and during faster laminar motion exist to drive increased levels of pursuit compensation. Testing this idea would require further investigation of speed tuning for pursuit and laminar motion in individual MSTd neurons.
Pursuit compensation experiment
We found that individual neurons are able to increase their compensatory tuning curve shift as the speed of pursuit increases (Fig.9). In fact, most MSTd neurons (34/40) increase their compensation in this fashion. We also found that most (34/40) individual neurons are able to increase their compensatory shift as the speed ofsimulated pursuit increases (Fig. 9). Although some neurons' compensatory shift increased in direct proportion with pursuit speed (Fig. 9 A), most did not exhibit such a simple, linear dependence (Fig. 9 B). Across our entire population, however, we found a roughly linear increase in the compensatory shift as a function of pursuit speed (Fig. 11, A andB). This finding is particularly elegant from the perspective of a downstream cortical area reading out a representation of heading in MSTd: regardless of pursuit speed, the MSTd representation has compensated on average for approximately 55% of the FOE displacement. About 42% of the FOE displacement, regardless of pursuit speed, is compensated for by retinal cues alone.
Comparing these results to recent human psychophysical studies reveals similarities and substantial differences. Human observers judge their heading only moderately accurately during real pursuit across simulated approaches to a wall (average compensation index of 39%) (Crowell et al. 1998b, 1999; Crowell and Andersen 2001). This psychophysical study is most comparable to the present study. The average compensation index was lower (only 21%) for pursuit across rotary patterns. The compensation index from the present study (51%, Fig. 10 A; 55%, Fig. 11 B) includes a few rotation-selective neurons, but our previous reports indicate that expansion-, contraction-, and rotation-selective neurons compensate similarly (Bradley et al. 1996; Shenoy et al. 1999). Therefore it appears that human psychophysical performance is slightly worse (39%) than neural performance (51 or 55%). However, the striking difference is that compensation was completely absent (0%) in the psychophysical simulated pursuit condition (Crowell et al. 1998b, 1999; Crowell and Andersen 2001), while the compensation index from the present study was quite substantial (30%, Fig. 10 A; 42%, Fig. 11 B) under similar conditions. A possible reason for this difference is the difference in display size; displays subtended 40 × 40° in the psychophysical experiments, but only 20 × 20° in the present physiology experiments. We have previously found that larger displays (50° × 50°) are less effective at invoking neural pursuit compensation during simulated pursuit (Bradley et al. 1996). This result is also counterintuitive: computational studies suggest that larger fields of view are required for retinally based pursuit compensation (Koenderink and van Doorn 1987), and accurate psychophysical self-motion judgments during simulated pursuit and approach to a wall have only been reported for very large displays (Grigo and Lappe 1999). We speculate that our small display size makes the pursuit-related motion of the stimulus boundary more salient and causes it to fall more completely within a neuron's receptive field.
As we described above, retinal and extraretinal pursuit-related signals are integrated to drive greater levels of pursuit compensation than either signal may be capable of alone. Across the population, retinal cues alone cause compensation for 30% (Fig.10 A) of the pursuit-related focus displacement, whereas the combination of retinal and extraretinal signals leads to compensation for 51% (Fig. 10 A) of the total focus shift. Importantly, it is not possible to truly assess what the purely extraretinal contribution would be as retinal and extraretinal signals may well combine nonlinearly. Therefore we interpret the median, cell-by-cell difference between real pursuit and simulated pursuit CIs (21%, Fig.10 B) only as the excess compensation observed when extraretinal signals are also present.
A possible mechanism for this integration was discovered bySakata et al. (1978, 1983) and was more fully investigated by Komatsu and Wurtz (1988b). When one tracks a nearby object, the image of the background moves across the retina in the direction opposite that of the eye movement. If MSTd neurons preferred opposite directions of pursuit and visual motion, then their responses would be enhanced by the normal combination of retinal and extraretinal signals that arises during pursuit. Komatsu and Wurtz found that most MSTd neurons do in fact have such a systematic antialignment of preferred pursuit and visual motion directions, consistent with their observation that responses arestronger while pursuing across a stationary visible background than during pursuit in the dark. We also found a tendency toward opposite preferred pursuit and visual motion directions (Fig.12). We hypothesize that this systematic antialignment exists for the functional purpose of mediating pursuit compensation, with retinal and extraretinal cues combining and ultimately driving shifts in tuning curves.
An important point to consider in comparing our results with those ofKomatsu and Wurtz (1988b) is the stimulus size and speed regime. Komatsu and Wurtz reported that MSTd neurons' preferred visual motion directions are a function of both stimulus size and visual motion speed: the preferred direction of visual motion is aligned with the preferred pursuit direction for small stimuli, butreverses as the size of the stimulus increases beyond a certain size that depends on its speed. They reported that the preferred direction of motion reverses at a diameter of 20–30° for motion at approximately 10°/s; the cross-over size decreases for slower visual motion speeds. Thus the fact that we used a stimulus size of 20° and a pursuit speed of 4.33°/s in our preferred optic-flow experiment suggests that our stimuli were probably larger than the cross-over size. In other words, we believe we were operating within the regime under which Komatsu and Wurtz also found opposite preferred pursuit and visual motion directions.
In summary, these results indicate that many MSTd neurons change their tuning curves for the retinal location of the FOE, or center of rotation, in a manner that compensates for the effects of changes in pursuit velocity. We previously reported that these tuning curve shifts were in the appropriate direction for pursuit compensation (Bradley et al. 1996; Shenoy et al. 1999); here we report that the shifts increase in magnitude with increases in pursuit speed. Taken together, these findings further implicate MSTd as a critical stage in the computation of egomotion.
We thank Dr. D. C. Bradley for scientific discussions, B. Gillikin for veterinary assistance, and Dr. M. Sahani for developing the real-time control software HYDRA2. We also thank C. Reyes for administrative assistance and V. Shcherbatyuk for computer management.
This work was supported in part by National Eye Institute Grant EY-07492, postdoctoral grant EY-06752 to K.V. Shenoy, the Sloan Foundation for Theoretical Neurobiology at Caltech, the Office of Naval Research, and the Human Frontiers Scientific Program.
Present addresses: K. V. Shenoy, Department of Electrical Engineering and Neurosciences Program, Stanford University, Stanford, CA 94305; JA Crowell, Integrated Systems Lab, Beckman Institute for Advance Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL 61801.
Address for reprint requests: R. A. Andersen, James G. Boswell Professor of Neuroscience, Mail Code 216–76, Division of Biology, California Institute of Technology, Pasadena, CA 91125 (E-mail:).
- Copyright © 2002 The American Physiological Society