INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Optic flow is the patterned visual motion seen by moving observers (Gibson 1966) that combines with vestibular signals (Berthoz et al. 1975) to support heading perception (Israel et al. 1993) during movement in darkness or light. Neurons in dorsal medial superior temporal cortex (MSTd) respond to simulated optic flow (Duffy and Wurtz 1991; Graziano et al. 1994; Orban et al. 1992; Saito et al. 1986; Tanaka and Saito 1989) with heading preferences (Duffy and Wurtz 1995) that are maintained with complex visual scenes (Upadhyay et al. 2000). MSTd neurons also respond to rotational (Thier and Erickson 1992) and translational (Duffy 1998) self-movement, likely transduced by the canals and otoliths.

MSTd neurons combine pursuit signals (Komatsu and Wurtz 1988b) with optic flow responses (Bradley et al. 1996) for the population encoding of heading (Page and Duffy 1999). This may overcome the distorting effects of pursuit eye movements on the retinal image of optic flow (Longuet-Higgins and Prazdny 1980) in a manner that is consistent with human perceptual capacities (Banks et al. 1996; Warren and Hannon 1990).

MSTd's neuronal responses to optic flow and vestibular signals (Duffy 1998) might be similarly integrated to enhance population heading estimation in light. Alternatively, these signals might be exchanged to maintain population heading estimation in light or darkness. The former hypothesis is consistent with the classical view of signal interactions in association cortex. The latter hypothesis suggests that these areas act as "auxiliary cortex," switching between operational modes to focus on the best available source of information (Duffy 1984).

We now test the hypothesis that MSTd neurons combine visual and vestibular responses to code the heading of self-movement. We recorded MSTd neuronal responses to combined visual, vestibular, and pursuit stimuli resulting from whole body translational self-movement in light and darkness, during fixation and pursuit. We find that MSTd neurons integrate sensory signals and maintain population heading encoding under diverse, naturalistic self-movement conditions using the most robust heading cue available.

These results have been briefly described previously (Page and Duffy 2001).

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Animal preparation

The responses of individual neurons were recorded from four cerebral hemispheres of three adult Rhesus monkeys. Surgical preparation began with sedation by ketamine (15 mg/kg, im) and Robinul (0.011 mg/kg, im), followed by venous catheterization, endotracheal intubation, and general anesthesia using inhaled isoflurane. Scleral search coils were implanted surgically around the limbus in both eyes (Judge et al. 1980). A head holder and bilateral recording cylinders were placed over 2-cm trephine holes centered above area MSTd (AP, 2 mm; ML, ±15 mm; angle, 0°) and encased in a dental acrylic cap. Postoperative analgesia with banamine (1 mg/kg, im) was administered as judged appropriate by veterinary staff. All protocols were approved by the University of Rochester Committee on Animal Research and complied with Public Health Service and Society for Neuroscience policy on laboratory animals.

Following recovery from surgery, the monkeys were trained to sit in a primate chair and perform a visual fixation task that was monitored with magnetic search coils (Robinson 1963). The frame of the eye coil system was covered by black, opaque plastic that enclosed the animal and limited its field of view to the projection screen or wall-mounted light array. Trials began with a stationary, red fixation point centered on the screen or wall. If the monkey maintained fixation (±3°) throughout the 5-s trial, an auditory tone was sounded and liquid reward was dispensed. Recording sessions were begun after the animal consistently completed trials with an accuracy of >90%.

Stimulus conditions

Ten stimulus conditions were used to examine factors influencing heading selective responses. Each condition included stimuli for eight observer movement directions. Recording of an isolated neuron always began with six, pseudo-randomly interleaved stimulus conditions. These consisted of four basic conditions of naturalistic self-movement: 1) gaze fixed movement in darkness, 2) gaze fixed movement in light, 3) landmark pursuit movement in darkness, and 4) landmark pursuit movement in light, and also included 5) pursuit in darkness without movement and 6) pursuit in light without movement. Subsequent studies of isolated neurons included four additional conditions in subsets of the neuron sample: 1) gaze fixed movement in darkness without a fixation point, 2) video-simulated optic flow without movement, 3) gaze fixed movement in light with two depth-planes, and 4) landmark pursuit movement in light with two depth-planes. The monkey's head was directed toward the wall-mounted light array except during optic flow simulation, when the monkey's chair was rotated 180° to face the rear projection screen with the same neutral head-on-body orientation.

Gaze fixed movement

During gaze fixed trials, the target was positioned directly in front of the monkey and moved horizontally with the monkey so that no conjugate eye movements were necessary to maintain fixation (Fig. 1A), although small (<0.5°) vergence movements are evoked in some movement directions. In a small number of neurons, the fixation point was shut-off during movement. These trials began with centered fixation that was maintained after the fixation point was extinguished and sled movement proceeded in complete darkness.

View larger version (24K): [in this window] [in a new window]   Fig. 1. Dorsal medial superior temporal cortex (MSTd) neurons were recorded in two naturalistic self-movement conditions: (A) gaze fixed straight ahead and (B) pursuit to fixate a stationary landmark. The 2-axis monkey sled moved across the room in 8 directions; right-forward movement is illustrated. In both conditions, the monkey continuously viewed the far wall that was covered by 600 small white lights. A: in the gaze fixed condition, the monkey maintained neutral gaze throughout the movement by fixating a target that moved to remain directly in front on the animal. B: in the landmark pursuit condition, the monkey made pursuit eye movements to maintain its gaze on an earth-fixed target in the middle of the wall. C: speed profile of sled movement for all 8 directions included 1 s of acceleration, 3 s of steady-speed movement, and 1 s of deceleration.

Landmark pursuit movement

During landmark pursuit trials, the target remained at the center of the wall (Fig. 1B). To maintain fixation during lateral self-movement, the monkey made smooth pursuit eye movements to continue fixating the target. The ensuing rotation of the eye added rotation movement to the retinal image of the optic flow, creating a complex pattern of visual motion that transitions to stationary points at the pursuit target (Fig. 2B). Pursuit dynamics depended on the speed and direction of movement and distance from the target; self-movement far from the target elicited much slower pursuit than the movement near the wall (Fig. 3, A and B). No pursuit was necessary during forward or backward movements, so these trials were identical in the gaze fixed and landmark pursuit conditions. The fixation point was remote, ranging from a distance of 1.6 to 2.8 m with vergence angles from 1.14° to 0.68°, yielding a maximum vergence velocity of 0.09°/s.

View larger version (27K): [in this window] [in a new window]   Fig. 2. The gaze fixed and landmark pursuit conditions create very different patterns of visual motion on the retina. A and B: each box represents the retinal flow field containing visual motion (thin arrows) during self-movement in the indicated direction (thick arrows). A: in the gaze fixed condition, the monkey maintained neutral gaze (filled circle) and the retinal flow field reflected the heading of self-movement. B: in the landmark pursuit condition, the monkey made a smooth pursuit eye movement (open arrows) to maintain fixation on an earth-fixed target. This created a retinal flow field that combined the effects of self-movement and eye rotation to yield a complex moving pattern that transitions to stationary points near the pursuit target.

View larger version (32K): [in this window] [in a new window]   Fig. 3. Horizontal eye movements during pursuit responses. A: calculated ideal eye movements during forward and backward movement required steady gaze (flat, solid lines). Lateral movements required symmetric movements of intermediate amplitude (curved, solid lines). Oblique movements required greater deviation when the animal was nearer the target image on the wall (dashed lines). B: records from gaze fixed movement in light yielded pursuit gains averaging 0.99 ± 0.02. C: records from landmark pursuit of a stationary target during movement in light yielded pursuit gains averaging 0.95 ± 0.04. Each record shows eye movements during12 trials recorded on 2 typical days.

Pursuit only

During pursuit only trials, the pursuit dynamics made in landmark pursuit trials were replicated by moving the pursuit target while the platform remained stationary at the center position. The target moved across the wall for 5 s with the exact profile necessary to induce pursuit movements comparable to those in landmark pursuit trials (Fig. 3, B and C). The fixation point was remote, ranging from a distance of 2.20 to 2.28 m with vergence angles from 0.80° to 0.83°, yielding a maximum vergence velocity of 0.006°/s. Responses to these pursuit movements are represented in the vector domain of ground plane self-movement to facilitate comparison to the landmark pursuit responses.

Optic flow only

During the optic flow only trials, the monkey was turned to view a rear-projection (ECP4100, Electrohome) tangent screen (Fig. 1A). Eight optic flow stimuli were presented in a pseudo-random sequence. These stimuli simulated the visual motion seen by an observer during the eight directions of ground plane linear translation presented in movement trials (Fig. 2A). The movement platform and fixation target were stationary during these 1-s stimuli.

Translational movement stimuli

The monkey chair, eye coil, and video display systems were mounted on a 1 × 2 m platform on a double-rail drive apparatus (Acutronics; Fig. 1). Platform movements were controlled with position feedback from the drive motors, sampled at 125 Hz, and stored as an analog record. Stimulus presentation and experimental conditions were controlled by the PC-based real-time experimental system (Hays et al. 1982).

At the start of each movement trial, the platform was moved 60 cm from the center of the room to one of eight starting positions. The platform then moved on a straight path through the center to stop after a total excursion of 120 cm. During these movements, the platform accelerated at 30 cm/s² for 1 s, maintained a constant velocity of 30 cm/s for 3 s, and then decelerated at 30 cm/s² for 1 s (Fig. 1C). Such movements are well above human vestibular thresholds (Benson et al. 1986) and allow stable neuron recordings.

Visual stimuli

The wall-mounted light array consisted of 600 small, white lights uniformly distributed across a 322 × 168 cm wall that was 220 cm from the monkey's centered position. The lights, 3 × 6 mm incandescent bulbs subtending 0.2-0.4^o depending on the viewer's distance and relative orientation, were at 7.36 cd/m² against a 0.7-cd/m² background. These lights were stationary so any visual motion was the result of observer movement. All of the lights were always in the monkey's field of view, although the array subtended different horizontal angles depending on the distance of the sled from the wall.

In a smaller sample of neurons, we separated two co-extensive sets of lights in the single depth-plane light array to create two light arrays; the second was mounted parallel to the first and 1 m closer to the animal. All of the lights of the second array were always visible to the animal with the horizontal extent of the display varying with distance. When both light arrays were used, they created two distinct depth-planes that presented robust distance cues during sled movement.

The fixation/pursuit target was produced by a red laser light-emitting diode and was reflected by a mirror-galvanometer on to a smooth black surface on the wall. It moved to stay in front of the monkey in gaze fixed trials, remained stationary in the center of the wall in landmark pursuit trials, and moved past the stationary monkey in pursuit trials. This target was the only visible light in the room during darkness trials.

The simulated optic flow stimuli consisted of 500 white dots (0.19° at 2.61 cd/m²) on a black background (0.18 cd/m²), stimulating the central 90° × 90° of the visual field. All dots were replaced by lifetime expiration (33-1,000 ms) or by a smoothing algorithm that maintained a uniform and consistent dot density across the stimulus in all frames. Dots for these radial patterns accelerated across the screen with speeds that were a sine × cosine function of each dot's angular distance from the focus-of-expansion as viewed by the monkey. Speeds were adjusted to maintain an average speed of 40^o/s across the stimulus. These parameters matched those used in our previous studies of MSTd neuronal responses.

Neuron recording

Single neurons were recorded using epoxy-coated tungsten microelectrodes (FHC and Microprobe) that were passed through a trans-dural guide tube positioned within the recording cylinder. Neuronal activity was monitored to determine transitions between gray and white matter and the relative depth of physiological landmarks.

After isolating a neuron, the boundaries of its receptive field were mapped by a handheld projector. MSTd neurons were identified by their physiologic characteristics: large receptive fields (>20° × >20°) that included the fixation point, a preference for large moving patterns rather than moving bars or spots, and direction-selective responses (Duffy and Wurtz 1991, 1995; Komatsu and Wurtz 1988a). A dual window discriminator was used to digitize neuronal discharge times that were stored with stimulus and behavioral event markers for off-line analysis.

Data analysis

Neuronal activity was averaged over six stimulus presentations for each stimulus. To analyze response dynamics to the 5-s movement stimuli, firing rates were averaged in ten 500-ms intervals. When comparing movement and optic flow responses, firing rates were averaged in five 1-s movement intervals to match the 1-s optic flow stimuli.

Neuronal responses to a particular stimulus were tested for statistically significant differences from control activity (Student's t-test, P  0.05) that was recorded in darkness without movement. The neuronal responses were displayed as polar plots to illustrate the strength and directionality of the responses. In the polar plots for all stimulus conditions, the eight thin radial lines represent responses to each of the corresponding eight directions of real or simulated self-movement. The length of the eight radial lines is proportionate to the neuronal firing rate during the corresponding movement direction. The thick radial line in each polar plot indicates the vector sum of the eight individual response vectors. Circular statistical analyses for data sampled at 45° intervals around 360° were used (Batschelet 1981), including the Rayleigh Z statistic to test for significant directionality in a circular response profile (P  0.05).

Population vectors

We followed the approach of Georgopoulos et al. (1986) by combining the vectors from the response profiles of each neuron. The following equation was used to derive a neuronal population response for each test direction as the sum, across all neurons, of the dot products of each neuron's preferred direction and its firing rate for the test direction

(1)

where PV is the neuronal population vector, i is one of eight movement directions, j is the self-movement condition, k is one of N neurons, R_ijk is the response amplitude of the kth neuron to movement direction i and condition j, and RC_k is the response amplitude of the kth neuron during control trials with no movement and display lights turned off. D_k is the preferred direction for the movement response for the kth neuron. The preferred direction was derived either from the particular test condition under analysis (Figs. 16 and 17) or from the sum of all four self-movement conditions (gaze fixed and landmark pursuit, movement in darkness, and movement in light; Figs. 18 and 19).

Population vectors were displayed in polar plots with thin polar limbs for each neuron's vector and a thick polar limb for the population. The direction of the vector matches the angle at which its limb emanates from the origin. The length of each limb indicates the strength of the response.

Recording sites

The stereotaxic positioning of the recording chambers and the depths of microelectrode penetrations directed neuron recordings into cortical area MSTd. At the end of experiments, electrolytic marks (25 µA × 25 s) were made along the penetration tracks in three guide tubes in three hemispheres of two monkeys. After perfusing the animal and fixing the tissue, posterior cortical blocks were cut into 50 µm thick sections. Every fourth and fifth section was stained by the Nissl and Luxol Fast Blue methods, respectively. The electrolytic lesions were identified relative to anatomic landmarks to extrapolate the position of the recording sites. Histologic analysis indicates that the neurons studied were located in the anterior bank of the superior temporal sulcus that is included in MSTd (Komatsu and Wurtz 1988b).

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

We studied the responses of 130 neurons from four hemispheres of three Rhesus monkeys. Recordings sites were on the anterior bank of the superior temporal sulcus. All neurons had large receptive fields, often more than a quadrant of the visual field, including the fovea. They preferred large pattern motion to moving bars and showed strong direction selectivity, characteristics typical of MSTd neurons (Komatsu and Wurtz 1988b).

We used four naturalistic self-movement conditions to test heading selectivity: gaze fixed movement in darkness, gaze fixed movement in light, landmark pursuit movement in darkness, and landmark pursuit movement in light. The gaze fixed (Fig. 1A) and landmark pursuit (Fig. 1B) conditions create different retinal images of optic flow. With gaze fixed (Fig. 2A), the center of motion in the retinal image of optic flow indicates the axis of observer movement as a focus-of-expansion (FOE) during forward movement and a focus-of-contraction (FOC) during backward movement. With landmark pursuit (Fig. 2B), the retinal image of optic flow is combined with visual motion from eye rotation during pursuit (Table 1; Fig. 3) to shift the image of the FOE/FOC on the retina.

Responses to self-movement stimuli were divided into five 1-s intervals to assess the impact of accelerating, steady-speed, and decelerating movement (Fig. 1C). All stimulus intervals evoked significant heading selective responses (Z value of circular distribution with P < 0.05) from similar percentages of neurons (Table 2). About one-quarter (18-36%) of the neurons responded to movement in darkness and twice as many (51-60%) responded to movement in light. Overall, 82% (89/109) of the neurons yielded significant heading selectivity in at least one interval of at least one of the four basic translational self-movement stimulus conditions.

Gaze fixed movement in darkness

We recorded MSTd neuronal activity during gaze fixed movement in darkness to assess vestibular responsiveness. Figure 4A shows the directional responses of a neuron with transient activation during gaze fixed movement in darkness. This neuron preferred left-forward movement in darkness during acceleration (0-1 s) that reversed to become a preference for right-backward movement during deceleration (4-5 s). This reversal of direction preference from acceleration to deceleration suggests a vestibular response mechanism.

View larger version (25K): [in this window] [in a new window]   Fig. 4. The time-course of MSTd neuronal responses to gaze fixed movement in darkness. A and B: directional responses shown as spike density histograms (SDHs) averaging responses to 6 repetitions of the 5 s self-movement stimuli in the direction indicated by the SDH's position relative to the center of the figure. Below the SDHs are polar plots for activity in each of the five 1 s stimulus intervals. Polar plots show the relevant self-movement stimulus direction as radial limb direction and response amplitude as radial limb length, with significant responses indicated by a filled circle, the net vector shown by a heavy line, and control activity indicated by a dashed circle. A: neuron showing transient responses to acceleration and deceleration. Acceleration evokes left-forward heading selectivity during the 1st 1 s of movement in darkness. Deceleration evokes right-backward heading selectivity in the 5th 1 s of movement in darkness. These responses are consistent with activation by acceleration, likely transduced by the vestibular otoliths. B: neuron showing responses with increasing left-backward heading selectivity throughout the 5 s of movement in darkness. These effects are consistent with a time-integrated response to movement. C: significant directional responses to gaze fixed movement in darkness were obtained from 68 of the 109 neurons tested. The largest number of neurons responded during, or immediately after, the 1 s acceleration period at the beginning of translational movement. The interval histogram (top) contains a horizontal line for each neuron that yielded at least 1 significant response. That line is marked for each of the ten 500 ms intervals (abscissa) that yielded a significant directional response in that neuron. Almost all of the neurons show transient responses, many related to the acceleration and deceleration periods of movement. The bar graph (bottom) shows the percentage of responsive neurons (ordinate) having their 1st significant response in the indicated 500 ms interval (abscissa). This shows the broad distribution of response onset during movement in darkness.

In contrast, Fig. 4B shows the directional responses of a neuron with sustained activation during gaze fixed movement in darkness. This neuron showed the gradual build-up of left-backward direction selectivity throughout the 5 s of movement, consistent with the temporal integration of the directional response to movement in darkness.

The latency and duration of gaze fixed movement in darkness responses varied across neurons. Most neurons (63%, 68/109) showed significant responses to movement in darkness in at least one of ten 500 ms stimulus intervals. These responses usually (59%, 40/68) included one of the first three response intervals corresponding to the 1,500 ms period during, and immediately after, translational acceleration (Fig. 4C).

The varying time-course of these responses and their reversal of preferred direction during deceleration support their vestibular origin. The alternative explanation of a relationship to vestibulo-ocular reflex responses (VOR) is unlikely because fixation control suppressed the VOR. This raises the possibility that a VOR cancellation signal created by fixation might affect these neurons. To test the effects of VOR cancellation, we repeated the gaze fixed movement in darkness experiment with the fixation point shut-off during movement. This placed the monkey in complete darkness with no available fixation target but the continued requirement that it maintain steady, centered gaze. The responses shown in Fig. 5, A and B, illustrates transient movement in darkness effects with and without a fixation point. Most neurons (57%, 12/21) responded with and without a fixation point and all with similar response directionality in the two conditions (<40°; Fig. 5C). Three neurons had weak responses in one interval of one condition and six neurons did not respond in either condition.

View larger version (24K): [in this window] [in a new window]   Fig. 5. Vestibular origin of MSTd neuronal responses to movement in darkness. A: directional responses of an MSTd neuron to movement in darkness with (left) and without (right) the centered fixation point. SDHs (format as in Fig. 3) from both conditions show right-backward heading preferences with similar responses in both conditions. B: polar plots of self-movement direction with 1 s acceleration period (left), 1 s middle of the steady-speed period (middle), and 1 s deceleration period (right) show transient responsiveness with left-forward heading preferences during acceleration and right-backward heading preferences during deceleration. The transient time-course of these responses, their reversing heading preference with the reversal of acceleration, and their presence with and without the fixation point suggest that they are of vestibular origin. C: bar graph showing the similarity of preferred headings during movement in darkness with and without the centered fixation point. Twenty-one neurons were tested with both stimuli; 12 showed significant directional responses to both stimulus sets and were the basis for this comparison.

We assessed the potential impact of pursuit errors on responses to gaze fixed movement in darkness. In each neuron, the gaze gain for the ten 500 ms intervals was compared with the strength of the directional response in that interval. Gaze gain is the ratio of the speeds of eye movement and target movement across the facing wall, as the target moved to remain directly in front of the moving monkey. The strength of directional responses was measured as the Z value of the polar distributions, the significance of which was shown in Fig. 4C. Linear regression showed no relationship between gaze gain and directional responses in these neurons (slope = 0.001, r² = 0.002, P = 0.4)

We conclude that MSTd neurons respond to gaze fixed movement in darkness from vestibular signals with a wide range of time-courses and direction preferences.

Gaze fixed movement in light

We compared MSTd neuronal responses to gaze fixed movement in darkness and light to assess the relative influence of vestibular and visual signals. Most MSTd neurons showed stronger responses to gaze fixed movement in light than in darkness. Figure 6 shows the movement in light responses of the neurons used to illustrate movement in darkness responses in Fig. 4. In light, the transient response neuron (Fig. 4A) maintained its brief activation and reversing heading preferences, with some increase in direction selectivity (Fig. 6A). The sustained response neuron (Fig. 4B) showed a greater increase in both response amplitude and duration during movement in light (Fig. 6B).

View larger version (30K): [in this window] [in a new window]   Fig. 6. The time-course of responses to gaze fixed movement in light (neurons and format as in Fig. 3). A: transient responses to acceleration and deceleration with left-forward heading selectivity during the 1st 1 s of movement in light and right-backward heading selectivity in the 5th 1 s. Such transient responses are less affected by light, but some directional enhancement is evident. B: sustained responses to movement in light with left-backward heading selectivity throughout the 5 s stimulus. This suggests interactions between vestibular and visual effects that alter the amplitude and time-course of the responses. C: significant responses to gaze fixed movement in light were obtained from 87% (95/109) of the neurons tested. The interval histogram (top) contains a horizontal line for each neuron that yielded at least 1 significant response. That line is marked for each of the ten 500 ms intervals (abscissa) that yielded a significant directional response in that neuron. A large number of these neurons (37%, 35/95) showed sustained responses (bottom traces). Other neurons showed transient responses like those seen with movement in darkness (top traces). The bar graph (bottom) shows that the majority of neurons (75%, 71/95) showed their 1st significant response interval in the 1st 1 s of the stimulus. Thus movement in light evoked still more acceleration responses than did movement in darkness.

The sample of neurons studied with movement in light reflected the effects shown by these examples. There was a substantial increase in the number of responsive neurons from 59% in darkness to 87% (95/109) in light. There were far more sustained responses in light and almost all of the responses began at stimulus onset (Fig. 6C). Thus MSTd neurons are generally more responsive during gaze fixed movement in light than during gaze fixed movement in darkness.

MSTd neuronal responses to gaze fixed movement in light were often attributable to the visual effects of optic flow. In many neurons, gaze fixed movement in darkness did not evoke significant responses, whereas optic flow without translational movement evoked robust responses. These directional responses to optic flow were commonly reflected in the responses to gaze fixed movement in light (Fig. 7, A-C).

View larger version (23K): [in this window] [in a new window]   Fig. 7. MSTd neuron with responses to gaze fixed movement in light that reflect its optic flow responses. A: interval-wise polar plots of self-movement direction (format as in Fig. 5) that shows little responsiveness to gaze fixed movement in darkness. B: SDHs and the polar plot of responses to the simulated self-movement direction. These figures show a strong left-forward heading preference in directional responses to optic flow presented without translational movement. C: interval-wise polar plots showing sustained left-forward heading preferences in responses to gaze fixed movement in light. The strength of the heading preference is proportionate to the speed of movement, potentially mediated by the related speed of visual motion in the resulting optic flow. D: bar graph showing larger response to optic flow than to movement in darkness. More than one-half of the neurons (62%, 29/47) showed substantially larger optic flow responses (>0.2), with few (17%, 8/47) showing substantially larger (< 0.2) movement in darkness responses. Thus visual responses were commonly stronger than vestibular responses.

We compared the strength of vestibular and visual responses in 66 neurons tested with gaze fixed translational movement and optic flow without translational movement. The movement stimuli were divided into five 1 s intervals to match the 1 s duration of the optic flow stimuli. This analysis compared gaze fixed movement in darkness and optic flow responses in intervals that yielded significant responses to gaze fixed movement in light, to focus the analysis on the relative contribution of vestibular and visual mechanisms to significant responses. Most neurons (71%, 47/66) showed significant directional responses (Z of net vector, P  0.05) to at least one of the five 1 s movement intervals or to optic flow alone. Most of those neurons (62%, 29/47) showed stronger responses to optic flow (vector length contrast > 0.2), but some (17%, 8/47) responded more strongly during at least one of the movement intervals (contrast < 0.2; Fig. 7D).

Many neurons showed different combinations of vestibular and visual effects across the five 1 s stimulus periods. In some neurons, responses to gaze fixed movement in light reflected the early predominance of vestibular effects, as seen during gaze fixed movement in darkness, and the later predominance of visual effects as seen during optic flow (Fig. 8A). In other neurons, responses to gaze fixed movement in light reflected the early predominance of visual effects, as seen during optic flow, with the later predominance of vestibular effects as seen during gaze fixed movement in darkness (Fig. 8B). Thus directional responses to gaze fixed movement in light reflected differences in the time-course and preferred headings of the vestibular and visual responses as quantitative response variation, not as winner-take-all effects.

View larger version (23K): [in this window] [in a new window]   Fig. 8. Comparison of heading preferences from net vectors of directional responses to gaze fixed movement in darkness (thin lines), in-light (thicker lines), and optic flow alone (bold line). Time-course dependent interactions between vestibular and visual effects are seen across real and simulated self-movement directions. Net vectors with significant directionality (Z value of circular distribution with P < 0.05) are indicated by filled circles. A: responses of a neuron that shows early responses to movement in darkness preferring a left-forward heading and a response to optic flow that prefers a left-backward heading. The response to movement in light is initially in the left-forward direction of the movement in darkness response and later it is in the left-backward direction of the optic flow response. B: responses of a neuron that shows late responses to movement in darkness preferring a left-forward heading and a response to optic flow that prefers a left-backward heading. The response to movement in light initially prefers the left-backward heading seen in the optic flow responses, and later prefers the left-forward heading seen in the movement in darkness responses. C: bar graph showing the wide range of differences between preferred headings in response to gaze fixed movement in darkness and optic flow without movement. Twenty-six intervals showed significant directional responses to both stimulus sets, with one-half (62%, 16/26) having similar heading preferences (differences <60°) but a substantial number (23%, 6/26) having very different heading preferences (differences > 120°). Thus vestibular and visual heading preferences tend to be either very similar or very different.

We compared heading preferences in response to gaze fixed movement in darkness and optic flow. Responses to the first 4 s of movement, excluding deceleration's directional reversals, and the 1 s of optic flow yielded 26 significant heading comparisons from 14 neurons (Z of resultant vectors with P  0.05 to both stimuli). There was a broad range of relative directions with most responses (65%, 17/26) showing similar heading preferences (<60^o difference) and the remainder showing larger differences, many (19%, 5/26) with nearly opposite preferred headings (>120° difference; Fig. 8C). One-dimension K-means cluster analysis confirmed the presence of two groups: All 26 data points were accommodated in the two-cluster model. Group one had 19 members with a mean directional difference of 33°; group 2 had 7 members with a mean directional difference of 127° (cluster analysis ANOVA, F(1,24) = 87.92, P < 0.001).

Differences in heading preferences with gaze fixed movement in darkness and optic flow resulted in time-dependent variation in heading preferences with gaze fixed movement in light. When gaze fixed movement in darkness and optic flow heading preferences were nearly opposite, these effects could cancel each other to yield no significant heading response to gaze fixed movement in light. We conclude that MSTd neurons combine visual and vestibular responses in a manner that reflects the time-course and direction selectivities of those responses.

Landmark pursuit movement in darkness

We recorded activity during landmark pursuit movement in darkness to determine whether vestibular and pursuit signals co-exist in single MSTd neurons. Some directional responses to landmark pursuit movement in darkness mainly reflected responses to gaze fixed movement in darkness (Fig. 9A). Other landmark pursuit responses mainly reflected responses to pursuit in darkness (Fig. 9B), shown as self-movement vectors to illustrate their potential contribution to landmark pursuit responses. In both cases, landmark pursuit responses were also affected by the nondominant influence of either gaze fixed or pursuit responses. This was seen in the similarity of landmark pursuit heading vectors and the vector sum of gaze fixed and pursuit headings.

View larger version (21K): [in this window] [in a new window]   Fig. 9. Directional responses to landmark pursuit movement in darkness (dotted lines) combine the effects of gaze fixed movement in darkness (solid lines) and pursuit in darkness (dashed lines) for the corresponding self-movement directions. A: directional responses of a neuron that shows strong gaze fixed and landmark pursuit effects with similar directionality but only small responses to pursuit in darkness. B: directional responses of a neuron that shows strong landmark pursuit and pursuit in darkness effects with similar directionality but only small responses with gaze fixed. C: bar graph showing the wide range of amplitude differences in response to gaze fixed movement in darkness and pursuit in darkness. Many neurons (48%, 52/109) showed substantially larger gaze fixed responses (< 0.2), with few (13%, 14/109) showing substantially larger (>0.2) pursuit in darkness responses. Thus vestibular responses were more commonly stronger than pursuit in darkness responses.

We compared the strength of gaze fixed movement in darkness and pursuit in darkness responses in 109 neurons tested with both stimulus sets (Fig. 9C). One-half of the neurons (48% 52/109) showed substantially (< 0.2) larger responses to the gaze fixed stimuli, whereas only 13% (14/109) showed substantially (>0.2) larger responses to the pursuit stimuli with the remainder showing similar responses to both. Overall, vestibular responses to gaze fixed movement in darkness had larger responses, but pursuit in darkness clearly influenced many neurons. Thus we conclude that single MSTd neurons combine vestibular and pursuit responses, mostly with stronger vestibular effects but in some cases with stronger pursuit effects.

Landmark pursuit movement in light

We recorded activity during landmark pursuit movement in light to test whether visual, vestibular, and pursuit signals are combined in single MSTd neurons. Neurons with large responses to landmark pursuit movement in light, but not to gaze fixed movement in light, had pursuit in light responses that accounted for the landmark pursuit effects (Fig. 10A). Conversely, neurons with large responses to gaze fixed movement in light, but not to landmark pursuit movement in light, had pursuit in light responses with the opposite directionality of their gaze fixed responses (Fig. 10B). These pursuit effects canceled the directional effects of vestibular stimulation during landmark pursuit.

View larger version (25K): [in this window] [in a new window]   Fig. 10. Responses to landmark pursuit movement in light (dotted lines) combine the effects of gaze fixed movement in light (solid lines) and pursuit in light (dashed lines) for the corresponding self-movement directions. A: directional responses of a neuron that shows strong landmark pursuit movement in light responses and strong pursuit in light responses with similar directionality but no significant gaze fixed movement in light responses. B: directional responses of a neuron that shows strong gaze fixed movement in light effects with opposite directionality in strong pursuit in light responses that seemingly cancel to block any significant landmark pursuit movement in light responses. C: comparison of the directional differences between each neuron's pursuit in light responses and its responses to gaze fixed or landmark pursuit movement in light. This graph shows the percentage of intervals with significant directionality (ordinate) and directional differences (abscissa). Neurons with larger landmark pursuit responses (open bars) have the similar directions of landmark pursuit and pursuit in light responses (as in A). Neurons with larger gaze fixed responses (filled bars) have opposite directions of gaze fixed and pursuit in light responses (as in B).

Such effects were evident in all neurons that showed differences between gaze fixed and landmark pursuit responses: Gaze fixed preferring neurons usually had >90° differences between the directions of the responses to gaze fixed movement in light and pursuit in light (Fig. 10C, filled bars). Landmark pursuit preferring neurons usually had <90^o differences between the directions of the responses to landmark pursuit movement in light and pursuit in light (Fig. 10C, open bars). This suggests that movement and pursuit responses with similar heading preferences combine to create larger landmark pursuit responses. When movement and pursuit heading preferences are dissimilar, they cancel to prevent significant landmark pursuit directionality.

Responses to gaze fixed and landmark pursuit movement in light were not substantially affected by the three-dimensional structure of the stationary light array. Similar heading preferences were obtained with one and two depth-planes of stationary lights during gaze fixed (Fig. 11A) and landmark pursuit (Fig. 11B) movement in light. The strength of heading selectivity during gaze fixed and landmark pursuit conditions was similar with one and two depth-planes. Intervals with significant responses to both gaze fixed and landmark pursuit stimuli showed similar heading preferences with two depth-planes (81%, 63/78 with differences <40°; Fig. 11C) as they did with one depth-plane.

View larger version (26K): [in this window] [in a new window]   Fig. 11. Responses to gaze fixed and landmark pursuit movement in light showing similar responses with one (A) or two (B) depth-planes of stationary lights across self-movement directions. A: directional responses of a neuron that shows similar preferred directions during gaze fixed and landmark pursuit movement in light with one depth-plane of stationary lights. B: directional responses of the same neuron showing similar gaze fixed and landmark pursuit headings with 2 depth-planes of stationary lights. C: bar graph showing differences between heading preferences evoked with one vs. two depth-planes in the visual array. Comparisons for gaze fixed movement in light and for landmark pursuit movement in light both yielded similar results and are combined in this figure. The percentage of response intervals (ordinate) is shown for the range of one-plane vs. two-planes directional differences (abscissa) with few responses yielding directional differences >40°.

Thus MSTd neurons combine visual, vestibular, and pursuit responses with effects that depend partly on each input signal and partly on idiosyncratic signal interactions.

Response interactions

We characterized single neuron response interactions for vestibular, visual, and pursuit effects in both the directional domain and the neuronal firing rate domain. Interactions in the directional domain were compared across the five 1 s movement intervals (Fig. 12). Directional responses to gaze fixed movement in darkness and optic flow were arranged to reflect their vector sum. This vector sum was compared with the directional responses to gaze fixed movement in light that combined movement and optic flow stimulation. The example neuron's vector summed responses are similar to those evoked by combined stimuli (Fig. 12A).

View larger version (25K): [in this window] [in a new window]   Fig. 12. The vector sum of directional responses to elementary stimuli predicted responses to combined stimuli. A: vector sum of directional responses to optic flow (thick line) and gaze fixed movement in darkness (dashed line) closely approximated the directional responses to gaze fixed movement in light (solid line). B: vector sum of gaze fixed movement in light (dashed line) and pursuit in light (thick line) closely approximated the directional responses to landmark pursuit movement in light (solid line). C: differences between the direction of the vector sum of gaze fixed movement in darkness and optic flow compared with the direction of gaze fixed movement in light. In most cases the vector sum and the combined stimuli yielded similar directions (55%, 52/95, <40°), but many responses yielded substantially different directions. D: differences between the directions of the vector sum of gaze fixed movement in light and pursuit in light compared with the direction of landmark pursuit movement in light. In almost all cases, the vector sum and the combined stimuli yielded similar directions (88%, 112/127, <40°). Responses from all five 1 s movement intervals were included when at least one of the two elementary response vectors showed significant directionality (Z resultant vector, P  0.05). These findings suggest a vector summation model of response interactions with greater nonadditivity in visual-vestibular interactions than with pursuit interactions.

Similarly, directional responses to gaze fixed movement in light and pursuit in light without movement were vector summed. This vector sum was compared with the directional responses to landmark pursuit movement in light that combined movement, optic flow, and pursuit stimulation. This neuron's vector summed responses are again similar to those evoked by combined stimuli (Fig. 12B). These findings suggest a vector summation model of response interactions in MST neurons.

We tested the vector summation model across the sample of neurons and found a wide range of response interactions. Responses to gaze fixed movement in darkness were vector summed with responses to optic flow alone. This summed vector was compared with the directional responses evoked by the corresponding combined stimulus, gaze fixed movement in light. The summed and combined response vectors had directions that were either very similar (55%, 52/95, <40°) or very different (21%, 20/95, >140°; Fig. 12C).

Similarly, responses to gaze fixed movement in light were vector summed with responses to pursuit in light. This summed vector was compared with the directional responses evoked by the corresponding combined stimulus, landmark pursuit movement in light. The directions of the summed and combined response vectors were consistently similar, despite the fact that movement in light and pursuit in light responses commonly showed large directional differences (Fig. 12D).

Such response interactions were also seen in the firing rate domain. Single neuron response amplitudes were summed for each direction and each 1 s stimulus interval. Summed responses to gaze fixed movement in darkness and optic flow (abscissa) were compared with gaze fixed movement in light responses (ordinate) and fit with a regression line. The three neurons illustrated show additive effects (positive slope), nonadditive effects (flat slope), and subtractive effects (negative slope) (Fig. 13A). Similarly, summed responses to gaze fixed movement in light and pursuit in light (abscissa) were compared with landmark pursuit movement in light responses (ordinate). The same three neurons all show clearly additive effects (positive slopes; Fig. 13B).

View larger version (44K): [in this window] [in a new window]   Fig. 13. Single neuron firing rates show greater nonadditivity for visual-vestibular interactions than for pursuit interactions. A: responses of three neurons shown as the sum of response amplitudes to gaze fixed movement in darkness plus optic flow (abscissa) compared with response amplitudes to gaze fixed movement in light (ordinate). Each point represents responses to one of five 1 s intervals and 1 of 8 heading directions. Neuron 12LP70 showed additive effects, 12LP116 showed nonadditive effects, and 12LP64 showed subtractive effects. B: responses of three neurons shown as the sum of response amplitudes to gaze fixed movement in light plus pursuit in light (abscissa) compared with response amplitudes to landmark pursuit movement in light (ordinate) (neurons and format as in A). All three neurons showed additive effects. C and D: slope of regression lines (abscissa) fit to firing rate response relations as shown in A and B for the percentage of responses (ordinate) across intervals and neurons. C: sum of gaze fixed movement in darkness and optic flow responses compared with gaze fixed movement in light responses yielded mostly sub-additive interactions (68%, 65/95, <0.6). D: sum of gaze fixed movement in light and pursuit in light responses compared with landmark pursuit movement in light responses yielded mostly additive interactions (53%, 67/127, 0.8-1.2). This supports the notion of response additivity, more generally for visual-vestibular interactions than for pursuit interactions.

This pattern of firing rate response interactions was seen throughout the sample of MSTd neurons. Regression slopes for summed gaze fixed movement in darkness plus optic flow responses versus the combined gaze fixed movement in light responses (abscissa) were derived for all neurons with significant responses (ordinate). These responses mainly yielded flat or negative slopes (68%, 65/95, <0.6), reflecting sub-additive or subtractive interactions (Fig. 13C). Regression slopes for summed gaze fixed movement in light and pursuit in light responses versus combined landmark pursuit movement in light responses (abscissa) were also derived. These slopes tended to be positive (53%, 67/127, 0.8-1.2), reflecting additive interactions (Fig. 13D).

Response interactions in the directional and firing rate domains were linked. Gaze fixed responses that showed large differences between the summed and combined vectors in the directional domain also showed negative slopes, suggesting inhibitory interactions, in the firing rate domain (Fig. 14A). Landmark pursuit responses did not show large differences between the summed and combined vectors, and they did not show subtractive interactions in the firing rate domain (Fig. 14B). These findings suggest that directional and firing rate interactions share common mechanisms, such as directionally selective excitatory and inhibitory synaptic drive.

View larger version (21K): [in this window] [in a new window]   Fig. 14. Response interactions are linked in the directional and firing rate domains. A: gaze fixed responses shown as the slope of firing rate relations between summed and combined responses (abscissa, from Fig. 13C) vs. the difference in summed and combined net directions (ordinate, from Fig. 12C). Responses with subtractive interactions in the firing rate domain show large differences in the directional domain (r2 = 0.45). B: landmark pursuit responses shown as the slope of firing rate relations between summed and combined responses (abscissa, from Fig. 13D) vs. the difference in summed and combined net directions (ordinate, from Fig. 12D). There are few sub-additive effects or large directional differences (r2 = 0.18).

Vestibular, visual, and pursuit interactions create heading selective responses during gaze fixed and landmark pursuit movement in light; some neurons are active under both conditions and others in only one. We measured heading selectivity as the Z value of the circular distributions in the five response intervals of all 109 neurons. Gaze fixed or landmark pursuit movement in light evoked significant directionality in 301 responses intervals (Fig. 15A). One-half of the responses (48%, 143/301) were significant with both stimulus sets (filled circles) and usually (66%, 95/143) shared similar heading preferences (<40° differences; Fig. 15B). The remaining responses were significant in only one condition: 31% in gaze fixed (squares) and 21% in landmark pursuit (triangles).

View larger version (29K): [in this window] [in a new window]   Fig. 15. Comparison of heading selectivity during gaze fixed and landmark pursuit movement in light across self-movement directions. A: strength of heading selectivity is measured as the Z value of the circular distribution of heading responses for gaze fixed (abscissa) and landmark pursuit (ordinate) responses during movement in light. The greatest number of responses showed significant directionality with both gaze fixed and landmark pursuit (48%, 143/301, filled circles). An additional 31% (95/301) showed significant directionality only with gaze fixed (squares) and another 21% (63/301) showed significant directionality only with landmark pursuit (triangles). Intervals without significant responses in either condition are not shown (open area near origin). B: bar graph of directional differences between preferred headings in response to gaze fixed and landmark pursuit movement in light when both yielded significant responses. Two-thirds of the responses (66%, 95/143) had preferred directions within 40° of each other.

We conclude that single neurons show idiosyncratic interactions between visual, vestibular, and pursuit responses. These effects are generally additive for vestibular-visual and pursuit responses and nonadditive for vestibular and visual responses.

Population responses

We examined whether MSTd neuronal responses might combine to support the population encoding of heading under the naturalistic self-movement conditions tested. We used population vector analysis to assess heading estimation using each neuron's preferred direction in a given stimulus condition. Population vectors were derived by weighting each neuron's vector by the amplitude of its response to a given stimulus direction and then vector summing across all neurons (see METHODS, Eq. 1).

We first compared population responses to optic flow and gaze fixed movement in light using the 66 neurons tested in both conditions. This analysis was restricted to the middle 1 s of the 5 s movement period in which visual stimulation was most comparable to optic flow stimuli. These conditions yielded comparable population net vectors with no significant differences (Mann-Whitney U, P > 0.05) between population net vector accuracy with optic flow (mean = 22°) and movement (mean = 19°), or population net vector amplitude with optic flow (mean = 255 spikes/s) and movement (mean = 182 spikes/s; Fig. 16).

View larger version (19K): [in this window] [in a new window]   Fig. 16. Optic flow (A) to the middle 1 s of gaze fixed movement in light (B) yielded similar population responses in the sub-sample of 66 neurons tested with both stimuli. These plots provide a vector representation of each neuron's responses to the 8 stimulus directions for the given study. Each neuron is represented by a vector (thin lines) having that neuron's preferred direction derived from the vector sum of all responses during that study. The length of each neuron's vector for each of the 8 directions is proportionate to the amplitude of its response to that direction. The population vector for each direction (bold lines) is the sum of the vectors for individual neurons.

We compared the population vector's representation of the eight stimulus headings during optic flow and gaze fixed movement in light. Heading accuracy was similar under these two conditions, with the population net vector's heading and the stimulus heading yielding good linear regression fits for both optic flow (r² = 0.97, slope = 1.12) and for gaze fixed movement in light (r² = 0.97, slope = 1.11). The magnitude of the population net vectors was also similar with optic flow (51 ± 24 spikes/s) and gaze fixed movement in light (36 ± 8 spikes/s; Fig. 17). Thus adding translational movement to visual stimulation did not create greatly enhance population responses.

View larger version (43K): [in this window] [in a new window]   Fig. 17. Population vectors for optic flow (A and C) and gaze fixed movement in light (B and D). A and B: scatter plots show the direction of the population vector (ordinate) for the 8 movement directions (abscissa). A least squares linear regression fit these points (solid line), all of which represent significant net vectors (Z value of the circular distribution with P < 0.05). Optic flow (A, r2 = 0.97, slope = 1.12) and gaze fixed movement in light (C, r2 = 0.97, slope = 1.11) were similarly accurate. C and D: bar graphs show the average amplitude of the population net vectors (ordinate) for each of the 8 movement directions (abscissa). Population net vectors for optic flow were larger and had greater variability (51 ± 24 spikes/s) than those for gaze fixed movement in light (36 ± 8 spikes/s).

Population vector heading representation assumes that a neuron's firing rate signals the similarity of a stimulus to its preferred heading. If the population-encoding scheme changed under different self-movement conditions then decoding it would require additional information specifying current conditions. We hypothesize that population vector heading representation is robust to changes in self-movement conditions and does not require such additional information. We tested this hypothesis by deriving each neuron's preferred heading from its responses to all four naturalistic self-movement conditions and then comparing population responses across conditions.

In both the gaze fixed and landmark pursuit conditions, population vector responses to movement in darkness were smaller (Fig. 18, A and B) than those obtained during movement in light (Fig. 18, C and D). Population responses were stronger with forward headings during movement in darkness and relatively symmetrical during movement in light. These differences may reflect inadvertent neuronal sampling bias favoring forward vestibular responses.

View larger version (38K): [in this window] [in a new window]   Fig. 18. Population responses to the four naturalistic self-movement conditions: gaze fixed movement in darkness (A), landmark pursuit movement in darkness (B), gaze fixed movement in light (C), and landmark pursuit movement in light (D) (format as in Fig. 15). The preferred direction for each neuron was derived from the vector summation of responses to all directions across all 4 conditions. The length of each neuron's vector for each of the 8 directions is proportionate to the amplitude of its response to that direction in that condition. Population vectors more closely approximate the stimulus direction during movement in light regardless of pursuit effects.

We compared the population vector's representation of heading for the eight stimulus directions, in the five 1 s stimulus intervals, for these four self-movement conditions. During movement in darkness, the smaller population response amplitudes yielded several nonsignificant responses with gaze fixed (Fig. 19A) and with landmark pursuit (Fig. 19B). Regression lines relating stimulus heading to population vector heading showed strong associations (slope approximately 1, r² ~ 0.80). During movement in light, the population vector amplitudes were about four times larger than those obtained in darkness; all intervals yielded significant responses with gaze fixed (Fig. 19C) and with landmark pursuit (Fig. 19D). Regression lines relating these stimulus directions to the population vector directions showed stronger associations (slopes approximately 1.15, r² ~ 0.97).

View larger version (39K): [in this window] [in a new window]   Fig. 19. Population vector accuracy and amplitude across five 1 s stimulus periods based on the combined responses to all 4 basic types of self-movement. Each scatter plot shows the population vectors for 1 of the 4 basic self-movement conditions. Each point shows the direction of the population vector for one of the five 1 s movement intervals (ordinate) in relation to the direction in that movement stimulus (abscissa), and a least squares linear regression was used to fit these points (solid line). Filled points represent significant net vectors (Z value of the circular distribution with P < 0.05). The accompanying bar graphs show the average amplitude of those population net vectors (ordinate) for each of the five 1 s stimulus intervals (abscissa). A: population vectors for gaze fixed movement in darkness show a clear relationship between the population vector direction and the movement direction (r2 = 0.79, slope = 1.03) but with relatively small population vector amplitudes in all intervals. B: landmark pursuit movement in darkness shows similar effects (r2 = 0.84, slope = 1.07). C: population vectors for gaze fixed movement in light show a more consistent relationship between population vector direction and movement directions (r2 = 0.96, slope = 1.14) with much larger population vector amplitudes. D: landmark pursuit movement in light shows similar effects (r2 = 0.97, slope = 1.15).

We conclude that MSTd integrates visual, vestibular, and pursuit signals for the population representation of self-movement heading. This representation shows little benefit of vestibular input during movement in light, but vestibular input seemingly preserves heading selectivity during movement in darkness.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Vestibular responses in MSTd

We used four self-movement conditions to assess MSTd's contributions to heading estimation: gaze fixed movement in darkness, gaze fixed movement in light, landmark pursuit movement in darkness, and landmark pursuit movement in light (Figs. 1-3). Responses to gaze fixed movement in darkness greatly reflect vestibular effects that reverse directionality from acceleration to deceleration phases of the same movement direction (Fig. 4A). The time-course of these responses varies across neurons, with some showing brief duration acceleration effects and others showing a gradual build-up of activity through the steady-speed phase of the stimulus, apparently integrating activation over time (Fig. 4, B and C).

MSTd neurons respond to gaze fixed movement in total darkness (Fig. 5). This suggests the presence of signals from the vestibular otoliths and speaks against a relationship to the translational VOR (Paige and Tomko 1991; Schwarz and Miles 1991): During gaze fixed movement in darkness with a fixation point there are no VOR movements, and in total darkness there is no VOR cancellation stimulus. The presence of translational movement vestibular otolith signals in MSTd is consistent with rotational movement vestibular canal signals in posterior parietal (Kawano et al. 1984) and MST neurons (Erickson and Thier 1992). These findings can be combined with our previous studies of translational vestibular responses in MSTd (Duffy 1998) to support the conclusion that this area receives input from both the vestibular canals and otoliths. This is consistent with vestibular projections to these areas (Akbarian et al. 1994; Faugier-Grimaud and Ventre 1989). The varying time-course of these responses might reflect the neural integration of vestibular signals in MSTd or an intermediate site (McNaughton et al. 1996) to support path-integration (Froehler and Duffy 2002) during self-movement in darkness. Thus we conclude that MSTd accesses vestibular signals about self-movement.

Sensory integration in MSTd

Gaze fixed movement in light responses were more sustained (Fig. 6) and larger (Fig. 7) than responses evoked by movement in darkness. Heading preferences during movement in light often combined movement in darkness and optic flow responses (Figs. 7 and 8). This revealed time-dependent response interactions reflecting the varying time-course and the broad range of relative heading preferences of the vestibular effects of movement in darkness and the visual effects of optic flow.

Many MSTd neurons responded to both gaze fixed and landmark pursuit movement in darkness (Fig. 9). Differences between the preferred headings of those responses were attributable to the relative directionality of vestibular and pursuit effects. Vestibular and pursuit effects combined so that the preferred heading in landmark pursuit was the vector sum of the preferred headings in gaze fixed movement in darkness and pursuit in darkness. Thus these pursuit vectors contribute to MSTd's heading estimation during landmark pursuit movement in darkness and may reflect the potential use pursuit signals as a heading cue.

Landmark pursuit movement in light combined vestibular, visual, and pursuit effects (Fig. 10). The presence of more than one depth-plane in the light array did not greatly alter MSTd neuronal heading selectivity (Fig. 11). This is consistent with our earlier findings in which simulated depth-planes strengthened the heading selectivity of some neurons but did not alter their preferred headings (Upadhyay et al. 2000). The absence of significant changes with the addition of the second depth-plane in this study may reflect the fact that the single depth-plane light array presented points at widely differing distances from the monkey along with other depth cues in the room. We conclude that individual MSTd neurons access multi-sensory input during self-movement.

Multi-sensory interactions

Interactions between visual, vestibular, and pursuit responses were consistent with the vector summation of heading responses (Fig. 12). The vector sum of responses to gaze fixed movement in darkness and optic flow often predicted responses to gaze fixed movement in light (Fig. 12C) and the vector sum of responses to gaze fixed movement in light and pursuit in light predicted responses to landmark pursuit movement in light (Fig. 12D). However, in the former case, a substantial number of responses yielded directional preferences that were nearly the opposite of that predicted by vector summation. This suggests a number of nonadditive visual-vestibular interactions.

Single neuron firing rates showed an even greater tendency toward nonadditive interactions between responses to gaze fixed movement in darkness and optic flow in predicting responses to gaze fixed movement in light (Fig. 13, A and C). In contrast, firing rates showed consistently additive interactions between responses to gaze fixed movement in light and pursuit in light summing to predict responses to landmark pursuit movement in light (Fig. 13, B and D). Interactions in the directional and firing rate domains are linked (Fig. 14), suggesting that they share the same underlying mechanisms. One such mechanism is the combination of directionally selective excitatory and inhibitory synaptic drive. These effects can result in additive, divisive (Koch et al. 1983), and multiplicative (Pena and Konishi 2001) response interactions. The diverse response interactions in our data are consistent with these mechanisms creating a sigmoidal relationship between synaptic activation and neuronal firing rate (Chapeau-Blondeau 2001). This sigmoidal relationship might vary greatly between neurons and might vary within individual neurons under the influence of stimulus parameters such as movement acceleration and visual contrast.

Interactions between movement in light and pursuit in light responses having opposite heading preferences often canceled each other when they were combined in landmark pursuit. This effect created subsets of responses that were active during gaze fixed conditions, landmark pursuit conditions, or both (Fig. 15A). Responses activated by both gaze fixed and landmark pursuit stimuli typically showed similar preferred headings under both conditions (Fig. 15B). Such responses might be well suited to serving heading estimation with fixed gaze or pursuit (Royden et al. 1992; Stone and Perrone 1997; Warren and Hannon 1990).

Responses to pursuit in light and pursuit in darkness cannot be equated with retinal and extra-retinal inputs; pursuit in light can evoke nonvisual signals and pursuit in darkness can evoke visual effects of the pursuit target. Nevertheless, the pursuit responses recorded in these studies are consistent with both retinal and extra-retinal mechanisms (Newsome et al. 1988). Pursuit interactions with visual stimuli in MSTd can show either summation or winner-take-all effects, depending on stimulus conditions (Recanzone and Wurtz 2000). This emphasizes that such summation and winner-take-all effects are not mutually exclusive. Thus we might expect that both types of effects could influence visual, vestibular, and pursuit responses under different stimulus conditions.

Population encoding of heading

We used population vector analysis (Georgopoulos et al. 1986) to see if single neuron responses combine to create a coherent heading signal. Optic flow yielded population responses that were comparable to those obtained with gaze fixed movement in light (Figs. 16 and 17). The accuracy and amplitude of this optic flow population response was less than that seen with a larger sample (Page and Duffy 1999). Nevertheless, these findings show that simulated optic flow evokes a population response comparable to that evoked by steady-speed self-movement in light.

We tested whether MSTd might signal heading without the exogenous specification of stimulus conditions. Responses to all four naturalistic self-movement conditions were combined to derive single neuron heading preferences and population responses to gaze fixed and landmark pursuit and movement in darkness and light (Figs. 18 and 19). The largest difference between conditions was from movement in light, yielding larger population responses than were obtained with movement in darkness.

Single neuron responses combine vestibular, visual, and pursuit signals by nonadditive weighted summation to create diverse responses reflecting the relative strength, time-course, and heading selectivity of those signals. These effects alter single neuron responses across conditions but allow the population encoding of heading (Fig. 20). This is consistent with population models of heading estimation in MST (Lappe and Rauschecker 1993; Perrone and Stone 1998) and with pursuit altering single neuron optic flow responses while the population heading vector remained veridical (Page and Duffy 1999).

View larger version (29K): [in this window] [in a new window]   Fig. 20. Summary of findings and a model of heading estimation in MSTd. A: single neuron responses to observer movement in light can be classified into 3 groups: 1) continuous responses that are dominated by visual effects from the optic flow maintaining the same pattern throughout the movement; 2) variable responses combining visual and vestibular effects depending on the alignment of their directions and the time-course of the vestibular effects; and 3) reversing responses that are dominated by vestibular effects reflecting the acceleration or speed profiles of the movement. B: visual, vestibular, and eye movement signals combine to create single neuron responses. Pursuit exerts retinal and extra-retinal influences on these responses. The weighting of the optic flow (wo), vestibular (wv), and pursuit (wp) signals, along with the time-course and directional alignment of optic flow and vestibular effects, shape the responses of individual MST neurons. These response interactions reflect nonadditive response dynamics, here characterized as a sigmoidal activation-response curve that yields sub-additive effects (A and E), additive effects (B and D), and super-additive effects (C). All of these signals combine by vector summation to provide a population representation of heading across a continuum of self-movement conditions dichotomized as the movement in darkness and movement in light conditions used in these studies.

In summary, vestibular input maintains a coarse population heading representation during movement in darkness. During movement in light, visual input provides a more robust heading signal that is not greatly enhanced by vestibular signals, although parametric changes in the stimuli might alter this relationship. Under the conditions of these experiments, single neuron response interactions by weighted summation support winner-take-all competition in the population. Thus MST's function depends on stimulus conditions; in darkness, it is dominated by vestibular input, and in light, it is dominated by visual input. Intermediate circumstances might emphasize response interactions such that the population response is enhanced by the combination of inputs (Duffy and Wurtz 1994).

We cannot address how population responses are created or used in cortex (Sanger 1996; Zemel et al. 1998). Trained networks yield much more accurate heading estimates with such data, but no training algorithm or conditional weightings of single neuron responses are provided by our experiments. Our summing responses without trained weightings may set lower bounds on MSTd's contributions to heading estimation. This does not deny the potential benefits of training, the contributions of other vestibular cortical areas (Guldin et al. 1992; Hietanen and Perrett 1996), differences between heading estimates needed by perceptual (Harris et al. 2000) and motor (Merchant et al. 2001) systems, or the possible influence of heading discrimination tasks on neuronal processing. Our population vector analysis simply supports the conclusion that MSTd has the information needed for heading estimation under a variety of naturalistic self-movement conditions.