The Journal of Neurophysiology Vol. 80 No. 4 October 1998, pp. 1816-1827
Copyright ©1998 by the American Physiological Society

MST Neurons Respond to Optic Flow and Translational Movement

Charles J. Duffy

Departments of Neurology, Neurobiology and Anatomy, Ophthalmology, Brain and Cognitive Sciences, and the Center for Visual Science, University of Rochester Medical Center, Rochester, New York 14642

	ABSTRACT

Abstract Introduction Methods Results Discussion References

Duffy, Charles J. MST neurons respond to optic flow and translational movement. J. Neurophysiol. 80: 1816-1827, 1998. We recorded the responses of 189 medial superior temporal area (MST) neurons by using optic flow, real translational movement, and combined stimuli in which matching directions of optic flow and real translational movement were presented together. One-half of the neurons (48%) showed strong responses to optic flow simulating self-movement in the horizontal plane, and 24% showed strong responses to translational movement. Combining optic flow stimuli with matching directions of translational movement caused substantial changes in both the amplitude of the best responses (44% of neurons) and the strength of direction selectivity (71% of neurons), with little effect on which stimulus direction was preferred. However, combining optic flow and translational movement such that opposite directions were presented together changed the preferred direction in 45% of the neurons with substantial changes in the strength of direction selectivity. These studies suggest that MST neurons combine visual and vestibular signals to enhance self-movement detection and disambiguate optic flow that results from either self-movement or the movement of large objects near the observer.

	INTRODUCTION

Abstract Introduction Methods Results Discussion References

The medial superior temporal area (MST) of monkey extrastriate cortex contains neurons with large receptive fields, direction selectivity for visual motion, and response selectivity for particular optic flow stimuli (Duffy and Wurtz 1991a,b, 1995; Graziano et al. 1994; Orban et al. 1992; Saito et al. 1986; Tanaka and Saito 1989), making MST neurons likely candidates for involvement in visual motion processing for self-movement perception.

However, optic flow from self-movement can be indistinguishable from the visual motion created by the movement of a large object near the observer (Cornilleau-Peres and Droulez 1994; Gibson 1954; Probst et al. 1986). To disambiguate optic flow from object movement we might use vestibular (Israel et al. 1993; Young and Meiry 1968) signals about translational acceleration in the horizontal plane. Vestibular input could be combined with optic flow signals to support a veridical sense of body translation through three dimensional space (Lishman and Lee 1973; Telford et al. 1995). Thus optic flow neurons in area MST might be considered more well suited to self-movement perception if they are also influenced by vestibular signals.

To test whether MST neurons are affected by vestibular signals we recorded their responses to translational self- movement on a motorized sled. A substantial number of MST neurons showed clear responses to translation movement in darkness with most showing differences between responses to optic flow alone versus optic flow with translational movement. These findings suggest that MST neurons might contribute to the integration of visual and vestibular sensory information about self-movement.

A brief report of this work appeared previously (Duffy 1996).

	METHODS

Abstract Introduction Methods Results Discussion References

Two adult Rhesus monkeys were used in these experiments and remain the subjects of continuing studies. Behavioral parameters, stimulus presentation, and neuronal recordings were controlled by the real-time experimental system on a personal computer (Hays et al. 1982). All protocols were approved by the University of Rochester Committee on Animal Research and complied with Public Health Service and Society for Neuroscience policy on the humane care and use of laboratory animals.

Optic flow stimuli

Optic flow stimuli were identical under stationary and moving observer conditions with the projector and screen mounted on the sled so that they moved with the animal. Optic flow stimuli were created with a liquid crystal projector (Sony 8500U) back-projecting onto a (90°)² tangent screen. Stimuli were created off-line and shown as animated sequences in pseudorandom order. The stimuli consisted of 500 white dots (0.2°, 2.61 cd/m²) on a dark background (0.16 cd/m²) with dot locations in the first frame assigned at random and then adjusted to assure uniform dot density. Subsequent frames were projected at a 60-Hz frame rate with the location of each dot moving according to the algorithm for that stimulus. All dots had a limited screen lifetime randomly assigned from 16 to 2,000 ms with a constant, uniform dot density maintained across the stimulus.

Twelve basic optic flow stimuli were used in the initial testing of each neuron; eight planar motion stimuli (45° intervals in 360°) simulated translation in the frontoparallel plane. Two radial motion stimuli (inward and outward) simulated translation along the anteroposterior axis. Two circular stimuli (counterclockwise and clockwise) simulated rotation around the anteroposterior axis.

Eight other optic flow stimuli simulated the visual motion seen by an observer during linear translation in the horizontal plane in one of eight directions distributed at 45° intervals around 360° (Fig. 1A). Optic flow for rightward (0°) and leftward (180°) self-movement was simulated as planar dot movement to the left and right, respectively. Optic flow for forward (90°) and backward (270°) self-movement was simulated with the center of expansion or contraction at the center of the screen. The other four stimuli of this set contained radial centers of motion along the horizontal meridian displaced by 45° from the center of the stimulus.

View larger version (32K): [in this window] [in a new window]   FIG. 1. Schematic representations of optic flow and translational movement stimuli presented during centered fixation. A: optic flow stimuli were projected on to a (90°)2 screen to simulate observer self-movement in eight directions in the horizontal plane. To simulate forward self-movement (90°) we presented symmetric, radially expanding optic flow with the center of motion in the center of the screen. B: 2-dimensional sled system was used to present translational movement stimuli, moving the chaired animal and visual stimulation system in the horizontal plane. Sled stimuli consisted of 8 directions of translational movement (0-315°) matching to the 8 directions simulated with the optic flow stimuli.

In all of the planar motion stimuli, dot speed was uniformly maintained at 40°/s with dots disappearing at the terminal edge and reappearing at a location chosen to maintain uniform dot density. In all of the radial optic flow stimuli, dot speed increased as a sin(t) * cos(t) function where t is the angle from the center of motion in the stimulus to a given dot from the perspective of the observer. Dot speed was set to 40°/s at points that were 22.5° from the center of motion such that they simulated observer translational toward or away from the stimulus plane at 0.5 m/s. Because dot speed at a given location did not increase across successive frames, the outward radial stimuli simulate approach toward a surface that does not get perceptibly closer during the stimulus. As in previous optic flow studies, we accepted this nonnaturalistic property to maintain a constant speed range comparable with speeds in other stimuli. Outward radial dots disappeared by going off the outer edge or by lifetime expiration; inward radial dots disappeared when they increased local dot density near the center of motion or by lifetime expiration. Dots in radial patterns were replaced at locations chosen to optimize stimulus density homogeneity.

Translational movement stimuli

Translational movements were made with the monkey chair mounted on two double-rail drive systems (Acutronic, Fig. 1B). The 1 × 2 m movement platform could be moved 125 cm from center in any direction at speeds 100 cm/s and accelerations 100 cm/s². Table movements were controlled with position feedback from the motors providing good position and speed stability but not specifically controlling performance in the acceleration domain. The table carries the neurophysiological and eye coil recording systems with the monkey. The visual stimulation system is mounted to rear-project onto the tangent screen 48 cm from the animal. The frame of the eye coil system is covered by black, opaque plastic that encloses the animal and limits its field of view to the projection screen.

The onset time of motion is commanded by the experimental control computer with direct feedback from the controller so that the time of motion onset is precisely matched to other conditions in each experimental trial. In addition, feedback from the translator's position sensing system is passed to the experimental control computer, sampled at 125 Hz, and stored in the analog data record of each trial. In the current experiments we operated the system in the range of 10-60 cm/s maximum speed and 10-60 cm/s² peak acceleration to be above the human vestibular threshold (Benson et al. 1986) and below the level at which mechanical effects prevent stable recordings from single neurons.

Surgical procedures

Surgical preparation of the animal began with sedation by ketamine (15 mg/kg im) and Robinul (0.011 mg/kg im), followed by venous catheterization and endotracheal intubation. A surgical plane of anesthesia was induced with inhaled isoflurane with heart rate, blood oxygen saturation, and reflex functions monitored throughout the procedure. Scleral search coils were surgically implanted around the limbus of both eyes (Judge et al. 1980). Plastic recording cylinders were placed over 2-cm trephine holes centered at stereotaxic positions of 2 mm anteroposterior and ±15 mm mediolateral to access area MST in both hemispheres. A plastic head holder was placed in the midline behind the supraorbital ridge, and all prostheses were encased in a dental acrylic cap. Postoperative analgesia with Banamine (1 mg/kg im) was administered as judged appropriate by the veterinary staff.

Behavioral protocols

During training and recording sessions the monkey sat in a primate chair for 6 h while performing visual fixation of a red spot for liquid reward. The monkey's eye position was monitored with the magnetic search coils (Robinson 1963) to require that the eyes remained within the fixation window, centered on the fixation point at the center of the screen, during all stimulus periods. After each session the animal was returned to the vivarium.

Recording sessions were begun when the animal consistently succeeded in 90% of the training trials with a square fixation window 3° on each side and presenting the stimuli to be used in the studies. During recording sessions the fixation window was expanded to 6° per side so that saccades from the fixation point aborted a trial, but shifts of the system after hundreds of movement trials were tolerated without recalibration. In all studies, eye position was visually monitored to confirm accurate and consistent fixation under all stimulus conditions.

Each trial began with the appearance of the red fixation point 0.3° in diameter and located at the center of the screen. The monkey was required to fixate the red spot within 500 ms and continue fixation until the end of the trial. Visual motion stimuli of 2-s duration and/or translational movement stimuli of 4- to 5-s duration were presented with a 2- to 5-s intertrial interval. In most studies that compared visual, movement, and combined stimuli these were presented in a randomly interleaved sequence along with control trials in which the fixation point appeared but no optic flow or translational movement stimulus was presented. Studies combining visual and movement stimuli with different directions presented together (anticongruent combination stimuli) were conducted as a separate block of trials with pseudorandomly sequenced combinations of the visual and movement stimuli.

Neurophysiologic techniques

We used epoxy coated tungsten microelectrodes (Frederick Haer, 1.5-2.0 M at 1 kHz) that were advanced by a hydraulic microdrive (Frederick Haer). A positioning grid was secured within the recording cylinder to locate recording sites and temporarily fix a transdural needle placed to a depth of 3-5 mm above the cortical target (Crist et al. 1988). The activity of single neurons was digitized with a window discriminator and stored with stimulus and behavioral event markers.

The discharges of single neurons were electrically isolated while the animal was engaged in the behavioral task with the presentation of optic flow stimuli. If stable recording was attained the neuron was tested with hand-held projectors, first with a 3- to 6-cm × 10- to 20-cm white bar and then a (20-60 cm)² random dot pattern, to map the receptive field and derive an initial impression of visual movement responsiveness and directional preferences. Neurons having large visual receptive fields with direction selective visual motion sensitivity were accepted as putative MST neurons for further study. Neurons that satisfied some of the visual response criteria or responded to translational movement and were located near typical MST neurons were also studied.

When the location of likely MST neurons was established, we obtained magnetic resonance (MR) imaging of the brain with a microelectrode located near an optimal recording site and the transdural needle removed to lessen metal artifact. The animal underwent general anesthesia (xylazine 1.5 mg/kg, ketamine 15 mg/kg, Robinul 0.011 mg/kg iv) and was placed into an MR compatible stereotaxic head-holder (Crist). Scanning was conducted on a 1.5-T magnet (General Electric) obtaining sagital images through both hemispheres by a fast-spoiled gradient echo technique. The microelectrode was tracked from its location in the positioning grid to its terminus in the anterior bank of the superior temporal sulcus (STS), confirming a recording site in area MST. The receptive field dimensions of the neurons are consistent with previous descriptions of MST (Duffy and Wurtz 1995; Komatsu and Wurtz 1988; Tanaka et al. 1986) with 66% (120/182) occupying at least a quadrant of the stimulus area.

Data analysis

Neuronal discharge records were plotted as spike rasters and spike density histograms in which discharges were replaced by 20-ms-wide Gaussian pulses, averaged across trials, and converted to spikes/s (MacPherson and Aldridge 1979). We measured response amplitude as the mean rate of neuronal discharge during the stimulus period of 6-12 repetitions of each stimulus. Control activity was measured as the discharge rate during fixation without optic flow or translational movement stimuli.

To characterize the effects of each type of stimulus we calculated a t-value for each neuron by subtracting control activity from the maximal stimulated activity and dividing the result by the square root sum of the variances of control and stimulated activity. This provided a measure of the magnitude of the effect of the stimulus that is adjusted for the background firing rate of each neuron and the variability in the background and evoked activity. We also used this t-statistic to compare the optic flow and translational movement responses with the control activity level with Student's t-test (P < 0.01). Statistically significant responses were labeled on all data plots with filled symbols.

Circular statistical analysis

Mean discharge rates were displayed in polar plots having eight limbs, one for each direction of observer self-movement simulated by optic flow and/or induced by translational movement. The direction in which the limb extends from the origin represents the direction of simulated or translational observer movement; the length of each limb represents the mean discharge rate evoked by that stimulus. The control activity level was depicted as a dotted line circle on these polar plots with its center at the origin of the polar plot and a radius on the scale of the limbs in the polar plot.

The direction and amplitude for each response limb was used in algorithms for describing directional data by a net vector (Batschelet 1981; Mardia 1972). The net vector is the vector sum of the response limbs and summarizes the entire response profile in its mean direction and resultant length. The net vectors were plotted in each polar plot as a bold line on the same scale as the response limbs and reflect the magnitude and direction selectivity of the responses.

We calculated the mean resultant length of the net vectors that normalizes resultant length for response magnitude by dividing the resultant length by the sample size, the total number of spikes. This measure allows the quantitative comparison of the strength of direction preferences between neurons having different average activity levels. We used the Rayleigh Z statistic to test whether the net vector of a a response profile indicated the existence of a directional preference as the square of the mean resultant length multiplied by the total number of spikes. A threshold value of Z  4 was used as in a previous study of MST responses (Duffy and Wurtz 1997) and approximating a significance level P < 0.05. This statistic is used as a measure of the directionality in a response profile, indicating whether there is a clear preferred direction in any given circular distribution.

	RESULTS

Abstract Introduction Methods Results Discussion References

We studied 189 neurons recorded in the STS of three hemispheres from two monkeys. Recordings were preceded by mapping the receptive field boundaries of each neuron; neurons having large receptive fields with directional responses preferring pattern stimuli were tested with optic flow stimuli. Each study began with the presentation of 12 optic flow stimuli (8 planar, 2 radial, and 2 circular stimuli) to characterize the visual response properties of the neuron. If stable recording was maintained studies proceeded as detailed below.

Responses to optic flow simulating translation in the horizontal plane

We used optic flow stimuli simulating the visual motion patterns seen during translational self-movement in the horizontal plane. Each stimulus contained a pattern simulating self-movement along a straight line path in one of eight directions in the horizontal plane at 45° intervals around 360° (Fig. 1A). Figure 2 shows the responses of a neuron to the eight optic flow stimuli demonstrating excitation by leftward and forward optic flow and inhibition by leftward and backward. This direction selectivity is evident in the polar plot response profile (Fig. 2B) with statistically significant (Student's t-test, P < 0.01) excitatory and inhibitory responses. The net vector from the polar plot (heavy line) shows a mean direction that is to the left of forward and a resultant length that is greater than any of the individual responses. This suggests a clear preference for optic flow stimuli simulating the leftward and forward direction of self-movement.

View larger version (24K): [in this window] [in a new window]   FIG. 2. Responses of an medial superior temporal area (MST) neuron with strong direction selectivity for optic flow. A: spike density histograms (top) and raster displays (bottom) show responses to 12 presentations of each stimulus. Responses are arranged so that their relative position corresponds to the direction of simulated self-movement (Fig. 1A); the fixation control, without optic flow, is shown in the middle. The horizontal bar in the top left indicates the 2-s stimulus period. Vertical bar in the indicates a firing rate of 50 spikes/s. B: polar plot representing the amplitude of the responses shown in A. Average discharge rates for each of the 8 stimuli are shown as the polar limbs pointing in the direction of the simulated self-movement. The control activity level is indicated by the dashed circle. Filled symbols at the end of each limb mark responses that are significantly different from the control activity level (Student's t-test, P < 0.01). The heavy bar in the polar plot represents the net vector, derived by summing the vectors of each response, and here indicating a strong preference for the left-and-forward direction of simulated self-movement.

Each neuron's largest response to optic flow was measured as a t-value, normalized to its control activity by calculating the difference between the response and the control and dividing by the square root of their summed variances. The distribution of these t-values is shown in Fig. 3A for the neurons tested with optic flow. For the degrees of freedom in these studies, t-values >3 would be significant (P < 0.01). As shown here, almost one-half (48%, 72/150) of the neurons generated responses in that range, reflecting large responses to optic flow.

View larger version (26K): [in this window] [in a new window]   FIG. 3. Range of responses evoked by the 8 optic flow stimuli. A: percentage of neurons (ordinate) in which the amplitude of the largest response yielded the given t-value (abscissa) in comparison with activity during unstimulated control trials. Values of 3 are significant at the P < 0.01 level for the degrees of freedom in these studies (df  10). As shown here, 48% (72/150) of the neurons generated responses in that range. B: net vectors from each neuron's responses to the optic flow stimuli. The direction of each limb is the mean direction from the net vectors; their length is the mean resultant length of those net vectors normalized for overall response amplitude so they range from 0 to 1.

We measured the strength of each neuron's direction selectivity for optic flow by using the mean resultant length of the polar distribution. The mean resultant length is the resultant length of the net vector (Fig. 2B, heavy line) adjusted to account for overall response amplitude and scaled to range from 0 (minimal directionality) to 1 (maximal directionality) (Batschelet 1981). The mean resultant length of the net vector for the polar response profiles of each neuron shows a fairly uniform assortment of preferred directions (Fig. 3B). Most neurons (70%, 105/150) show some degree of direction selectivity with mean resultant lengths >0.2; for comparison, the distribution in Fig. 2B has a mean resultant length of 0.49. Thus, many of these neurons show direction selective responses to the eight optic flow patterns simulating self-movement in a horizontal plane.

Responses to translational movement stimuli

Translational movement was created with a two-dimensional sled system, occluding the monkey's view but for the projection screen (Fig. 4A). Eight directions of translational movement were presented to match the eight directions of movement simulated with optic flow. The sled followed straight-line paths from the center of the system to the periphery or from the the periphery to the center (Fig. 4B). We used trapezoidal velocity waveforms consisting of a 1-s period of acceleration, a 2- or 3-s period of steady speed movement, and then a 1-s period of deceleration (Fig. 4C). The steady speed used in these studies was 30 cm/s, with accelerations and decelerations at 30 cm/s².

View larger version (19K): [in this window] [in a new window]   FIG. 4. Sled movements used as stimuli in these studies. A: sled movement stimuli consisted of straight line translational movements in 1 of 8 directions, from 0 to 315° in 45° intervals. B: 8 directions of movement are shown as 8 radial limbs emanating from the center (0,0) starting position. Each series of circles marks the medial-lateral (abscissa) and anterior-posterior (ordinate) position of the sled at 200-ms intervals recorded during 5-s movements across 120 cm. C: sled movement speed profiles of 6 forward translational movement stimuli with time during the 5-s stimuli (abscissa) plotted against sled speed (ordinate). The sled moved with a trapezoidal speed waveform accelerating for 1 s (30 cm/s2), maintaining a steady speed for 2 or 3 s (30 cm/s), and then decelerating for 1 s.

Figure 5 shows the responses of an MST neuron with a strong response evoked by translational movement to the animal's left. The spike density histograms (Fig. 5A) show increasing activity during acceleration with continued firing during steady speed movement that gradually subsides during deceleration. A smaller transient response is seen in the deceleration period of the response to translational movement in the opposite direction (rightward). The latter response can be attributed to the fact that the forces experienced during acceleration in one direction are the same as those experienced with deceleration in the opposite direction. Thus this neuron shows acceleration responses along with sustained activation during steady speed movement to the left.

View larger version (20K): [in this window] [in a new window]   FIG. 5. Responses of an MST neuron with strong direction selectivity to translational movement. A: spike density histograms and raster displays of responses to translational movement, arranged so that their relative position corresponds to the movement direction (Fig. 4A). Control trial, visual fixation without translational movement, is shown in the middle. The horizontal bar in the top left corner indicates the 4-s stimulus period. The vertical bar in the top left indicates a firing rate of 25 spikes/s. B: polar plot representing the amplitude of the responses in A. Average discharge rates for each stimulus is shown as a polar limb pointing in the direction of movement. This neuron showed a strong preference for leftward movement with a mean direction of 182° and a mean resultant length of 0.44.

We focused our analysis of translational movement responses on the 2-s period of steady speed movement, although comparable analyses of the acceleration and deceleration responses yielded similar results. The polar plot in Fig. 5B is based on the steady speed responses and shows large excitatory response to leftward movement with a much smaller response to rightward movement.

The amplitude of translational movement responses was measured by the same means applied to the optic flow responses. The largest amplitude translational movement response was compared with control activity for each neuron by calculating a t-value. Figure 6A shows that 24% (35/146) of the neurons yielded a t-value >3, one-half as many neurons as reached this level with optic flow stimulation. Direction preferences for translational movement were determined for each neuron with the net vector from the polar plots. The polar distribution of the net vectors is shown in Fig. 6B with a large number of small effects and only 13% (19/146) of the neurons showing mean resultant lengths >0.2. The smaller responses to translational movement, compared with those evoked by optic flow, do not necessarily account for the smaller mean resultant lengths because mean resultant length adjusts for overall response amplitude. These findings suggest that there is far less direction selectivity in the translational movement responses than in the optic flow responses. Thus a substantial number of MST neurons respond to translational movement in darkness, but the responses are generally smaller and show less direction selectivity than the responses to optic flow.

View larger version (21K): [in this window] [in a new window]   FIG. 6. Range of responses evoked by translational movement (format as in Fig. 3). A: percentage of neurons in which the amplitude of the largest response yielded the given t-value in comparison with control activity. Twenty-four percent (35/146) of the neurons generated responses with significant t-values (3). B: net vectors from each neuron's responses to the translational movement stimuli. The mean resultant lengths of these responses are smaller than those evoked by optic flow, suggesting that less direction selectivity is evoked by translational movement.

We tested whether responses to translational movement stimuli might reflect events accompanying the animal's movement rather than the movement itself. Vocal and mechanical noises made during receptive field mapping did not evoke responses, and we heard no sounds during sled movement that might account for the direction selectivity or time course of the translational movement effects. Furthermore, loud high-pitch tones were delivered to indicate a successful trial, and loud low-pitch tones were delivered to indicate a failed trial; neither evoked responses. To control for directional cues from sled movement, we repeated the movement studies in 28 neurons after rotating the monkey 180° to face a light-emitting diode suspended at the back of the enclosure. All of the neurons showed a reversal of directional effects, confirming the link to the direction of animal movement rather than any potential artifact related to the direction of sled movement. We also tested whether apparent movement of the fixation point, induced by translational movement, might contribute to these responses. In 21 neurons we repeated translational movement studies in light-tight darkness, shutting-off the fixation point during movement, and finding no significant changes in neuronal responses.

Responses to combined optic flow and translational movement

To see how MST neurons might combine visual and vestibular signals about self-movement we compared activity evoked by optic flow alone, translational movement alone, and combined optic flow and translational movement. In the combined stimuli, the same eight directions of optic flow and translational movement were always presented with matching directions, that is, rightward translational movement was presented with the leftward visual motion optic flow stimulus, forward translational movement was presented with the centered outward radial optic flow stimulus, etc. (Fig. 1A). The optic flow stimuli were presented in the 2-s steady speed period of the translational movement stimuli (see Fig. 4C), and combined responses were measured during that period for comparison with responses to optic flow and translational movement presented alone.

Figure 7 shows the responses of an MST neuron to optic flow alone (A), translational movement alone (B), and combined optic flow and translational movement (C). The spike density histograms show strong direction selective responses to optic flow, with strong excitatory responses to optic flow simulating left-forward self-movement and strong inhibitory responses to many other optic flow stimuli (Fig. 7A). In contrast, translational movement evoked strong direction-selective responses with backward movement (Fig. 7B). Combined optic flow and translational movement evoked responses that combined the responses to optic flow and translational movement presented separately (Fig. 7C). The combined responses appeared to be dominated by optic flow responses (leftward), a mixture of optic flow and translational movement responses (left-backward), or dominated by translational movement responses (backward). This neuron is typical in that the addition of translational movement to optic flow alters the amplitude of its responses. This neuron is unusual in that it showed strong unidirectional response selectivity for both optic flow and translational movement with different preferred directions such that it clearly illustrates the effects of combined stimulation.

View larger version (15K): [in this window] [in a new window]   FIG. 7. Responses of an MST neuron in which optic flow and translational movement responses are combined in responses to combined stimulation. A-C: spike density histograms averaging responses to 6 stimulus presentations with the vertical bar marking the beginning of the 4-s peristimulus period. Icons represent the 4-s peristimulus period for each data set, graphing translational movement speed and shading the optic flow stimulus interval. Polar plots show response amplitude during the 2 s of optic flow stimulation. A: responses to optic flow stimuli showing a strong directional preference for simulated movement in the left-forward direction, with significant inhibition to simulated right-backward movements. B: responses to translational movement stimuli showing a strong directional preference for backward movement. C: responses to combined optic flow and translational movement stimuli showing strong activation to both left-forward and backward directions, combining the activation from optic flow and movement.

Across neurons in the sample we observed a similar continuum of effects in responses to combined stimuli. In some cases, the response profiles to combined stimuli were not substantially different from those evoked by optic flow alone (Fig. 8A) or translational movement alone (Fig. 8B). In other cases, responses to optic flow and translational movement were combined in the responses to combined stimuli. In some cases there were additive combinations of the optic flow and translational movement responses (Fig. 8C). In other cases there were combination responses that appeared to suppress activation seen when optic flow was presented alone (Fig. 8D). Thus MST neurons showed a wide variety of response changes with the addition of translational movement to optic flow stimuli, suggesting the potential for their differentiating between these conditions.

View larger version (31K): [in this window] [in a new window]   FIG. 8. Various patterns of neuronal activation were evident in comparisons of responses with optic flow, translational movement, and combined stimulation. A-D: responses of 4 neurons to optic flow alone (dashed lines), movement alone (dotted lines), and combined optic flow and movement stimuli (solid lines). In each graph the abscissa is the direction of simulated or real self-movement; the ordinate is response amplitude averaged across the ±2-s period of optic flow stimulation, steady speed movement, or combined stimulation. Error bars indicate ±SE of the response amplitude. Arrows indicate activity level during unstimulated control trials. A: neuron showing strong direction selectivity for optic flow simulating left-forward (135°) self-movement, with no response to translational movement alone, and a response to combined stimulation similar to that obtained with optic flow alone. B: neuron showing little direction selectivity for optic flow presented alone but strong direction selectivity for left-forward translational movement alone (135°) and a response to combined stimulation like that obtained with movement alone. C: neuron preferring optic flow simulating backward self-movement (270°) and a strong preference for backward translational movement with a response to combined stimulation that combines the responses to optic flow and movement. D: neuron preferring optic flow simulating backward self-movement (270°) and no response to translational movement with combined optic flow and movement appearing to suppress the optic flow response.

Contributions of optic flow and translational movement to combined responses

In examining the effects of translational movement on the magnitude of responses we considered the possibility that there might be larger effects in neurons that are more selectively responsive to visual simulations of translational movement. To address this issue we compared radial optic flow and translational movement responses in neurons that showed stronger responses to radial versus circular optic flow stimuli. Radial and circular preferring neurons were identified based on responses in our preliminary study of each neuron by using the same 12 basic optic flow patterns (8 planar, 2 radial, and 2 circular) used in earlier studies of MST (Duffy and Wurtz 1995, 1997). We considered all neurons with at least one significant radial or circular response and classified each neuron according to whether the largest of those responses was evoked by radial (n = 63) or circular (n = 63) motion. Response amplitude changes were measured as the ratio of the best response to optic flow alone compared with the response evoked by that stimulus when it was combined with translational movement. The addition of translational movement caused substantial response amplitude changes (ratios > 1.25) in radial (38%, 24/63) and circular (51%, 32/63) neurons, with about the same number of neurons having larger responses to optic flow alone or combined stimuli (Fig. 9, A and B). (For comparison, the change in Fig. 8B is a decrease with a ratio of 1.15; the change in Fig. 8C is an increase with a ratio of 1.57.) Thus a number of both radial and circular neurons showed substantial response changes with the addition of translational movement to radial optic flow stimuli.

View larger version (47K): [in this window] [in a new window]   FIG. 9. Both radial and circular preferring neurons showed substantial effects of combining radial optic flow with translational movement. A-D: each graph shows the percentage of neurons (ordinate) vs. the ratio of the larger response amplitude over the smaller response amplitude (abscissa). Neurons having larger responses to optic flow alone are represented on the left side of each graph, and neurons having larger responses to optic flow with translational movement are represented on the right. A and B: effects of translational movement on responses to the preferred optic flow stimulus for radial (A) and circular (B) preferring neurons. Substantial effects of adding translational movement (ratios >1.25) were seen in 38% (24/63) of radial neurons and 51% (32/63) of circular neurons, with about equal numbers of neurons preferring optic flow alone and optic flow with translational movement. C and D: effects of translational movement on the strength of direction preferences as measured by the mean resultant length of the response profiles. Substantial effects of adding translational movement (ratios >1.25) were seen in 72% (45/63) of radial neurons and 70% (44/63) of circular neurons. Two to 3 times more neurons showed larger mean resultant lengths to optic flow alone (radial, 49%; circular, 55%) and showed larger mean resultant lengths to optic flow with translational movement (radial, 23%; circular, 15%).

We also examined the effects of translational movement on the direction selectivity of optic flow responses, measured as the ratio of the mean resultant lengths to optic flow alone and to combined stimuli. Adding translational movement caused substantial changes in mean resultant lengths (ratios > 1.25) in radial (72%, 45/63) and circular (70%, 44/63) neurons (Fig. 9, C and D), with more neurons preferring optic flow alone (radial, 49%; circular, 55%) than the combined stimuli (radial, 23%; circular, 15%). Thus radial and circular neurons show substantial increases or decreases in direction selectivity with the addition of translational movement to optic flow.

The general lack of an effect of translational movement on the preferred direction of responses to combined stimuli is illustrated in Fig. 10. The scatter plot in Fig. 10A shows the mean direction of the net vector of responses to optic flow alone (abscissa) and to translational movement alone (ordinate) for each neuron. The broad scatter of points shows that there is no clear relationship between the preferred direction in responses to optic flow and to translational movement. The presence of only four filled dots indicates that only those few neurons had strong direction selectivity (Rayleigh Z  4) in both studies. The scatter plot in Fig. 10B shows similar mean directions in responses to optic flow (abscissa) and combined (ordinate) stimuli, with 50% (65/131) of the neurons having strong direction selectivity in both studies. These findings suggest little effect of translational movement on the preferred direction of responses to combined stimuli.

View larger version (25K): [in this window] [in a new window]   FIG. 10. Mean direction of net vectors to each stimulus set. A and B: each point represents the responses of 1 neuron. The filled dots are those with strong direction selectivity in both responses (Rayleigh Z  4). A: there is no evident relationship between the mean directions of responses to optic flow (abscissa) and translational movement (ordinate). That only 4 neurons showed strong direction selectivity in both studies (filled dots) reflects the lack of strong direction selectivity in the movement responses. B: many neurons showed clear similarities in the mean directions of their responses to optic flow (abscissa) and combined optic flow and movement (ordinate). These similarities were most pronounced in neurons that showed strong direction selectivity in both studies (filled dots).

However, translational movement did exert some directional influences, as suggested by the analysis of the strength of direction selectivity in the combined responses (Fig. 9, C and D). The directional effects of translational movement are seen by comparing responses with congruently combined optic flow and translational movement (matching directions presented together) with responses to anticongruently combinations (opposite directions presented together). Figure 11A is from a neuron with only small responsives to translational movement alone (left graph, - - -) and responses to optic flow alone (left graph, ) that are much the same as responses to congruently combined stimuli (right graph, ). Nevertheless, translational movement could have a substantial effect on this neuron's responses as shown by the loss of direction selectivity produced by anticongruent combination stimuli (right graph, - - -). A different type of effect is evident in the responses shown in Fig. 11B, with left-forward and right-backward preferences for optic flow alone (left graph, ) and forward preferences for translational movement (left graph, - - -). With congruent combination stimuli the left-forward stimulus is preferred (right graph, ), but with anticongruent combination stimuli the left-backward direction of optic flow is preferred (right graph, - - -).

View larger version (33K): [in this window] [in a new window]   FIG. 11. Responses of 2 neurons illustrating that congruent combinations of optic flow and movement stimuli (matching directions presented together) often yielded different responses than those of anticongruent combinations (opposite directions presented together). A and B, left: responses to optic flow alone () or movement alone (- - -) plotted as stimulus direction (abscissa) vs. average response amplitude ±SE (ordinate). Right: responses to optic flow and movement presented as congruent combinations () or anticongruent combinations (- - -) plotted as the optic flow stimulus direction (abscissa) vs. response amplitude ±SE. A: neuron showing left-forward (135°) responses to optic flow (left graph, ) and slight rightward movement preferences (left graph, - - -). With congruently combined stimuli this neuron showed a strong preference for left-forward optic flow (right graph, ), but with anticongruently combined stimuli it showed a much weaker preference for that stimulus (right graph, - - -). B: neuron showing left-forward (135°) and right-backward (315°) responses to optic flow (left graph, ) and forward responses (90°) to movement (left graph, - - -). With congruently combined stimuli this neuron preferred left-forward (135°) optic flow (right graph, ), but with anticongruently combined stimuli it preferred right-backward (315°) optic flow (right graph, - - -).

The results of such studies are shown in Fig. 12 as a scatter plot of the mean direction in responses to congruent (abscissa) and anticongruent (ordinate) combination stimuli. Over one-half of the neurons (55%, 41/75) showed similar mean directions (45°) in these studies; the remaining neurons (45%, 34/75) showed different mean directions with a tendency toward 180° differences. Changes in mean direction were more common in neurons with weaker directionality, but both groups showed comparable effects of anticongruent combination stimuli on the strength of their direction selectivity. (The ratios of the larger over the smaller mean resultant lengths were >1.25 in 56% of the neurons that maintained their direction preferences and in 69% of the neurons that changed their direction preferences.) These studies confirm that translational movement can have a substantial effect on directionality in responses to combined stimulation, although these effects may not be evident in responses to congruently combined stimuli.

View larger version (27K): [in this window] [in a new window]   FIG. 12. Mean direction of net vectors to congruently (abscissa) vs. anticongruently (ordinate) combined optic flow and translational movement stimuli. Each point represents the responses of a single neuron with the filled dots indicating strong direction selectivity in both studies (Rayleigh Z  4). About one-half of the neurons (55%, 41/75) showed little effect of reversing the movement direction on responses to the combined stimuli (those near the positively sloped diagonal). The other one-half (45%, 34/75) showed substantial changes in mean direction with reversal of the direction of movement, many tending toward 180° rotations of the mean direction in responses to the anticongruently combined stimuli.

	DISCUSSION

Abstract Introduction Methods Results Discussion References

Responses to visual and vestibular stimuli

The current studies are the first use of optic flow stimuli simulating translational self-movement in directions distributed around the observer in the horizontal plane (Fig. 1). We found unidirectional variation of response amplitude with the direction of simulated movement around the observer (Fig. 2), showing response amplitude and direction selectivity comparable with that seen in MST neurons with other optic flow stimuli (Fig. 3) (Duffy and Wurtz 1991a). Taken together, these studies suggest that MST neurons might support self-movement perception in spherical coordinates around the observer (Duffy and Wurtz 1995).

A second unique aspect of these studies is our use of translational movement stimuli presented in darkness to test MST neurons. These stimuli provide vestibular cues about self-movement in directions corresponding to those simulated with our optic flow stimuli (Fig. 4). During translational movement there was no visual motion (the monkey saw only the black enclosure and the blank projection screen) or auditory cues, and no auditory responsiveness was detected. Fixation control was maintained, and potential effects of the fixation point were excluded in studies conducted in total darkness with the fixation point blinked out during movement. Nevertheless, some neurons showed clear responses to translational movement in darkness (Fig. 5), but most responses were smaller and less directional than the visual responses (Fig. 6). MST is surrounded by cortical vestibular areas including area 2v (Buttner and Buettner 1978; Fredrickson et al. 1966; Schwarz and Fredrickson 1971), area 7 (Faugier-Grimaud and Ventre 1989; Ventre and Faugier-Grimaud 1988), the parietoinsular vestibular cortex, and the retroinsular area (Akbarian et al. 1993, 1994; Guldin et al. 1992). MST may itself receive projections from the vestibular nuclei, as suggested by tracer injections (Faugier-Grimaud and Ventre 1989), and the responses of MST pursuit neurons to body rotation in darkness suggest input from the semicircular canals to MST (Thier and Erickson 1992) and the nearby superior temporal polysensory area (Hietanen and Perrett 1996). Thus translational movement responses may reflect otolithic activation such that MST is the first identified cortical center combining visual and vestibular signals about translational self-movement.

Interactions between visual and vestibular responses

A variety of visual-vestibular interactions were revealed in comparisons among responses to optic flow, translational movement, and combined stimuli (Figs. 7 and 8). To characterize the effects of adding translational movement to optic flow stimulation we compared responses with both sets of stimuli and found enhancing and suppressive effects on response amplitude in 44% of the neurons and on the strength of direction selectivity in 71% of the neurons (Fig. 9). These effects were equally evident in both radial and circular selective neurons such that vestibular signals from translational movement alter the activity of all MST neurons. This suggests that MST relies on visual signals to provide most of its response selectivity for translational versus rotational observer movement.

This is consistent with the relatively subtle direction selectivity to translational movement (Fig. 6B) with little effect on direction preferences to combined stimuli (Fig. 10B). The effects of translational movement on directionality to combined stimuli were more evident in comparisons between responses to congruently combined stimuli (matching optic flow and translational movement directions presented together) and responses to anticongruently combined stimuli (opposite directions presented together). These comparisons reveal effects of translational movement that are not readily predicted from translational movement responses and may not be evident in comparisons between responses to optic flow and congruently combined stimuli (Fig. 11).

The observation that 45% of the neurons change direction preference with anticongruent combination stimuli (Fig. 12) shows that translational movement can alter response directionality. However, comparisons between optic flow and congruent combination responses (Fig. 10B) did not show changes in direction preference. Together these findings suggest that the directional influences of optic flow and translational movement may be aligned in these neurons. Considered in the context of substantial effects of combined stimulation on response amplitude and direction selectivity, our findings suggest that visual-vestibular interactions strengthen the activation of some neurons and decrease responsiveness and selectivity in others. Thus, to derive enhanced heading detection from these visual-vestibular interactions, there must be a mechanism for selecting the output of the subpopulation of MST neurons that show greater responsiveness and selectivity during observer self-movement.

These findings extend observations of visual cortical neurons that are influenced by vestibular signals (Denney and Adorjani 1972; Horn et al. 1972; Tomko et al. 1981). This demonstration of multisensory integration of self-movement cues supports the proposal that MST plays a critical role in the perception of self-movement (Duffy and Wurtz 1991a, 1995, 1997; Graziano et al. 1994; Orban et al. 1992; Saito et al. 1986; Tanaka and Saito 1989). The absence of strong direction preferences in many responses to translational movement but a clear impact of combined stimulation on the magnitude and direction selectivity of optic flow responses suggests that vestibular signals mainly augment visual input about heading. These findings are consistent with the conclusions of Telford et al. (1995), who found that humans use vision to accurately judge heading regardless of the presence of vestibular input, although vestibular input can guide accurate heading judgments in the absence of vision.

Vestibular signals about self-movement might help to resolve ambiguities in optic flow by selecting subpopulations of MST neurons, activating some neurons, and inactivating others. Some MST neurons are tuned to respond when optic flow is seen in the absence of translational self-movement, whereas other MST neurons are tuned to respond when optic flow is combined with translational self-movement. Thus optic flow that might result from observer or object movement could be disambiguated by combining visual and vestibular signals in area MST.

	ACKNOWLEDGEMENTS

  The author thanks R. H. Wurtz, D. J. Logan, W. K. Page, and S. J. Tetewsky for comments on the manuscript and D. M. Welch for technical assistance.

  This work was supported by National Eye Institute Grant R01-EY-10287, the Sloan Foundation, the Human Frontier Science Program Grant RG71/96, and Research to Prevent Blindness.

	FOOTNOTES

  Address for reprint requests: C. J. Duffy, Dept. of Neurology, Box 673, University of Rochester Medical Center, 601 Elmwood Ave., Rochester, NY 14642-0673.

  Received 27 October 1997; accepted in final form 9 June 1998.

	REFERENCES

Abstract Introduction Methods Results Discussion References

• AKBARIAN, S., GRUSSER, O.-J., GULDIN, W. O. Corticofugal projections to the vestibular nuclei in squirrel monkeys: further evidence of multiple cortical vestibular fields. J. Comp. Neurol. 332: 89-104, 1993.[Medline]
• AKBARIAN, S., GRUSSER, O.-J., GULDIN, W. O. Corticofugal connections between the cerebral cortex and brainstem vestibular nuclei in the macaque monkey. J. Comp. Neurol. 339: 421-437, 1994.[Medline]
• BATSCHELET, E. In: Circular Statistics in Biology. New York: Academic, 1981.
• BENSON, A. J., SPENCER, M. B., STOTT, J.R.R. Thresholds for the detection of the direction of whole-body, linear movement in the horizontal plane. Aviat. Space Environ. Med. 57: 1088-1096, 1986.[Medline]
• BUTTNER, U., BUETTNER, U. W. Parietal cortex (2v) neuronal activity in the alert monkey during natural vestibular and optokinetic stimulation. Brain Res. 153: 392-397, 1978.[Medline]
• CORNILLEAU-PERES, V., DROULEZ, J. The visual perception of three-dimensional shape from self-motion and object-motion. Vision Res. 34: 2331-2336, 1994.[Medline]
• CRIST, C. F., YAMASAKI, D.S.G., KOMATSU, H., WURTZ, R. H. A grid system and a microsyringe for single cell recording. J. Neurosci. Methods 26: 117-122, 1988.[Medline]
• DENNEY, D., ADORJANI, C. Orientation specificity of visual cortical neurons after head tilt. Exp. Brain Res. 14: 312-317, 1972.[Medline]
• DUFFY, C. J. Real movement responses of optic flow neurons in MST. Soc. Neurosci. Abstr. 22: 1692, 1996.
• DUFFY, C. J., WURTZ, R. H. Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. J. Neurophysiol. 65: 1329-1345, 1991a.[Abstract/Free Full Text]
• DUFFY, C. J., WURTZ, R. H. Sensitivity of MST neurons to optic flow stimuli. II. Mechanisms of response selectivity revealed by small-field stimuli. J. Neurophysiol. 65: 1346-1359, 1991b.[Abstract/Free Full Text]
• DUFFY, C. J., WURTZ, R. H. Response of monkey MST neurons to optic flow stimuli with shifted centers of motion. J. Neurosci. 15: 5192-5208, 1995.[Abstract]
• DUFFY, C. J., WURTZ, R. H. Medial superior temporal area neurons respond to speed in patterns of optic flow. J. Neurosci. 17: 2839-2851, 1997.[Abstract/Free Full Text]
• DUFFY, C. J., WURTZ, R. H. Planar directional contributions to optic flow responses in MST neurons. J. Neurophysiol. 77: 782-796, 1997.[Abstract/Free Full Text]
• FAUGIER-GRIMAUD, S., VENTRE, J. Anatomic connections of inferior parietal cortex (area 7) with subcortical structures related to vestibulo-ocular function in a monkey (Macaca fascicularis). J. Comp. Neurol. 280: 1-14, 1989.[Medline]
• FREDRICKSON, J. M., FIGGE, U., SCHEID, P., KORNHUBER, H. H. Vestibular nerve projection to the cerebral cortex of the rhesus monkey. Exp. Brain Res. 2: 318-327, 1966.[Medline]
• GIBSON, J. J. The visual perception of objective motion and subjective movement. Psychol. Rev. 61: 304-314, 1954.[Medline]
• GRAZIANO, M.S.A., ANDERSEN, R. A., SNOWDEN, R. J. Tuning of MST neurons to spiral motion. J. Neurosci. 14: 54-67, 1994.[Abstract]
• GULDIN, W. O., AKBARIAN, S., GRUSSER, O. J. Cortico-cortical connections and cytoarchitectonics of the primate vestibular cortex: a study in squirrel monkeys. J. Comp. Neurol. 326: 375-401, 1992.[Medline]
• HAYS, A. V., RICHMOND, B. J., OPTICAN, L. M. A Unix-based multiple process system for real-time data acquisition and control. Wescon Conf. Proc. 2: 1-10, 1982.
• HIETANEN, J. K., PERRETT, D. I. A comparison of visual responses to object- and ego-motion in the macaque superior temporal polysensory area. Exp. Brain Res. 108: 341-345, 1996.[Medline]
• HORN, G., STECHLER, G., HILL, R. M. Receptive fields of units in the visual cortex of the cat in the presence and absence of bodily tilt. Exp. Brain Res. 15: 113-132, 1972.[Medline]
• ISRAEL, I., CHAPUIS, N., GLASAUER, S., CHARADE, O., BERTHOZ, A. Estimation of passive horizontal linear whole-body displacement in humans. J. Neurophysiol. 70: 1270-1273, 1993.[Abstract/Free Full Text]
• JUDGE, S. J., RICHMOND, B. J., CHU, F. C. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res. 20: 535-538, 1980.[Medline]
• KOMATSU, H., WURTZ, R. H. Relation of cortical areas MT and MST to pursuit eye movements. I. Localization and visual properties of neurons. J. Neurophysiol. 60: 580-603, 1988.[Abstract/Free Full Text]
• LISHMAN, J. R., LEE, D. N. The autonomy of visual kinaesthesis. Perception 2: 287-294, 1973.[Medline]
• MACPHERSON, J. M., ALDRIDGE, J. W. A quantitative method of computer analysis of spike train data collected from behaving animals. Brain Res. 175: 183-187, 1979.[Medline]
• MARDIA, K. V. In: Statistics of Directional Data. London: Academic, 1972.
• ORBAN, G. A., LAGAE, L., VERRI, A., RAIGUEL, S., XIAO, D., MAES, H., TORRE, V. First-order analysis of optical flow in monkey brain. Proc. Natl. Acad. Sci. USA 89: 2595-2599, 1992.[Abstract/Free Full Text]
• PROBST, T., BRANDT, T., DEGNER, D. Object-motion detection affected by concurrent self-motion perception. Psychophysics of a new phenomenon. Behav. Brain Res. 22: 1-11, 1986.[Medline]
• ROBINSON, D. A. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans. Biomed. Eng. 10: 137-145, 1963.[Medline]
• SAITO, H.-A., YUKIE, M., TANAKA, K., HIKOSAKA, K., FUKADA, Y., IWAI, E. Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey. J. Neurosci. 6: 145-157, 1986.[Abstract]
• SCHWARZ, D.W.F., FREDRICKSON, J. M. Rhesus monkey vestibular cortex: a bimodal primary projection field. Science 172: 280-281, 1971.[Abstract/Free Full Text]
• TANAKA, K., HIKOSAKA, K., SAITO, H.-A., YUKIE, M., FUKADA, Y., IWAI, E. Analysis of local and wide-field movements in the superior temporal visual areas of the macaque monkey. J. Neurosci. 6: 134-144, 1986.[Abstract]
• TANAKA, K., SAITO, H. Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. J. Neurophysiol. 62: 626-641, 1989.[Abstract/Free Full Text]
• TELFORD, L., HOWARD, I. P., OHMI, M. Heading judgement during active and passive self-motion. Exp. Brain Res. 104: 502-510, 1995.[Medline]
• THIER, P. AND ERICKSON, R. G. Vestibular input to visual-tracking neurons in area MST of awake rhesus monkeys. In: Sensing and Controlling Motion: Vestibular and Sensorimotor Function, edited by B. Cohen, D. L. Tomko, and F. Guedry. New York: Ann. NY Acad. Sci. 1992, p. 960-963.
• TOMKO, D. L., BARBARO, N. M., ALI, F. N. effect of body tilt on receptive field orientation of simple visual cortical neurons in unanesthetized cats. Exp. Brain Res. 43: 309-314, 1981.[Medline]
• VENTRE, J., FAUGIER-GRIMAUD, S. Projections of the temporo-parietal cortex on vestibular complex in the macaque monkey (Macaca fascicularis). Exp. Brain Res. 72: 653-658, 1988.[Medline]
• YOUNG, L. R., MEIRY, J. L. A Revised dynamic otolith model. Aerospace Med. 39: 606-608, 1968.[Medline]

0022-3077/98 $5.00 Copyright ©1998 The American Physiological Society