| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1Centre for Neuroscience Studies, Canadian Institutes of Health Research Group in Sensory-Motor Systems, Department of Physiology, Queen’s University, Kingston, Ontario, Canada; 2Department of Anatomy and Neurobiology, Virginia Commonwealth University School of Medicine, Richmond, Virginia; and 3Institute for Neuroscience, Department of Biophysics, Radboud University Nijmegen, Nijmegen, The Netherlands
Submitted 29 November 2004; accepted in final form 5 February 2005
 |
ABSTRACT |
|---|
|
|
|
INTRODUCTION |
|---|
|
for review). For example, saccadic eye movements to spatially aligned visual and auditory stimuli have reduced reaction times and increased accuracy over those generated to unimodal stimuli (Colonius and Arndt 2001; Corneil et al. 2002; Frens et al. 1995). Conversely, visually guided saccades made amid spatially misaligned auditory distractors can have increased reaction times (Corneil and Munoz 1996; Frens et al. 1995). Our research focuses on how these behavioral effects of crossmodal integration are represented in the brain.
One area that has been implicated in both crossmodal integration and orienting behavior is the midbrain superior colliculus (SC) (see Stein and Meredith 1993 for review). Neurons in the intermediate/deep layers of the SC (dSC) are involved in the initiation and control of saccades (for review, see Munoz and Fecteau 2002; Sparks 1999; Sparks et al. 2001) and many of these same neurons are also capable of integrating converging sensory inputs, resulting in stronger responses (Bell et al. 2003; Frens and Van Opstal 1998; King and Palmer 1985; Meredith and Stein 1986; Populin and Yin 2002; Wallace et al. 1996). Crossmodal interactions depend on several factors. For example, they are greatest for spatially aligned stimuli and decrease in magnitude as spatial disparity increases (Bell et al. 2001a; Frens and Van Opstal 1998; Meredith and Stein 1996). Stimulus intensity also influences the magnitude of crossmodal interactions: they are greatest for pairings of weaker stimuli ("principle of inverse effectiveness"; Meredith and Stein 1986). These same factors are known to influence the behavioral benefits of crossmodal integration (Corneil et al. 2002; Stein et al. 1989).
The objective of the current study is to investigate how crossmodal processing in the dSC contributes to the reduction of SRTs to combined audiovisual stimuli. We recorded single-unit activity from neurons in the dSC while monkeys generated saccades to visual and audiovisual stimuli. We varied both the intensity and spatial relationship of the visual and auditory stimuli to elicit crossmodal interactions of different strength. Two different pairs of auditory and visual stimulus intensities were selected and on combined audiovisual trials, the visual and auditory stimuli were placed either in spatial alignment or in opposite hemifields. These stimulus combinations were presented in 2 interleaved saccade tasks. The gap-saccade task was used to facilitate the merging of the sensory- and motor-related activities, thereby increasing the probability that changes to the sensory response will affect behavior. The delayed-saccade task was used to dissociate sensory and motor activity, allowing assessment of the effect of stimulus condition on the sensory and motor responses independently. We show that the nature of the behavioral benefit provided by aligned audiovisual stimuli is different for high- versus low-intensity stimuli and that these behavioral effects are correlated with early versus later influences of the additional auditory stimulus. Preliminary data have been presented in abstract form (Bell et al. 2001b).
|
|
METHODS |
|---|
|
All procedures were approved by the Queen’s University Animal Care Committee and were in accordance with the Canadian Council on Animal Care policy on the use of laboratory animals in research. Three adult male rhesus monkeys (Macaca mulatta), weighing between 6 and 12 kg were used in this study. Animals were prepared for chronic experiments in one aseptic surgical session. Anesthesia was induced with an injection of ketamine hydrochloride [10 mg/kg, administered intramuscularly (im)] and maintained during surgery with isofluorane (1–2%), which was administered by an endotracheal tube. Heart rate, respiratory rate, and body temperature were monitored closely throughout the surgical session.
A single craniotomy (19 mm in diameter) was performed using stereotaxic techniques and a stainless steel recording chamber was centered on the midline and oriented 38–40 ° posterior from vertical, which allowed access to both SC. The recording chamber and a stainless steel head post (for securing the animal’s head during the recording sessions) were embedded in an explant formed with dental acrylic. The explant was secured to the animal’s skull using stainless steel, surgical-quality screws. Eye coils, preformed from insulated stainless steel wire (19–20 mm, 3 turns), were inserted under the conjunctiva of each eye (Judge et al. 1980) and used to measure eye position (Robinson 1963). The eyecoil leads were led subcutaneously to the explant and the connectors were embedded within the acrylic.
Animals were given prophylactic antibiotics (enrofloxacin, 5 mg/kg im) and analgesic medication (buprenorphine hydrochloride, 0.01 mg/kg im), administered daily for 7 days after surgery. They were allowed a recovery period of 
2 wk before initiation of behavioral training.
Experimental procedures and behavioral paradigm
The experiments took place in darkened, sound-attenuated rooms. The animals were seated in a primate chair (Crist Instruments) with their heads restrained. Monkeys Z and R were seated 94 cm from a tangent screen spanning approximately ±45o of the visual field. Experiments with Monkey O were performed in a different lab, in which the monkey was seated 86 cm from a tangent screen spanning approximately ±35o of the visual field. Between trials, the tangent screen was illuminated diffusely to prevent dark adaptation.
The monkeys were trained to perform 2 interleaved audiovisual saccade tasks (Fig. 1). Each trial began with the removal of the background light and the appearance of a central visual fixation point (FP) that was back-projected onto the tangent screen. The monkeys fixated the FP for a period of 800–1,200 ms, after which one of the following 2 tasks was presented: on half of the trials, the saccade target was presented but the FP remained illuminated for an additional 400–800 ms before being extinguished ("DELAYED-SACCADE TASK"; Fig. 1A). The removal of the FP was the animal’s cue to generate a saccade toward the target. On the other half of the trials, the FP was extinguished 200 ms before the presentation of the target ("GAP-SACCADE TASK"; Fig. 1B). The 2 different saccade tasks were randomly interleaved with equal distribution.
|
The visual stimulus was generated with either a laser (Power Technologies) or an LED to yield 2 very different intensities (8.0 and 0.05 cd/m2, respectively). The auditory stimulus consisted of a white noise burst, produced by small 4-cm, 8.0-
speakers suspended in front of the tangent screen, facing the animal. The stimuli were grouped to yield 2 intensity pairs: high-intensity stimuli (visual: 8.0 cd/m2; auditory: 46–51 dB, A-weighted) and low-intensity stimuli (visual: 0.05 cd/m2; auditory: 43.5 dB, A-weighted). Monkey Z performed the experiment exclusively with high-intensity stimuli, whereas Monkeys O and R performed the experiment under both conditions.
Monkeys were given a liquid reward if they maintained central fixation (i.e., held their eyes stable within 2–3° of the FP) for the duration of the fixation period and generated a saccade within 400 ms of FP disappearance (in the delayed-saccade task) or visual target onset (in the gap-saccade task) without first orienting to the auditory stimulus (in the case of the misaligned audiovisual condition). They worked until fully satiated, at which point they were returned to their home cages. Weight and water intake were recorded on a daily basis and the institute veterinarian monitored the animals closely throughout the study.
Recording techniques and receptive field mapping
Extracellular, single-neuron activity was recorded using tungsten microelectrodes (Frederick Haer) with impedances of 0.5–3 M at 1 kHz. Electrodes were driven with a hydraulic microdrive (Narishige MO-95) through stainless steel guide tubes supported by a Delrin grid placed inside the recording chamber (Crist et al. 1988). Single-neuron activity was sampled at 1 kHz after passing through a window discriminator (Bak Electronics) that excluded action potentials that did not meet both amplitude and temporal constraints. The behavioral paradigms as well as storage of eye position and neural activity were controlled by a Pentium PC running a real-time data-acquisition software package (REX Ver. 5.4; Hays et al. 1982). Horizontal and vertical eye positions were sampled at 500 Hz.
To approximate a neuron’s visual receptive field, a handheld ophthalmoscope was used to back-project moving spots and bars of light onto the tangent screen while the monkey maintained central fixation. In many instances, visual stimuli were also systematically presented throughout the visual field. The center of the receptive field was defined as the point where the maximum visual response was elicited.
Data analysis
The data were analyzed off-line using a Sun Ultra 60 Sparcstation running user-generated programs and a Pentium PC running MatLab software (The MathWorks). Data were first run through an automated saccade detection program, which identified the beginning and end of each saccade based on velocity and acceleration template matching (Waitzman et al. 1991). All marks were later verified by the experimenter and adjusted when necessary. Before analysis, all incorrect trials were eliminated. Incorrect trials were defined as: 1) those where the saccade was generated before the removal of the FP in the delayed-saccade task or during the gap in the gap-saccade task; 2) the saccade was generated toward the auditory stimulus during misaligned audiovisual trials; 3) the saccade landed outside the 2–3° acceptance window around the target; or 4) saccades with latencies >400 ms.
Neuronal responses were analyzed by constructing spike density functions (Richmond and Optican 1987) based on a normal (Gaussian) probability distribution, as follows
 | (1) |
) of 4 ms. The individual pulses were summed together to yield a single spike density function for each trial.
Neuronal responses were quantified in several ways by measuring activity in the following epochs. To evaluate the effect of the auditory stimulus on the activity preceding the onset of the visual response (Bell et al. 2003, 2004; Wallace et al. 1996), the activity 40–50 ms after target onset was measured (previsual epoch). The sensory epoch was used to measure the magnitude of the target-aligned sensory response and differed according to task and stimulus intensity. In the delayed-saccade task, the sensory epoch was defined as the peak spike density 0–200 ms after target onset for both stimulus intensities. In the gap-saccade task, the sensory epoch was defined as the peak spike density 50–100 and 70–120 ms after target onset for high- and low-intensity stimuli, respectively. The different epochs were necessary as a result of the different response onset latencies for high- versus low-intensity stimuli (see RESULTS). To minimize the inclusion of motor activity in the calculation of sensory response magnitude in the gap-saccade task, trials with SRTs <120 and <140 ms for high- and low-intensity stimuli, respectively, were excluded for this portion of the analysis.
The premotor epoch, used as an estimate of the relative state of motor readiness after the onset of the initial sensory response but before the saccade-aligned burst of activity (see RESULTS for details), was defined as the peak activity 80–90 ms after target onset for high-intensity stimuli. The threshold epoch, used as an estimate of the amount of activity necessary to evoke a saccade ("saccadic threshold"; see RESULTS and DISCUSSION; Carpenter and Williams 1995; Hanes and Schall 1996; Paré and Hanes 2003), was defined as the average level of activity from 20 to 10 ms immediately before saccade onset. This epoch was chosen because it approximates the delay between activity in the SC and saccade initiation. Microstimulation studies have shown that saccades can be elicited as early as 20 ms after dSC stimulation (Robinson 1972; Stanford et al. 1996) and perturbed or modified in midflight about 10 ms after SC stimulation (Gandhi and Keller 1999; Miyashita and Hikosaka 1996; Munoz and Wurtz 1993; Sparks and Mays 1983). The saccade-aligned activity was defined as the average activity ±5 ms surrounding saccade onset.
To estimate the onset of target-related activity, we generated a second spike density function based on an exponential growth/decay function (Thompson et al. 1996). This asymmetric activation waveform mimics an excitatory postsynaptic potential and is physiologically more plausible for estimating response onset latency than a Gaussian activation function. A spike exerts an influence only forward and not backward in time and so the symmetrical Gaussian function underestimates response onset latencies. The spike density function was obtained by convolving each spike with the following function
 | (2) |
g, the growth time constant that was set to 1 ms, and d, the decay time constant that was set to 20 ms. Response onset latency was defined as the point where the activation level exceeded baseline (defined as 400–0 and 100–0 ms before target onset in the delayed- and gap-saccade tasks, respectively) plus 3SDs. The activity had to remain above this level for a minimum of 10 ms to be classified as a valid response. Population analyses of the behavioral and neuronal data were performed using pairwise Wilcoxon signed-rank sum tests unless otherwise stated. In all instances, an alpha of 0.05 was chosen as significant. All data are presented as mean ± SE, unless otherwise indicated. For display purposes only, population spike density functions are shown as floating averages of 10-ms bin widths, plotted every 5 ms (i.e., 0–10, 5–15, 10–20 ms, etc.).
Neuron classification
Neurons in the dSC were classified into one of 3 different categories (sensory only, sensory-motor, motor only) based on their sensory and motor response properties, assessed using the delayed-saccade task. All responses were first analyzed using an automated classification scheme. A sensory response was defined as activity >50 spikes/s above baseline (see above) after target onset. Neurons with sensory activity were then classified as visual, auditory, or bimodal (responsive to both unimodal visual and auditory stimuli) based on their individual responses. Auditory-only neurons were identified as those with no visual response but that were responsive to the auditory stimulus when it alone was presented to the receptive field in the misaligned audiovisual condition. A motor response was defined as saccade-aligned activity >80 spikes/s for saccades to the neuron’s preferred direction and eccentricity in the delayed-saccade task. The experimenter later verified accuracy and consistency of all neuron classifications.
Laminar distribution of the neurons was estimated based on activity landmarks and relative depths. The superficial (dorsal-most) border of the SC was identified from the onset of brisk, visually evoked multiunit activity. The intermediate layers were then estimated to begin about 1,000–1,500 µm below this point, corresponding to the appearance of saccade-related activity (Ma et al. 1991). Neurons shallower than 1,500 µm below the dorsal-most superficial border but that displayed auditory and/or motor-activity were also classified as intermediate layer neurons (Wallace et al. 1996). Neurons recorded from the superficial layers of the SC were excluded from all further analyses.
|
|
RESULTS |
|---|
|
Three monkeys performed a total of 12,936 correct trials over the course of the recording sessions (Monkey R: 6,457, Monkey O: 4,340, Monkey Z: 2,129). Stimulus modality had no significant effect on saccadic reaction times (SRTs) in the delayed saccade task for both high-intensity (mean SRT for visual: 250 ± 2 ms, aligned audiovisual: 251 ± 2 ms, misaligned audiovisual: 254 ± 2 ms; P values >0.3) and low-intensity stimuli (mean SRT for visual: 254 ± 1 ms, aligned audiovisual: 253 ± 1 ms, misaligned audiovisual: 256 ± 1 ms; P values >0.4). Likewise, there was no significant difference in mean SRT across the 2 stimulus intensities in the delayed-saccade task for any of the 3 stimulus conditions (P values >0.5).
In the gap-saccade task, stimulus modality and intensity had a significant influence on the mean and distribution of SRTs. We first describe the behavior elicited by the 2 stimulus intensities independently and then summarize with a comparison of the two.
Saccades to high-intensity stimuli.
The distribution of SRTs for the 3 stimulus conditions (visual, aligned audiovisual, misaligned audiovisual) is shown in Fig. 2. For high-intensity stimuli, correct stimulus-triggered saccades began at about 65 ms after target onset for all stimulus conditions (Fig. 2, A, C, and E). Trials with SRTs <65 ms were classified as anticipatory responses because they had equal probability of being directed toward or away from the target (Fig. 3A) and were eliminated from further analysis.
|
|
2 = 9.35, P < 0.01). SRTs for correct, stimulus-triggered saccades formed a bimodal distribution, corresponding to express (SRTs: 65–95 ms; shaded portion, Fig. 2, A, C, and E) and regular-latency saccades (SRTs: >95 ms; Fischer and Boch 1983; Paré and Munoz 1996). These ranges did not change across the 3 stimulus conditions.
A detailed breakdown of the effect of stimulus condition on express saccade generation and mean SRT is shown in Fig. 3, B and C. High-intensity stimuli evoked relatively few express saccades, which were distributed evenly across the 3 stimulus conditions (Fig. 3B; 2 = 2.12, P > 0.3). There was no significant difference in the mean SRT of express saccades across the 3 stimulus conditions ("ES only," Fig. 3C; mean SRT for visual: 80 ± 1 ms, aligned audiovisual: 82 ± 1 ms, misaligned audiovisual: 81 ± 1 ms; Wilcoxon rank-sum tests, P values >0.2). When express saccades were included in the calculation of mean SRT, the advantage of placing an auditory stimulus on the same side as the visual target failed to reach significance ("with ES," Fig. 3C; mean SRT for visual: 158 ± 2 ms, aligned audiovisual: 154 ± 2 ms, misaligned audiovisual: 165 ± 2 ms; Wilcoxon rank-sum test, P < 0.15). However, when express saccades were excluded, a significant advantage of the aligned audiovisual condition was revealed ("no ES," Fig. 3C; mean SRT for visual: 164 ± 2 ms, aligned audiovisual: 158 ± 2 ms, misaligned audiovisual: 170 ± 2 ms, P < 0.05). There was also a trend for longer SRTs to the misaligned audiovisual condition that approached significance in both cases (P = 0.07 and P = 0.06, respectively).
Saccades to low-intensity stimuli.
Contrary to what was observed after high-intensity stimuli, the proportion of saccades generated toward versus away from the low-intensity visual target was influenced by stimulus condition (Fig. 2, B, D, and F). Figure 4A plots the performance as a function of time from target onset for each of the low-intensity stimulus conditions. Initially, all 3 low-intensity stimulus conditions averaged an equal number of correct versus incorrect responses, identifying these saccades as anticipatory responses. However, at about 55 ms after target onset, the proportion of correct saccades directed toward the aligned audiovisual stimulus (blue traces; Fig. 4A) increased. At about the same time, the proportion of correct saccades to the misaligned audiovisual stimulus (green traces; Fig. 4A) decreased substantially. This suggests that the auditory stimulus was contributing to the triggering of saccades, in the correct direction when aligned with the visual target and in the incorrect direction when presented to the opposite hemifield.
|
80% correct in the respective conditions and is thus well above "chance." To evaluate the extent to which the behavioral effects observed for low-intensity, aligned audiovisual stimuli are attributable to the additional trials with SRTs 70–100 ms (which, in the case of the other 2 low-intensity stimulus conditions, were classified as anticipatory and thus excluded), we report the values and statistics for when these trials are included and excluded. It is also important to note that cutoffs within ±10 ms of those selected yielded the same results. Saccades generated away from the saccadic target, but with SRTs beyond the range for anticipatory responses (i.e., direction errors), showed no systematic effect of stimulus condition (visual: 84/269, aligned audiovisual: 80/269, misaligned audiovisual: 105/269; 2 = 4.02, P > 0.1). The mean SRTs for the 3 low-intensity stimulus conditions are shown in Fig. 4B. Saccades to the aligned audiovisual stimulus (mean SRT: 155 ± 1 ms) had significantly shorter SRTs compared with the unimodal visual stimulus (mean SRT: 160 ± 1 ms; P < 0.001). When saccades to the aligned audiovisual stimulus with SRTs 70–100 ms were eliminated (mean SRT: 158 ± 1 ms, light blue bar, Fig. 4B), the trend persisted but failed to reach statistical significance (P = 0.08). Furthermore, saccades to the misaligned audiovisual stimulus (mean SRT: 165 ± 1) had, on average, significantly longer SRTs compared with the unimodal visual stimulus (P < 0.01).
The above analyses revealed several important differences between saccades generated to high- versus low-intensity stimuli in the gap-saccade task, which we propose are linked to early versus later influences of the auditory stimulus on activity in the dSC. To address this hypothesis, we separated the remaining analyses according to stimulus intensity into 2 independent investigations of crossmodal integration at the neuronal level and their consequences on behavior.
Analysis of neuronal activity
A total of 132 neurons were recorded from the dSC of 3 monkeys [Monkey R: 57 (high-intensity: 9; low-intensity: 48), Monkey Z: 36 (all high); Monkey O: 39 (high: 13; low: 29; N.B. 3 neurons were recorded using both intensities)]. Of these, 109 (83%) exhibited sensory and/or motor-related activity that satisfied our criteria for analysis (see METHODS). Over half of these neurons exhibited both stimulus- and saccade-related activity (64/109; 59%). Smaller proportions had stimulus-related activity but no saccade-related activity (25/109; 23%) or only saccade-related activity (20/109; 18%). The majority of neurons with stimulus-related activity responded exclusively to visual stimuli (79/89; 89%). A smaller proportion responded to both visual and auditory stimuli ("bimodal"; 9/89; 10%) and one neuron in our sample population responded to auditory stimuli only (1/89; 1%). Because this latter neuron responded only to the combined audiovisual target (and not the unimodal visual target) and because of the small sample size, it was eliminated from further analysis.
Crossmodal integration underlying saccades to low-intensity stimuli
The behavioral advantage provided by low-intensity, aligned audiovisual stimuli was driven by a reduction in the onset of the earliest correct stimulus-triggered saccades relative to the other 2 conditions (Fig. 4). Auditory stimuli are known to elicit responses in the dSC with shorter response onset latencies but decreased response magnitude compared with those elicited by visual stimuli (Bell et al. 2003, 2004; Jay and Sparks 1987; Wallace et al. 1996). We examined the effect of the auditory stimulus on activity preceding the onset of the visual response (previsual activity), the response onset latency (ROL), and the magnitude of the target-aligned sensory response.
Variable response onset latency and previsual activity.
Figure 5, A–C compares the mean activity of all sensory and sensory-motor neurons for trials where the visual target appears inside (solid traces) versus opposite to (dashed traces) the receptive field of the neuron. The point where these 2 curves diverge approximates when activity related to the sensory stimuli is being registered by the neurons. In the case of the aligned audiovisual stimulus condition (Fig. 5B), the curves initially diverge at about 30 ms, which likely represents the arrival of the earliest auditory input to the dSC. The curves then diverge dramatically at about 70–80 ms, likely representing the arrival of the visual input. Interestingly, there is also an early divergence in the case of the misaligned audiovisual condition but in the opposite direction. In this case, the auditory stimulus, located in the opposite hemifield as the visual target, falls into the receptive field of neurons represented by the dashed traces in Fig. 5 and so there is a slight increase in activity about 40 ms after target appearance (Fig. 5C).
|
|
|
Several studies have shown that when auditory and visual stimuli are presented simultaneously, the resulting sensory response in the dSC is often stronger in magnitude compared with the responses elicited by the stimuli when presented independently (see Stein and Meredith 1993 for review). To determine whether changes in the sensory response magnitude related to stimulus condition also contributed to the behavioral outcome in our task, we first examined sensory responses independent of motor processes using data obtained in the delayed-saccade task and then contrasted these results to those obtained in the gap-saccade task. Figure 7 compares the sensory response properties for the 3 low-intensity stimulus conditions for all sensory and sensory-motor neurons. Stimulus modality did not significantly influence the magnitude of the sensory response in either the delayed- (Fig. 7A) or gap-saccade tasks (Fig. 7B). A comparison of the sensory response to the visual versus audiovisual stimulus confirmed these observations (Fig. 7, C and D). These figures show the mean change in activity for the aligned- (gray) and misaligned- (black) audiovisual conditions relative to the visual condition on a neuron-by-neuron basis. Although there was a trend for stronger responses in the audiovisual conditions (particularly for the aligned audiovisual condition; gray boxes, Fig. 7, C and D) compared with the visual condition, it failed to reach significance for either stimulus condition in either task (P values >0.05). Given that there was a potent behavioral effect of the aligned audiovisual stimulus (Fig. 4), it would seem that the subtle differences in sensory response magnitude across the different stimulus conditions did not contribute to the determination of SRT in our study.
|
|
|
As shown in Fig. 3, the behavioral benefit of high-intensity aligned audiovisual stimuli was strongest for regular-latency saccades (i.e., SRTs >95 ms) and had little effect on the generation of express saccades (SRTs 65–95 ms). We assessed the contribution of changes to the ROL and sensory response magnitude to changes in SRTs of regular-latency saccades to high-intensity stimuli. Two correlation analyses were performed, examining the relationship between ROL and sensory response magnitude with the SRTs of high-intensity, regular-latency saccades (Fig. 10). All express saccades were removed from the following analyses.
|
To gain further insight into what was driving the changes in SRT, we performed a "floating correlation" analysis on all regular-latency saccades to the 3 high-intensity stimulus conditions (Fig. 11). All express saccades have been removed from this analysis, which is restricted to sensory-motor neurons only (accounting for why these curves do not match those in Fig. 9B). For this analysis, the mean spike density in 10-ms bins of all sensory-motor neurons was correlated with SRT, every 5 ms (i.e., 0–10, 5–15 ms, etc.). The initial sensory burst, which ranged from about 40 to 80 ms, exhibited relatively weak correlations. However, shortly after the peak of the sensory burst, neural activity became more and more negatively correlated with SRT, achieving the strongest negative correlation at about 100 ms after target onset (preceding saccade onset by about 50 ms). Beyond this point, activity once again became less and less correlated with SRT as it approached the range of saccade onsets. Importantly, the curves for all 3 stimulus conditions appeared similar.
|
Figure 12 compares the mean level of activity during the premotor epoch, defined here as the mean spike density 80–90 ms after target onset (shown as the shaded portion, Fig. 11), for all sensory-motor neurons (sensory-only neurons were omitted from the remaining analyses as they do not contribute to the motor output of the dSC). To avoid including activity that might be part of the saccade-aligned burst, while still including the majority of relevant trials, we excluded all trials with SRTs <110 ms.
|
To confirm that reductions in saccadic threshold or increases in the relative motor output of the dSC are not contributing to the reduction in mean SRT for regular-latency saccades to high-intensity, aligned audiovisual stimuli, we examined the effect of stimulus modality on these 2 motor epochs (see METHODS). Neither of these variables was well correlated with SRT (Fig. 13, A and B), nor did they exhibit any systematic relationship with stimulus modality (Fig. 13, C and D). Thus it would appear that high-intensity audiovisual stimuli drive changes in SRT by influencing the processes leading up to the crossing of saccadic threshold but are no longer influential once a saccade is triggered and about to be executed.
|
|
|
DISCUSSION |
|---|
|
; Corneil et al. 2002) and monkeys (e.g., Frens and Van Opstal 1998), saccades to aligned audiovisual stimuli had significantly shorter SRTs compared with either unimodal visual or misaligned audiovisual stimuli (Figs. 2–4). The reduction in mean SRT observed for low-intensity, aligned audiovisual stimuli was correlated with a reduction in the onset of the earliest correct stimulus-triggered saccades relative to the other 2 conditions, which contributed to a decrease in mean SRT (Fig. 4). High-intensity audiovisual stimuli reduced the SRTs of regular-latency saccades (Fig. 3C). These behavioral effects were associated with 2 different neural correlates (increase in activity preceding visual response with subsequent reduction in ROL for low-intensity stimuli, Figs. 5 and 6; vs. increase in premotor activity for high-intensity stimuli, Fig. 12), demonstrating how auditory stimuli and crossmodal processing can influence both the sensory and premotor activity of the dSC to evoke changes in behavior. Audiovisual stimuli accelerate the onset of stimulus-triggered saccades
Recent neurophysiological studies and models of saccadic initiation have postulated that neural activity among saccade neurons must exceed a threshold level to initiate a saccade ("saccadic threshold"; Carpenter and Williams 1995; Hanes and Schall 1996; Trappenberg et al. 2001). SRT is thus determined by the time it takes for a neural population to achieve this level of activation and any factors capable of influencing this time will have a direct impact on the SRT.
Presenting an auditory stimulus at the same time and place as the visual target in the low-intensity condition resulted in a significant increase in activity before the onset of the visual burst (Fig. 6), which appeared to facilitate the onset of the visual response relative to the other conditions (Fig. 5D). Increases in activity before the onset of the visual response have a strong impact on SRTs (Dorris et al. 1997). In both cases, the stronger activity preceding the onset of the visual burst likely facilitates the generation of shorter-latency saccades by providing a "head start" to the neural population ultimately responsible for triggering the saccade.
It would also appear that when the auditory stimulus appeared in the opposite hemifield as the visual target, a proportion of saccades was triggered to this distracting stimulus to eliminate any benefit of the additional auditory stimulus (Fig. 4). The presence of activity in the opposite dSC brought about by the misaligned auditory stimulus produced a competition between neurons in the 2 dSC (Munoz and Istvan 1998), potentially contributing to the longer SRTs observed after the misaligned audiovisual stimulus.
These data show how the combination of an auditory and visual stimulus could affect neural activity in the dSC leading to a change in behavior. This type of mechanism could represent a relatively low-level example of crossmodal integration where the response to one stimulus modality directly affects the response to another modality within the same structure. This type of mechanism could easily account for the "best-of-both worlds" result seen in crossmodal behavioral studies (e.g., Corneil et al. 2002), where subjects benefit from the spatial accuracy provided by the visual component and the shorter movement onset triggered by the auditory component of a spatially aligned audiovisual target.
One might expect a similar mechanism to influence the generation of express saccades to the high-intensity stimuli. However, this was not the case. The onset of the visual response occurred so soon after stimulus presentation (Fig. 8), leaving little opportunity for the auditory stimulus to bias the previsual activity before the arrival of the visual response in the dSC. Essentially, all 3 high-intensity stimulus conditions were evoking responses too soon after target onset to evoke a differential proportion of express saccades under the conditions used in this study.
Audiovisual stimuli enhance premotor activity in the dSC
As mentioned previously, saccades are believed to be triggered when neural activity exceeds saccadic threshold. It has been demonstrated in the dSC (Paré and Hanes 2003) and frontal eye fields (Hanes and Schall 1996) that variance in SRT is linked to changes in the rate of rise of premotor activity among saccade neurons: the steeper the rise, the sooner the threshold will be crossed and the shorter the SRT. These observations were made in a countermanding task, which produces a broad distribution of SRTs. An attempt to calculate the rate of rise of premotor activity (i.e., the slope of the spike density function preceding saccade onset) with our data set proved extremely difficult. Specifically, there was insufficient variance in SRT to allow subtle changes in slope to show meaningful correlations. In addition, because SRTs in our task were so short (Fig. 2), it was difficult to locate a period of time that consistently offered a stable linear rise in neural activity from trial to trial.
We therefore used a different approach to characterize the premotor activity of neurons in the dSC (Fig. 11). Using a floating correlation analysis, we identified a premotor epoch that was significantly correlated with SRT. High-intensity audiovisual stimuli evoked significantly greater premotor activity compared with the unimodal visual condition, thus facilitating the earlier crossing of saccadic threshold (Fig. 12). This significant enhancement of neural activity was strongest for spatially aligned audiovisual stimuli, likely representing a crossmodal interaction between the visual and auditory stimuli. Although the initial sensory response did not show a significant enhancement (Fig. 9), which could indicate a ceiling effect for such highly salient visual stimuli, these results show that the benefit of crossmodal interactions between auditory and visual stimuli can extend beyond the initial sensory response and affect premotor processing as well.
Interestingly, the misaligned audiovisual stimulus showed a similar trend that failed to reach significance (Fig. 12). This somewhat counterintuitive result suggests that the benefit of auditory stimuli could, in fact, be 2-fold. Presenting an auditory stimulus in combination with a visual target can provide additional spatial information (if presented in spatial alignment with the visual stimulus), in essence serving as a redundant target (Forster et al. 2002; Miller 1991; Murray et al. 2001). The auditory stimulus can also provide a potent nonspatial "alerting effect" (e.g., Farah et al. 1989; Ross and Ross 1981) that can further serve to facilitate orienting. These 2 effects are consistent with a recently proposed 2-stage model of crossmodal integration (Arndt and Colonius 2003; Colonius and Arndt 2001). These authors propose that the influence of stimulus intensity and location on SRT constitutes 2 separable mechanisms linked to the early unimodal pathways and later integrative processes, respectively. In the misaligned audiovisual condition, monkeys are still able to benefit from this latter effect, particularly with a highly salient auditory stimulus. The misaligned auditory stimulus likely served as a nonspatial "alerting" cue to trigger saccades with shorter SRTs, which may account for the increased premotor activity compared with the unimodal visual stimulus (Fig. 12).
One interesting question still remains: what is driving the observed effect of stimulus modality on premotor activity? Although our data do not address this question directly, one logical possibility is that the cerebral cortex is somehow involved. The importance of cortical inputs for crossmodal integration in the cat dSC has been demonstrated (e.g., Jiang et al. 1999, 2001; Wallace and Stein 1997, 1999; Wilkinson et al. 1996). Perhaps the effect of stimulus modality on premotor activity depends on descending cortical inputs that are absent or less active under unimodal conditions. Further study will be necessary to better understand the relationship between the SC, cortex, and crossmodal integration in awake monkeys.
Interaction between spatial location and stimulus intensity
Consistent with previous studies (Frens and Van Opstal 1998; Meredith and Stein 1996; Stein et al. 1989; Wallace et al. 1998), the behavioral and neuronal consequences of audiovisual stimulation depended on their spatial alignment and relative intensity. The interaction between these 2 variables, however, revealed an additional interesting point. The majority of direction errors in the high-intensity stimulus sessions were generated to the misaligned audiovisual stimulus, whereas those generated in the low-intensity stimulus sessions were equally distributed among the 3 stimulus conditions. Furthermore, if the saccade was directed toward the visual target correctly in the misaligned condition, monkeys did not show a significant increase in SRT for the high-intensity stimuli but did for the low-intensity stimuli (Figs. 3 and 4). Thus it is possible that in the high-intensity case, the monkeys were either able to ignore the spatial ambiguity of the auditory stimulus or were distracted by it to the point that they oriented toward the stimulus incorrectly (often correcting themselves in a subsequent saccade). In the case of the low-intensity stimuli, monkeys appeared less prone to orient to the auditory stimulus but still showed a negative effect on behavior. This effect of stimulus intensity is unlike the principle of inverse effectiveness previously described in the anesthetized preparation (Meredith and Stein 1986; Wallace et al. 1996) and thus represents one more example of how crossmodal integration in the awake monkey is subject to different/additional factors compared with the anesthetized animal (e.g., state of visual fixation: Bell et al. 2003; spatial attention: Bell and Munoz 2002).
In conclusion, we have shown that aligned audiovisual stimuli facilitate shorter latency saccades in at least 2 ways. In the case of low-intensity stimuli, aligned audiovisual stimuli reduced the mean SRT by increasing the proportion of shorter-latency saccades through a reduction in the ROLs of neurons in the dSC. In the case of high-intensity stimuli, the aligned audiovisual condition increased the premotor activity of dSC neurons, facilitating the generation of regular-latency saccades with shorter latencies. Thus our results demonstrate that crossmodal interactions in the dSC go beyond the sensory response to also influence premotor processing and orienting behavior.
|
|
GRANTS |
|---|
|
|
|
ACKNOWLEDGMENTS |
|---|
|
|
|
FOOTNOTES |
|---|
Address for reprint requests and other correspondence: D. Munoz, Centre for Neuroscience Studies, Queen’s University, Kingston, Ontario, Canada K7L 3N6 (E-mail: doug{at}eyeml.queensu.ca)
|
|
REFERENCES |
|---|
|
Bell AH, Corneil B D, Meredith MA, and Munoz DP. The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. Can J Exp Psychol 55: 123–132, 2001a.[Web of Science][Medline]
Bell AH, Corneil BD, Munoz DP, and Meredith MA. Engagement of visual fixation suppresses sensory responses and multisensory integration in the primate superior colliculus. Eur J Neurosci 18: 2867–2873, 2003.[CrossRef][Web of Science][Medline]
Bell AH, Fecteau JH, and Munoz DP. Using auditory and visual stimuli to investigate the behavioral and neuronal consequences of reflexive covert orienting. J Neurophysiol 91: 2172–2184, 2004.
Bell AH, Meredith MA, Van Opstal AJ, Van Wanrooij M, and Munoz DP. Multisensory integration in the primate superior colliculus: saccades to visual and audiovisual stimuli. Soc Neurosci Abstr 27: 681.2, 2001b.
Bell AH and Munoz DP. Activity in the primate superior colliculus related to performance in predictive vs. non-predictive multimodal cued-saccade tasks. Soc Neurosci Abstr 28: 560.5, 2002.
Calvert GA, Spence C, and Stein BE. The Handbook of Multisensory Processes. Cambridge, MA: MIT Press, 2004.
Carpenter RH and Williams ML. Neural computation of log likelihood in control of saccadic eye movements. Nature 377: 59–62, 1995.[CrossRef][Medline]
Colonius H and Arndt P. A two-stage model for visual-auditory interaction in saccadic latencies. Percept Psychophys 63: 126–147, 2001.[Web of Science][Medline]
Corneil BD and Munoz DP. The influence of auditory and visual distractors on human orienting gaze shifts. J Neurosci 16: 8193–8207, 1996.
Corneil BD, Van Wanrooij M, Munoz DP, and Van Opstal AJ. Auditory–visual interactions subserving goal-directed saccades in a complex scene. J Neurophysiol 88: 438–454, 2002.
Crist CF, Yamasaki DS, Komatsu H, and Wurtz RH. A grid system and a microsyringe for single cell recording. J Neurosci Methods 26: 117–122, 1988.[CrossRef][Web of Science][Medline]
Dorris MC, Paré M, and Munoz DP. Neuronal activity in monkey superior colliculus related to the initiation of saccadic eye movements. J Neurosci 17: 8566–8579, 1997.
Farah MJ, Wong AB, Monheit MA, and Morrow LA. Parietal lobe mechanisms of spatial attention: modality-specific or supramodal? Neuropsychologia 27: 461–470, 1989.[CrossRef][Web of Science][Medline]
Fischer B and Boch R. Saccadic eye movements after extremely short reaction times in the monkey. Brain Res 260: 21–26, 1983.[CrossRef][Web of Science][Medline]
Forster B, Cavina-Pratesi C, Aglioti SM, and Berlucchi G. Redundant target effect and intersensory facilitation from visual-tactile interactions in simple reaction time. Exp Brain Res 143: 480–487, 2002.[CrossRef][Web of Science][Medline]
Frens MA and Van Opstal AJ. Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus. Brain Res Bull 46: 211–224, 1998.[CrossRef][Web of Science][Medline]
Frens MA, Van Opstal AJ, and Van der Willigen RF. Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Percept Psychophys 57: 802–816, 1995.[Web of Science][Medline]
Gandhi NJ and Keller EL. Comparison of saccades perturbed by stimulation of the rostral superior colliculus, the caudal superior colliculus, and the omnipause neuron region. J Neurophysiol 82: 3236–3253, 1999.
Hanes DP and Schall JD. Neural control of voluntary movement initiation. Science 274: 427–430, 1996.
Hays AV, Richmond BJ, and Optican LM. A UNIX-based multiple process system for real-time data acquisition and control. WESCON Conf Proc 2: 1–10, 1982.
Jay MF and Sparks DL. ensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals. Neurophysiol 57: 35–55, 1987.
Jiang W, Wallace MT, Jiang H, Vaughan JW, and Stein BE. Two cortical areas contribute to the multisensory integration in superior colliculus neurons. Soc Neurosci Abstr 25: 1413, 1999.
Jiang W, Wallace MT, Jiang H, Vaughan JW, and Stein BE. Two cortical areas mediate multisensory integration in superior colliculus neurons. J Neurophysiol 85: 506–522, 2001.
Judge SJ, Richmond BJ, and Chu FC. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res 20: 535–538, 1980.[CrossRef][Web of Science][Medline]
King AJ and Palmer AR. Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp Brain Res 60: 492–500, 1985.[Web of Science][Medline]
Ma TP, Graybiel AM, and Wurtz RH. Location of saccade-related neurons in the macaque superior colliculus. Exp Brain Res 85: 21–35, 1991.[Web of Science][Medline]
Meredith MA and Stein BE. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56: 640–662, 1986.
Meredith MA and Stein BE. Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75: 1843–1857, 1996.
Miller J. Channel interaction and the redundant-targets effect in bimodal divided attention. J Exp Psychol Hum Percept Perform 17: 160–169, 1991.[CrossRef][Web of Science][Medline]
Miyashita N and Hikosaka O. Minimal synaptic delay in the saccadic output pathway of the superior colliculus studied in awake monkey. Exp Brain Res 112: 187–196, 1996.[Web of Science][Medline]
Munoz DP and Fecteau JH. Vying for dominance: dynamic interactions control visual fixation and saccadic initiation in the superior colliculus. Prog Brain Res 140: 3–19, 2002.[Web of Science][Medline]
Munoz DP and Istvan PJ. Lateral inhibitory interactions in the intermediate layers of the monkey superior colliculus. J Neurophysiol 79: 1193–1209, 1998.
Munoz DP and Wurtz RH. Fixation cells in monkey superior colliculus. II. Reversible activation and deactivation. J Neurophysiol 70: 576–589, 1993.
Murray MM, Foxe JJ, Higgins BA, Javitt DC, and Schroeder CE. Visual-spatial neural response interactions in early cortical processing during a simple reaction time task: a high-density electrical mapping study. Neuropsychologia 39: 828–844, 2001.[CrossRef][Web of Science][Medline]
Paré M and Hanes DP. Controlled movement processing: superior colliculus activity associated with countermanded saccades. J Neurosci 23: 6480–6489, 2003.
Paré M and Munoz DP. Saccadic reaction time in the monkey: advanced preparation of oculomotor programs is primarily responsible for express saccade occurrence. J Neurophysiol 76: 3666–3681, 1996.
Populin LC and Yin TC. Bimodal interactions in the superior colliculus of the behaving cat. J Neurosci 22: 2826–2834, 2002.
Richmond BJ and Optican LM. Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. II. Quantification of response waveform. J Neurophysiol 57: 147–161, 1987.
Robinson DA. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans Biomed Eng 10: 137–145, 1963.[Medline]
Robinson DA. Eye movements evoked by collicular stimulation in the alert monkey. Vision Res 12: 1795–1808, 1972.[CrossRef][Web of Science][Medline]
Ross SM and Ross LE. Saccade latency and warning signals: effects of auditory and visual stimulus onset and offset. Percept Psychophys 29: 429–437, 1981.[Web of Science][Medline]
Sparks DL. Conceptual issues related to the role of the superior colliculus in the control of gaze. Curr Opin Neurobiol 9: 698–707, 1999.[CrossRef][Web of Science][Medline]
Sparks DL, Freedman EG, Chen LL, and Gandhi NJ. Cortical and subcortical contributions to coordinated eye and head movements. Vision Res 41: 3295–3305, 2001.[CrossRef][Web of Science][Medline]
Sparks DL and Mays LE. Spatial localization of saccade targets. I. Compensation for stimulation-induced perturbations in eye position. J Neurophysiol 49: 45–63, 1983.
Stanford TR, Freedman EG, and Sparks DL. Site and parameters of microstimulation: evidence for independent effects on the properties of saccades evoked from the primate superior colliculus. J Neurophysiol 76: 3360–3381, 1996.
Stein BE and Meredith MA. The Merging of the Senses. Cambridge, MA: MIT Press, 1993
Stein BE, Meredith MA, Huneycutt WS, and McDade L. Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J Cogn Neurosci 1: 12–24, 1989.
Thompson KG, Hanes DP, Bichot NP, and Schall JD. Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol 76: 4040–4055, 1996.
Trappenberg TP, Dorris MC, Munoz DP, and Klein RM. A model of saccade initiation based on the competitive integration of exogenous and endogenous signals in the superior colliculus. J Cogn Neurosci 13: 256–271, 2001.[CrossRef][Web of Science][Medline]
Waitzman DM, Ma TP, Optican LM, and Wurtz RH. Superior colliculus neurons mediate the dynamic characteristics of saccades. J Neurophysiol 66: 1716–1737, 1991.
Wallace MT, Meredith MA, and Stein BE. Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80: 1006–1010, 1998.
Wallace MT and Stein BE. Development of multisensory neurons and multisensory integration in cat superior colliculus. J Neurosci 17: 2429–2444, 1997.
Wallace MT and Stein BE. The development of corticotectal influences is necessary for the appearance of multisensory integration in the superior colliculus. Soc Neurosci Abstr 25: 1999.
Wallace MT, Wilkinson LK, and Stein BE. Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76: 1246–1266, 1996.
Wilkinson LK, Meredith MA, and Stein BE. The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp Brain Res 112: 1–10, 1996.[Web of Science][Medline]
This article has been cited by other articles:
|
|
 |
|
  B. A. Rowland, S. Quessy, T. R. Stanford, and B. E. Stein Multisensory Integration Shortens Physiological Response Latencies J. Neurosci., May 30, 2007; 27(22): 5879 - 5884. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Avillac, S. Ben Hamed, and J.-R. Duhamel Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey J. Neurosci., February 21, 2007; 27(8): 1922 - 1932. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Ramat, R. J. Leigh, D. S. Zee, and L. M. Optican What clinical disorders tell us about the neural control of saccadic eye movements Brain, January 1, 2007; 130(1): 10 - 35. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. A. Whitchurch and T. T. Takahashi Combined Auditory and Visual Stimuli Facilitate Head Saccades in the Barn Owl (Tyto alba) J Neurophysiol, August 1, 2006; 96(2): 730 - 745. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
TOP
ABSTRACT
INTRODUCTION
P < 0.10).
