Cortex Controls Multisensory Depression in Superior Colliculus

Wan Jiang, Barry E. Stein


Multisensory depression is a fundamental index of multisensory integration in superior colliculus (SC) neurons. It is initiated when one sensory stimulus (auditory) located outside its modality-specific receptive field degrades or eliminates the neuron's responses to another sensory stimulus (visual) presented within its modality-specific receptive field. The present experiments demonstrate that the capacity of SC neurons to engage in multisensory depression is strongly dependent on influences from two cortical areas (the anterior ectosylvian and rostral lateral suprasylvian sulci). When these cortices are deactivated, the ability of SC neurons to synthesize visual-auditory inputs in this way is compromised; multisensory responses are disinhibited, becoming more vigorous and in some cases indistinguishable from responses to the visual stimulus alone. Although obtaining a more robust multisensory SC response when cortex is nonfunctional than when it is functional may seem paradoxical, these data may help explain previous observations that the loss of these cortical influences permits visual orientation behavior in the presence of a normally disruptive auditory stimulus.


One of the characteristic properties of superior colliculus (SC) neurons is their ability to integrate information from different senses, a property believed to facilitate the role of the SC in orientation behavior (Stein and Meredith 1993). Two response patterns have been shown to be reflective of this process of multisensory integration. The first, called multisensory enhancement, has as its criterion a significant increase in a neuron's response to a cross-modal stimulus pair (e.g., visual and auditory) above that generated by the most effective of these two modality-specific stimuli presented individually (Meredith and Stein 1983; see also Bell et al. 2001; Frens and Van Opstal 1997; King and Palmer 1985; Peck 1987; Wallace et al. 1996). The second, called multisensory depression, has as its criterion a significant decrease in the neuron's response to the cross-modal stimulus pair as compared with the most effective of these stimuli individually (Kadunce et al. 1997; Meredith and Stein 1985, 1986a,b).

These two opposing response patterns can be evoked in the same neuron, depending on the spatial relationships among the stimuli and their respective receptive fields. Multisensory enhancement is generally initiated when two different sensory stimuli originate from the same location (e.g., derived from the same event), such that they fall within the overlapping visual and auditory receptive fields of the same multisensory SC neuron. Similar response enhancement has been noted in the sensory cortex of animals (e.g., Barth et al. 1995; Fishman and Michael 1972; Jiang et al. 1994b; Morrell 1972; Wallace et al. 1992) and humans (e.g., see Calvert et al. 2000; Foxe et al. 2000; Giard and Peronnet 1999; Macaluso et al. 2002; Sams et al. 1991; Schroger and Widmann 1998). Multisensory depression is generally initiated when these sensory stimuli are spatially disparate (e.g., derived from different events) such that one is eccentric to its receptive field and the other is within its receptive field. Although multisensory depression is less well studied than enhancement, it too has been noted in multisensory cortical neurons of animals (Wallace et al. 1992) and in fMRI studies of human cortex (Laurienti et al. 2002).

The results of similar studies at the behavioral level support the contention that these neuronal properties parallel and may mediate an animal's multisensory orientation responses. Behavioral studies showed that an animal's success rate in orienting to a dim visual target was significantly enhanced by the presence of an auditory stimulus at the same location and significantly degraded by that auditory stimulus when it was disparate from the visual target. The auditory stimulus proved to be an effective modulator of the visual orientation response regardless of whether animals had been trained to respond to it, ignore it, or had no prior experience with it (Jiang et al. 2002; Stein et al. 1988, 1989; Wilkinson et al. 1996). Similar effects on behavior and perception have been obtained in studies with human subjects (see Amlot et al. 2003; Bermant and Welch 1976; Bernstein et al. 1969; Corneil and Munoz 1996; Doyle and Walker 2002; Engelken and Stevens 1989; Frasinetti et al. 2002; Frens et al. 1995; Harrington and Peck 1998; Hughes et al. 1994; Lovelace et al. 2003; Nozawa et al. 1994; Perrott et al. 1990; Radeau 1994; Stein et al. 1996; Thomas 1941).

Surprisingly, it has been shown that multisensory enhancement is not an inevitable consequence of an SC neuron receiving cross-modal inputs. The neuron's ability to integrate its multiple sensory inputs to enhance responses has been shown to require the addition of influences derived from two regions of cortex: the anterior ectosylvian sulcus (AES) and the rostral lateral suprasylvian sulcus (rLS). Both cortical areas have robust direct projections to the ipsilateral SC (see McHaffie et al. 1998; Stein et al. 1993) and both contain a mixture of neurons responsive to different sensory modalities (Jiang et al. 1994a,b; Wallace et al. 1992). However, the multisensory properties of the AES are better understood than those of the rLS. AES is divided into an auditory area (Clarey and Irvine 1986), a visual area (Benedek et al. 1988; Mucke et al. 1982; Olson and Graybiel 1987), and a somatosensory area (Burton and Kopf 1984; Clemo and Stein 1982, 1983). These areas are largely modality-specific, but multisensory neurons are found at the borders between these areas. When AES and/or rLS were reversibly deactivated, multisensory enhancement in SC neurons was eliminated despite the fact that the neuron continued to be multisensory and could thus respond to each of the previously effective modality-specific stimuli (Jiang et al. 2001). Analogous findings have been reported at the level of overt orientation behavior (Jiang et al. 2002; Wilkinson et al. 1996).

These experimental results have led to the belief that the general process of SC multisensory integration is dependent on cortical influences. However, this conclusion is based largely on studies of multisensory enhancement. There is not a comparable body of evidence on the other measure of multisensory integration: multisensory depression. Indeed, the functional loss of AES and rLS has been found to minimize rather than eliminate multisensory response depression at the level of overt orientation behavior (Jiang et al. 2002; also Wilkinson et al. 1996). It is not yet known whether AES and/or rLS deactivation would have a similar disruptive effect on response depression at the single SC neuron level and, if so, whether this effect would be comparable to that found in neuronal and behavioral studies of multisensory enhancement. The purpose of the present study was to explore this issue using auditory inhibition of visual responses as the model.


All survival surgery was conducted using aseptic techniques and in accordance with the Guide for the Care and Use of Laboratory Animals (National Institutes of Health Publication 86-23) and an approved IACUC protocol.

Surgical implantation

Animals were pretreated with ketamine hydrochloride (30 mg/kg im) and acepromazine maleate (3–5 mg/kg im), intubated, and anesthetized with isoflurane (0.5–3%). Body temperature (37–38°C) was maintained with a hot water heater. The animal was placed into a stereotaxic head-holder, and craniotomies exposed the left AES, rLS, and the cortex overlying both the left and right SC. The dura over AES and rLS was opened, the sulcal walls were separated, and two cooling coils were inserted into AES and one into rLS (see Jiang et al. 2001). The area was covered with gelfoam and sealed with orthopedic cement. A hollow cylinder was attached to the skull to provide access to the SC and to hold the animal's head during recording (McHaffie and Stein 1983). The animal was given postsurgical analgesics (butorphanol tartrate, 0.1–0.4 mg · kg–1 · 6 h–1) as needed, and received antibiotic treatments for 7–10 days (ceftriaxone, 20 mg · kg–1 · bid–1, or enrofloacin,5 mg · kg–1 · bid–1).


Anesthesia was induced with ketamine (30 mg/kg im, then 0.05–0.1 mg · kg–1 · h–1 iv), fluids (5% dextrose Ringer: 3–6 ml/h iv) were infused, and the animal was artificially respired and paralyzed (pancuronium bromide, initial dose: 0.3 mg/kg, then 0.1-0.2 mg · kg–1 · h–1 iv). End tidal CO2 was maintained at ∼4.0% and body temperature at ∼38°C. Heart rate was continuously monitored (Nonin 8600C). The eye contralateral to the SC that was recorded from was dilated with 1% atropine sulfate, and a contact lens corrected its refractive errors. An opaque lens occluded the other eye. At the end of the recording session, drugs were discontinued and, when stable respiration and locomotion returned, the animal was returned to its home cage. After the terminal recording session the animal was killed (pentobarbital sodium: 100 mg/kg iv).

Receptive field mapping

Visual-auditory neurons were sought using moving and stationary flashed visual stimuli and hisses, clicks, claps, whistles, and 20- to 20,000-Hz noise bursts. Visual receptive fields were mapped with moving bars and spots of light, and auditory receptive fields were mapped using broad-band noise bursts (generally 10 dB above threshold) using 16 hoop-mounted speakers placed 15° apart and 15 cm from the head on a rotating hoop so that elevation could be examined (see Meredith and Stein 1986a,b). Receptive fields were plotted on standardized representations of visual-auditory space (see Stein and Meredith 1993) and reexamined during different stages of the recording series (i.e., during and after cortical deactivation) to ensure that the relative positions of the stimuli had not changed during the testing session.

General testing paradigm

The onset, duration, physical parameters, cross-modal stimulus onset asynchronies, and intertrial intervals were controlled independently. Visual stimuli consisted of computer-controlled moving bars of light generated by a Barcodata projector. Bars and spots of light (0.11–13.0 cd/m2 against a background of 0.10 cd/m2) were projected onto the tangent screen and could be moved in all directions across the receptive field at amplitudes of 1–110° and speeds of 1–400°/s. Auditory stimuli were computer-controlled broad-band noise bursts delivered from any of the speakers. Auditory stimulus duration varied from 20 to 100 ms at intensities of 55–70 dB SPL against a background SPL of 51.4–52.0 dB. A neuron's multisensory integrative properties were first explored with spatially aligned visual-auditory stimuli (to induce multisensory enhancement). Responses to each modality-specific stimulus (visual alone, auditory alone) and to the multisensory combination (visual-auditory) were determined quantitatively by presenting each category of stimulus (interleaved) 8–10 times at 8- to 20-s intertrial interval. In multisensory trials, the two stimuli were either simultaneous or within 20–200 ms of one another. To study multisensory depression, the visual stimulus was presented in the center of the visual receptive field and the auditory stimulus was presented outside the auditory receptive field (either centrally or peripherally). The interstimulus interval and the parameters (e.g., intensity, size, position, motion direction, and speed, etc.) of each modality-specific stimulus were chosen based on previous studies (Kadunce et al. 1997; Meredith and Stein 1996) to maximize multisensory depression.

Cortical deactivation

Deactivation was induced via the indwelling cooling coils. The coils (7–11 × 3–7 mm) were shaped from loops of 21-gauge stainless steel hypodermic tubing to fit the sulci. Refrigerated water (0°C) was circulated through the coils to deactivate cortex and warm (36–38°C) water to reactivate cortex. Cortical temperature around the coil quickly decreased to ∼10°C in 2–3 min of coolant circulation, and the temperature stabilized at this level with the cooling continued. The neurons in AES and rLS ceased both spontaneous and sensory-evoked discharges at a temperature <17–19°C (Jiang et al. 2001; see also Horel 1991; Lomber et al. 1999). Deactivation was maintained for ∼10–20 min to complete all tests. The effective decrease in cortical temperature was circumscribed to an area of about a 2-mm radius around each coil. Beyond this point, the decrement in cortical temperature did not preclude neural activity (i.e., at 3 mm from the coil cortical temperature remained >24°C) (see Jiang et al. 2001). Given that the distance between the AES and rLS was ∼6 mm or more, there was little likelihood that deactivation of one area affected the other. Both the cortical temperature and the responsiveness of cortical neurons were reinstated ∼2 min after the onset of rewarming.

Data acquisition and analysis

Each neuron served as its own control. The identical stimulus parameters were used during the baseline, or predeactivation “control” condition, during cortical deactivation trials and during the reactivation condition. To avoid misinterpreting response changes due to possible mechanical factors, neurons were excluded from data analysis if they showed a significant difference in response patterns in the reactivation and predeactivation control periods. Each neuron's response (number of impulses) to each modality-specific and crossmodal stimulus combination was measured using the time window that bracketed the longest response train. Although spontaneous rates were low, responses were corrected by subtracting spontaneous activity (i.e., the number of impulses measured in a 1-s interval preceding the 1st stimulus and normalized for the time window in which responses were counted). The timing of onset and offset of a neuron's discharge train was determined by the beginning of the first, and the end of the last, bin (5 ms) at which the mean number of impulses significantly exceeded the average spontaneous firing (P < 0.05). The same time window was used to measure responses of a given neuron in each of the different stimulus conditions (i.e., modality-specific, multisensory, control, and cortical deactivation). Statistical analysis was performed with SYSTAT (SPSS). The criterion for multisensory depression was the same under all conditions: a statistically significant (Student's t-test, P < 0.05) decrease in the response to the cross-modal stimulus combination as compared with the dominant (in this case, the visual) modality-specific response. In instances in which the statistical criterion for multisensory response depression was reached, the magnitude of the multisensory response (% change) was then calculated using the same formula as used for calculating multisensory enhancement (see Meredith and Stein 1983) Math where CM = the mean number of impulses evoked by the multisensory (visual-auditory) stimulus and SMmax = the mean number of impulses evoked by the “dominant” (i.e., the most effective, in this case, visual) modality-specific stimulus.

When evaluations involved the effect of cortical deactivation on multisensory depression, the control response in the text refers to the depressed multisensory response obtained in a spatial disparity paradigm without cortical deactivation. The experimental or treatment response is the one obtained in the same stimulus paradigm, but during cortical deactivation. For each neuron, the effect of cortical deactivation on the modality-specific responses was determined by a Student's t-test. The difference in the multisensory responses obtained in the control and cortical deactivation conditions was assessed using ANOVA (a possible interaction between the treatment and the modality-specific response was incorporated into the analysis).


Multisensory depression

The modality-specific visual and auditory responses and the multisensory responses to the combined presentation of these stimuli in spatially disparate configurations were recorded from 124 visual-auditory SC neurons in four cats. These neurons were preselected for study from the larger population of multisensory neurons (n = 159) sampled in these experiments. The preselection was based on a neuron's ability to exhibit a significantly (P < 0.05) enhanced multisensory response when the visual and auditory stimuli were in spatial register with one another and within their respective receptive fields (e.g., see Jiang et al. 2001). This preselection criterion was used to focus the sample, as neurons capable of multisensory depression are a subset of neurons exhibiting multisensory enhancement (Kadunce et al. 1997; Meredith and Stein 1996).

The visual and auditory receptive fields of each multisensory neuron were in spatial register and thus overlapped one another (see also Stein and Meredith 1993). The test for multisensory depression involved positioning the two stimuli at different locations in space and at different locations with respect to their individual receptive fields. The visual stimulus was placed within each neuron's visual receptive field, and its parameters were manipulated to ensure that it evoked reliable discharges. The auditory stimulus was placed outside the neuron's auditory receptive field, where it evoked no responses. This particular stimulus configuration, in which the visual stimulus is within its receptive field and the auditory stimulus outside its excitatory receptive field, has been shown to produce the highest incidence and greatest magnitude of multisensory depression in SC neurons (Kadunce et al. 1997), and, in the present study, it resulted in 69% (86/124) of the sampled neurons exhibiting multisensory depression. Two disparity paradigms were used: a central disparity paradigm and a peripheral disparity paradigm. Recordings were always in the left SC (unless otherwise noted), and thus receptive fields were in the right hemifield. In the central disparity paradigm, the auditory stimulus was always presented to the left of the auditory receptive field and thus disparate to the excitatory visual stimulus. In many cases, the auditory stimulus was so far to the left that it crossed the midline into the opposite hemifield. An example of this is presented in Fig. 1. In this example the visual response was reduced by 82% in the presence of the eccentric auditory stimulus.

fig. 1.

Multisensory depression. A: the auditory (dark gray) and visual (light gray) receptive fields of a superior colliculus (SC) neuron are plotted on a diagram representing visual-auditory space. Each of the concentric circles in the diagram represents 10° of eccentricity and the adjacent half circle represents the unfolded 90–180° of contralateral auditory space. The visual and auditory stimuli were spatially disparate. The visual stimulus was a bar of light (shown as a dark rectangle for clarity) moved through the visual receptive field (arrow), and the auditory stimulus was a broad-band noise burst located central to its receptive field as illustrated by the icon labeled “A.” B: the electronic trace (ramp labeled “V”) represents the movement of the visual stimulus. This stimulus evoked a discharge train on each of 8 trials as shown in the raster display immediately below the electronic trace. Each dot in the raster display represents 1 impulse, and each row represents responses to a single stimulus presentation (see also the averaged responses shown in the peristimulus time histogram just below the raster display). Presentation of the auditory stimulus (square wave labeled “A”) failed to evoke discharges by itself and significantly depressed the neuron's responses to the visual stimulus (compare VA and V). Thus the visual-auditory (VA) response (C) was 82% less than the visual response (the vertical bar through each column in C represents 1 SE). S, superior; I, inferior; N, nasal; T, temporal. **P < 0.01, t-test (VA vs. V, t = 4.10, df = 14).

In the majority (67%, 83/124) of neurons in which the auditory stimulus was placed central (i.e., medial) to its own receptive field and, thus central to the visual stimulus, a similar result was obtained: a significantly (P < 0.05) weaker response to the multisensory stimulus combination than to the visual stimulus alone. Figure 2 summarizes the results of 83 neurons tested in the central disparity paradigm. Figure 2A shows that the multisensory responses were invariably smaller (y axis) than the modality-specific (x axis) visual responses in all neurons. Although no specific attempt was made to document an individual neuron's dynamic properties in terms of the effectiveness of different intensity levels of the visual stimulus, the population analysis indicated a trend in which the magnitude of the multisensory depression effected was generally greatest in neurons in which the visual response was the weakest (see Fig. 2B). This appeared to be the case regardless of the size or location of the neuron's receptive fields or the specific location and distance of the central auditory stimulus from its receptive field border. Whether any of these factors individually or in concert may have contributed to the comparatively low correlation coefficient (r = 0.46) requires further studies.

fig. 2.

Multisensory depression follows the principle of inverse effectiveness. A: plotted here are the multisensory responses as a function of the visual responses of 83 SC neurons tested in the central disparity paradigm. Each data point represents a single neuron, and its coordinates show the mean number of impulses evoked in a response. The broken gray line represents the visual response (line of unity), and the solid black line represents the best fit function of the multisensory responses described by the formula shown. B: the trend of an inverse relationship between the magnitude of multisensory depression and the corresponding visual response becomes apparent when percent multisensory depression is plotted against the visual response (mean number of impulses evoked in a response).

Central versus peripheral disparity paradigms

Response depression was also evident, albeit less frequently (30%, 7/23) and less strongly, when the auditory stimulus was peripheral (i.e., to the right) of its (auditory) receptive field and thus peripheral to the visual stimulus. The paucity of tests using the peripheral disparity paradigm compared with the central disparity paradigm reflects the large size of the auditory receptive fields of many SC cells, especially those whose centers were located outside central auditory space. Many of these auditory receptive fields had peripheral borders that extended well beyond the interaural line, often extending behind the head (see the example in Fig. 1) where peripheral disparity tests could not be conducted in the present conditions.

Despite the general problems inherent in using the peripheral disparity paradigm, in some neurons (i.e., those with more centrally located receptive fields), both spatial disparity configurations could be tested (n = 9), and in four cases, both configurations were found to be effective in inducing multisensory depression. In all but one case, the peripheral disparity paradigm yielded less potent multisensory depression (mean: –37% ± 12%) than did the central disparity paradigm (mean: –55% ± 20%). This is evident in the example illustrated in Fig. 3. In this case, the central disparity configuration reduced the responses to the visual stimulus by 66%, and the peripheral disparity configuration reduced the response to the visual stimulus by 35%. When the data were averaged across all neurons tested in the central and/or peripheral configurations, the magnitude as well as the probability of multisensory depression (see preceding text) proved to be substantially greater in the central than in the peripheral spatial disparity configuration (mean magnitude: –53 ± 20%, n = 83 vs. –24 ± 18%, n = 7, t = 3.71, df = 88, P < 0.05).

fig. 3.

Eliminating cortical influences reduces multisensory depression in both the central and peripheral disparity paradigms. A: in testing this representative SC neuron, the visual stimulus was paired with an auditory stimulus located either central (A1, central disparity paradigm) or peripheral (A2, peripheral disparity paradigm) to it and to the auditory receptive field. B: under control conditions, the neuron's visual response was degraded by 66% in the central disparity paradigm and by 35% in the peripheral disparity paradigm. C: deactivation of anterior ectosylvian sulcus (AES) and the rostral lateral suprasylvian sulcus (rLS; indicated by black on the schematic of cortex) significantly reduced the multisensory depression in the central disparity paradigm [VA1, ANOVA: F(1,14) = 91.24, P < 0.01] and eliminated it in the peripheral disparity paradigm [VA2, ANOVA: F(1,14) = 4.99, P < 0.05]. D: multisensory depression was reinstated after cortical reactivation [as compared with predeactivation control, VA1, ANOVA: F(1,14) = 0.4, P = 0.54; VA2, ANOVA: F(1,14) = 0.02, P = 0.89]. **P < .01, *P < .05. All other conventions are the same as in Fig. 1.

Effect of cortical deactivation on multisensory depression

Sixty-six of the neurons were held for sufficient time to investigate the effects of reversibly deactivating AES and/or rLS and thus temporarily eliminating any possible influence of neural activity in these cortices on the auditory inhibition of SC visual responses. The results from one of these neurons are illustrated in Fig. 3. Both the modality-specific and multisensory responses of this SC neuron were examined during cortical deactivation, and the disruptive effects of AES and rLS deactivation on the neuron's sensory responses were typical of the population of neurons studied. This population included 46 neurons studied during ipsilateral cortical deactivation and an additional 20 neurons examined during contralateral cortical deactivation (neurons studied with contralateral cortical deactivation are dealt with separately in the following text).

Although the deactivation procedure had no significant effect on this neuron's response to the visual stimulus when presented alone, it significantly disrupted the neuron's integrated multisensory response. In both the central and the peripheral disparity configurations, deactivation of ipsilateral AES and rLS reduced the amount of depression, thereby elevating the neuron's multisensory response (compare Fig. 3, VA1 and VA2 in C and B) to a level that was closer to its responses to the visual stimulus alone (compare VA1 and VA2 with V in Fig. 3C). The disruptive effect of cortical deactivation on multisensory depression was not equal for the two spatial disparity configurations. In the central disparity configuration (VA1), the multisensory response was elevated by cortical deactivation but remained significantly (i.e., 22%) below that of the referent modality-specific visual response. The multisensory depression induced by the peripheral disparity configuration (VA2), however, was nearly eliminated, thereby elevating the multisensory response to within 5% of the response to visual stimulus alone. Now the multisensory and modality-specific visual responses were no longer significantly different. Thus whereas cortical deactivation reduced multisensory integration in the central disparity condition, it eliminated it in the peripheral disparity condition. Reactivation of cortex reinstated multisensory depression and, hence, multisensory integration, to control levels in both configurations (Fig. 3D).

Few of the neurons exhibiting multisensory depression proved to be completely refractory to deactivation of cortex. This was most obvious in a sample of 20 neurons in which both AES and rLS were simultaneously deactivated. Although nine of these neurons were able to maintain significant (P < 0.05) multisensory depression, the magnitude of this depression was minimized significantly (P < 0.05) in five of these nine cases and the trend for minimization of depression was apparent in the other four. Furthermore, just as shown for the neuron illustrated in Fig. 3, most frequently the effects of cortical deactivation were not generalized to modality-specific responses but were specific to multisensory integration. Thus most (72%, 33/46) neurons showed no significant change in their modality-specific visual responses during this procedure. However, in 13 (28%) of them, cortical deactivation either increased (n = 7) or decreased (n = 6) their responses to the modality-specific visual stimulus. Ten of these 13 neurons exhibited multisensory depression in control conditions, and in these cases, the modality-specific response changes induced by cortical deactivation were not predictive of how this procedure would affect multisensory depression. Whereas cortical deactivation invariably reduced the multisensory depression in 8 of the 10 neurons, in 4, the visual responses were increased and in the other 4, they were decreased. In the remaining two neurons, multisensory depression was unchanged during cortical deactivation, yet the modality-specific visual responses were increased in one and decreased in the other. An additional three neurons that failed to show multisensory depression in control conditions were found to have their modality-specific visual responses increased (n = 2) or decreased (n = 1) by cortical deactivation.

The effect of cortical deactivation on the population of SC neurons studied is summarized in Fig. 4. Deactivation of AES and/or rLS systematically elevated the multisensory response at all levels of modality-specific visual stimulus effectiveness. Thus there was an increase in the slope of the linear regression between the multisensory and the visual responses as shown in Fig. 4, A–C (compare the solid and dashed black lines). However, further analysis showed no systematic relationship between the magnitude of the cortical deactivation effect on multisensory depression and the control level of multisensory depression (see Fig. 4, D–F). This result was unexpected given the observation that cortical deactivation was much more potent in degrading the comparatively weak multisensory depression induced in the peripheral disparity paradigm than the stronger depression induced by the central disparity paradigm (e.g., see Fig. 3). As shown in Fig. 5A, cortical deactivation reduced the number of neurons showing multisensory depression in the central disparity paradigm by ∼55% but eliminated this integrated multisensory response in every case in which it was generated in the peripheral disparity paradigm. Thus it is not clear whether the paradigmatic differences that were observed in the influences of cortex on multisensory depression were due to differences in their sample sizes or something more mechanistic. It was also interesting to note that neither deactivation of AES and rLS in combination nor deactivation of these areas individually was more effective in disrupting auditory-induced depression of visual responses. These observations suggest that the functional contributions of AES and rLS to SC multisensory depression were nearly equivalent.

fig. 4.

Cortical contributions to SC multisensory depression are similar regardless of the magnitude of the depression induced. A–C: the amplitude of the multisensory response plotted against the visual response (data from central disparity paradigm). Filled circles, the control; open circles, cortical deactivation values (mean number of imp/response). The data show a linear relationship between the multisensory and the modality-specific visual responses in the control condition (dashed lines) with a slope of <0.5 in all cases (i.e., predicts that a neuron's multisensory response will be <50% of its modality-specific visual response). Deactivation of cortex significantly [ANOVArm results: A, F(1,18) = 19.26, P < 0.01; B, F(1,27) = 18.51, P < 0.01; C, F(1,27) = 61.5, P < 0.01] increased the slope to >0.7, so that multisensory responses (solid black line) were elevated toward the neurons' visual responses (thin gray line of unity). Plotted in D–F are the degree of reduction of multisensory depression effected during cortical deactivation. They show that the degree of reduction of multisensory depression was not related to the neurons' predeactivation magnitude of multisensory depression (the apparent trend in D is not significant).

fig. 5.

Cortical deactivation affects the probability and relative magnitude of SC multisensory depression. A: ∼45% of the neurons tested in the central disparity paradigm, and none of the neurons tested in the peripheral disparity paradigm, continued to express statistically significant (P < 0.5) multisensory depression during deactivation of AES-rLS, AES, or rLS. Note that the disruption of this capacity was nearly equal across the population of SC neurons regardless of whether AES and rLS were deactivated simultaneously or individually. B, left: neurons retaining some multisensory depression during cortical deactivation exhibited a significantly (P < 0.5) reduced magnitude of that depression (values are normalized by their control multisensory depression); right: the data are averaged across all SC neurons tested in the central disparity paradigm. Numbers above the bars represent the number of neurons tested in a given condition.

Despite the essential nature of the higher-order inputs from AES and rLS for multisensory depression in all peripheral disparity tests and in the majority of the central disparity tests, ≥45% of the neurons in the central disparity paradigm retained some multisensory depression in the absence of influences from AES and/or rLS (Fig. 5A). Yet even in these neurons, the loss of cortical influences reduced the overall magnitude of multisensory depression by approximately half of that seen in the control tests (see Fig. 5B, left). Combining the two data sets in the central disparity paradigm (those that lost their multisensory depression and those that still retained their multisensory depression) revealed that cortical deactivation reduced the multisensory depression to an overall population mean of only ∼30% of the control level (see Fig. 5B, right).

Deactivation of contralateral cortex

To test the possibility that the opposite cortex can contribute to SC multisensory depression and might, thereby, have accounted for the multisensory depression that remained in the central disparity paradigm during ipsilateral cortical deactivation, 20 neurons were recorded in the SC and studied before, during, and after deactivation of the contralateral AES and rLS. In only two cases was multisensory depression compromised. In those cases, the response depression was degraded by 42 and 58% respectively, still leaving the now-elevated multisensory responses significantly below the referent modality-specific visual responses (compare C and B in Fig. 6). Thus while the magnitude of these changes may have been sufficient to account for the residual multisensory integration seen in some neurons described in the preceding text, the incidence of contralateral cortical contributions to multisensory depression (2/20, 10%) could not fully account for the residual multisensory depression seen after ipsilateral cortical deactivation (see Fig. 5A, left, 9/20, 45%, under AES-rLS deactivation). Thus it appears likely that there are several afferent sources that mediate multisensory depression in SC neurons.

fig. 6.

Contralateral cortex can influence SC multisensory depression. A: shown are the visual and auditory receptive fields of this neuron and the locations of test stimuli. Note that the recording was made in the right SC in this case, and the receptive fields were in left sensory space (deactivating coils were always in the left cortex). B: the control level of multisensory depression was 58.4%. C: deactivation of contralateral AES and rLS significantly reduced the multisensory depression [ANOVA: F(1,14) = 5.35, P < 0.05], but the multisensory response was still significantly weaker than the response to visual stimulus alone (t = 2.21, df = 14, P < 0.05). D: reactivation of cortex reinstated multisensory depression to the control level [ANOVA: F(1,14) = 0.13, P = 0.73]. E: cortical deactivation compromised multisensory depression in this neuron by 58%.


The finding that auditory-visual multisensory depression is produced in SC neurons when these stimuli are spatially disparate is consistent with previous observations (see Kadunce et al. 1997; Meredith and Stein 1986b, 1996; Wallace et al. 1996). However, the present observations also show that the inhibitory region bordering the auditory excitatory receptive field is extremely broad and that the inhibitory effect of an eccentric auditory stimulus on a visual response is demonstrable with the auditory stimulus either far central or peripheral to its receptive field. In addition, these inhibitory regions were nonhomogeneous. Thus an eccentric auditory stimulus was most effective in this context when it was placed in regions of space contralateral to the neuron's excitatory receptive field, and the magnitude of the depression effected in these tests was substantially higher in the central than in the peripheral spatial disparity paradigm.

Inverse effectiveness

The modulatory influence of the eccentric auditory stimulus also proved to be greatest when visual activation was weakest, a concept referred to as inverse effectiveness (Stein and Meredith 1993). Consequently, the stronger a neuron's visual response, the less affected it was by the presence of an inhibitory auditory stimulus. Presumably then, highly salient visual events would be proportionately less susceptible to degradation by spatially disparate auditory stimuli. It appears that evolution has developed a means of ensuring that the access of visual stimuli to the orientation circuitry of the SC is not easily impeded by the presence of innocuous nonvisual stimuli.

The propensity for weak visual responses to be proportionately more susceptible to modulation by an auditory stimulus appears to be a general multisensory principle. In the case of the other index of multisensory integration, multisensory enhancement, the weaker visual responses are subject to the greater amplification by a spatially coincident auditory stimulus (Jiang et al. 2001; Meredith and Stein 1986a,b; Wallace et al. 1996). This relationship between response vigor and potential for multisensory enhancement was evident across all bimodal combinations of visual, auditory, and somatosensory stimuli, and, presumably, the same generality of effect is true for multisensory depression, but this remains to be determined. These data lend credence to the idea that multisensory integration is of greatest utility when stimuli are weak or ambiguous.

Comparisons of cortical deactivation on multisensory depression and multisensory enhancement

similarities. Perhaps most striking are the similarities in the cortical control over these opposing multisensory processes in SC neurons. Like multisensory enhancement (see Jiang et al. 2001; Wallace and Stein 1994), multisensory depression has proven in this study to be also subject to the higher-order influences of AES and rLS. Consequently, deactivation of AES and rLS disrupts both forms of SC multisensory integration.

The SC plays an important role in orientation to external events (e.g., see Stein and Meredith 1991), and presumably substantial increases or decreases in the vigor of its neuronal responses to sensory stimuli would be linked to corresponding changes in overt orientation behaviors. In fact, these physiological manifestations of multisensory integration have striking behavioral parallels. As noted in the introduction, the same stimulus configurations that enhance the responses of SC neurons also enhance overt orientation behaviors (e.g., see Stein et al. 2004).

A similar physiological-behavioral relationship also exists for multisensory depression. Recently, Jiang et al. (2002) trained animals to orient to a visual target and ignore an auditory stimulus. When these stimuli were presented in a central disparity configuration, the rate of correct orientations to the visual target was significantly reduced. Deactivation of AES or rLS drastically disrupted this behavioral index of multisensory depression such that visual orientation in the presence of a centrally disparate auditory stimulus was significantly improved and performance approached that observed when the visual stimulus was presented alone.

differences. It is interesting to note that at both the behavioral and single neuron levels, cortical deactivation has minimized rather than eliminated multisensory depression. Presumably, even the reduced multisensory depression that remains in a subpopulation (i.e., 45%) of multisensory SC neurons is sufficient to support some level of multisensory depression at the behavioral level. Yet this retention of multisensory depression and its corresponding behavioral product, though minimal, still differs somewhat from the results of similar experiments with multisensory enhancement. In the latter cases, cortical deactivation eliminated the enhanced response at both the single neuron and behavioral levels (Jiang et al. 2001, 2002).

There are several possible sources of these differences in the susceptibility of multisensory enhancement and multisensory depression to cortical deactivation. Despite the fact that all of these studies used the same deactivation technique, small differences in the location of the cryogenic coils may have yielded different patterns of cortical deactivation in electrophysiological experiments. For example, in the multisensory enhancement electrophysiological experiments all relevant areas of AES and rLS may have been deactivated (see Jiang et al. 2001), whereas only portions of these areas may have been compromised in the present experiments. Nevertheless, this possibility is not likely. The construction and positioning of the deactivation coils were roughly equivalent to those used in studies of multisensory enhancement, the present results did not vary substantially among the animals tested as would be expected if small positional inaccuracies could have significant effects, and the retention of a minimal degree of multisensory depression during cortical deactivation was specific to only one of the spatial disparity paradigms. Last, these electrophysiological differences in the effectiveness of cortical deactivation on multisensory enhancement and depression parallel similar differences in behavioral experiments that were done with the same set of coils and in the same animals (Jiang et al. 2002).

It is more probable that there are subtle differences in the circuitry and thus the dependencies of these two forms of SC multisensory integration. The heterogeneity in the circuitry underlying multisensory depression appears to be greater than that underlying multisensory enhancement, with some SC neurons being totally dependent on AES and rLS influences and others for which alternative mechanisms have been crafted to help synthesize their multisensory product. These alternative mechanisms may involve influences from cortical areas other than AES and rLS and/or may be mediated, in part, by the intrinsic circuitry of the SC. Nevertheless, the similarities in the cortical dependencies of these two forms of multisensory integration far outweigh their differences.

Influences over the architecture underlying multisensory integration

In this context, it is interesting to note that despite being subject to higher-order control, both forms of multisensory integration are clearly evident in an anesthetized animal and as a consequence of pairing stimuli that lack any obvious inherent significance. This suggests that the system's underlying architecture has been constructed in a manner that maximizes the enhancement of signals arriving over different sensory channels only when they are derived from the same location in space (multisensory enhancement). This is one of the principles utilized in artificial sensor fusion networks (see Boss et al. 2001), as it is a simple way of maximizing the detection of events while simultaneously minimizing the probability of false positives. It also fits a Bayesian statistical model (see Anastasio et al. 2000). Presumably, this architecture is the result of normal life experience with visual and auditory events. For when such experiences are altered so that animals are reared in conditions in which they experience only spatially disparate visual and auditory stimuli, SC multisensory neurons do not exhibit enhanced responses when these stimuli are spatially coincident (Wallace et al. 2002).

The cortex, unlike the SC, is particularly sensitive to, and directly altered by, early sensory experience (see Buonomano and Merzenich 1998; Rauschecker 1995, 1999; Wickelgren and Sterling 1969). It seems likely that its essential role in this circuit early in life is to impose its experiential adaptations onto SC neurons. Later in life this would ensure that the multisensory products of SC neurons, and the behavioral functions dependent on them, are appropriate for the particular circumstances in which external events are encountered and for the specific needs of the organism at any given moment (see also, Jiang and Stein 2001, 2002; Wallace and Stein 1997). That eliminating cortical influences to the SC in the present context disrupted this processes of multisensory integration and compromised multisensory depression at the level of the single SC neuron may help explain the seemingly paradoxical behavioral finding that AES or rLS deactivation enhances visual orientation in the presence of a normally disruptive auditory stimulus (Jiang et al. 2002).


These experiments were supported by National Institute of Neurological Disorders and Stroke Grants NS-22543 and NS-36916 and by the Welch-Kempton Endowment to W. Jiang.


We thank Dr. Huai. Jiang for valuable help in cryogenic implantation surgery, Dr. Terrence Stanford for valuable discussions, and N. London for technical assistance.


  • The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.


View Abstract