HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) All Versions of this Article: 97/1/921    most recent 00497.2006v1 Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (8) Citing Articles via Google Scholar Google Scholar Articles by Wallace, M. T. Articles by Stein, B. E. Search for Related Content PubMed PubMed Citation Articles by Wallace, M. T. Articles by Stein, B. E.

REPORT

Early Experience Determines How the Senses Will Interact

Department of Neurobiology and Anatomy, Wake Forest University School of Medicine, Winston-Salem, North Carolina

Submitted 10 May 2006; accepted in final form 10 August 2006

	ABSTRACT

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
Multisensory integration refers to the process by which the brain synthesizes information from different senses to enhance sensitivity to external events. In the present experiments, animals were reared in an altered sensory environment in which visual and auditory stimuli were temporally coupled but originated from different locations. Neurons in the superior colliculus developed a seemingly anomalous form of multisensory integration in which spatially disparate visual-auditory stimuli were integrated in the same way that neurons in normally reared animals integrated visual-auditory stimuli from the same location. The data suggest that the principles governing multisensory integration are highly plastic and that there is no a priori spatial relationship between stimuli from different senses that is required for their integration. Rather, these principles appear to be established early in life based on the specific features of an animal's environment to best adapt it to deal with that environment later in life.

	INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
The different senses have evolved to work synergistically to enhance the salience of environmental events. This process of multisensory integration is ubiquitous. It has been demonstrated in every species tested, and it is clearly adaptive under widely differing environmental circumstances (Calvert et al. 2004; Stein and Meredith 1993). However, despite its utility it does not appear to be an innate ability because the earliest-appearing multisensory neurons in the superior colliculus (SC) are incapable of synthesizing cross-modal stimuli (Stein et al. 1973; Wallace and Stein 1997, 2001). Rather, they require a protracted period of postnatal maturation to develop this capability, suggesting that sensory experience could be an important determinant. Supporting this is the recent finding that SC neurons in dark-reared animals are unable to integrate visual and nonvisual cues (Wallace et al. 2004).

Presumably, multisensory maturation is protracted because considerable time is needed to gather the experience with combinations of cross-modal cues necessary to link them to common environmental events and to each other. Such coupling cannot be specified in the genome. What appears critical for multisensory interactions is the close temporal and/or spatial correspondence of the different sensory signals derived from common events (Stein and Meredith 1993). Hence, it appears likely that one or both factors are important in dictating which multisensory signals will be integrated and which will not. In this way, sensory experiences can craft a brain to deal with the specific cross-modal contingencies in that animal's world. Although a seemingly reasonable line of logic, there is little empirical evidence to support this view. Strong evidence could be provided if sensory experience was systematically altered during development to disrupt "normal" spatial and/or temporal cross-modal relationships and resulted in corresponding changes in the constraints necessary to elicit multisensory integration. Testing this possibility was the objective of the current study, in which cats were raised with visual and auditory stimuli that were paired in a temporally coincident but spatially disparate manner.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
Cats (n = 4) were raised in a modified sensory environment from birth until adulthood (>6 mo). For 12 h each day, the animals were deprived of all visual experiences in a light-tight environment. For the remaining 12 h, a series of controlled visual and auditory stimuli were presented from light-emitting diodes (LEDs) and speakers positioned 1 m from the front of the animal's enclosure. Stimuli consisted of the simultaneous illumination of a single LED and a noise burst from a single speaker. LEDs and speakers were located on separate 4 x 4 grids (16 total locations; individual spacing 0.25 m), and the grids were displaced by 0.75 m relative to one another. Active stimulus pairs were always at the same location on the grid (e.g., 2 over, 3 down), resulting in fixed stimulus separations of 0.75 m (resulting in angular disparities of 18–30° depending on the animal's position). These visual-auditory stimulus pairs were 75 ms in duration and were delivered at random interstimulus intervals (uniform distribution) ranging from 1 to 20 s.

Daily care, observations and routine veterinary procedures were conducted using infrared (IR) goggles and an IR viewing system. Confounding visual experience was precluded when animals were removed from the housing environment for surgery or recording sessions by anesthetizing them in the dark and fitting them with opaque occluding contact lenses and masks during transport and recovery. All data from such spatial-disparity-reared animals were compared with those generated from age-matched animals (n = 3) reared in standard illumination.

All procedures complied with the Guide for the Care and Use of Laboratory Animals (National Institutes of Health Publication 91–3207) at the Wake Forest University School of Medicine, which is accredited by the American Association for Accreditation of Laboratory Animal Care. Details of surgery, recording procedures, and data analyses are similar to those used previously (Meredith and Stein 1983; Wallace and Stein 1997) and are only briefly described.

As adults, each animal was prepared for surgery. Animals were rendered tractable with ketamine (5–25 mg/kg im) and acepromazine maleate (0.2–0.4 mg/kg im) and intubated. Surgical anesthesia was maintained with isofluorane (1.0–4.0%). Throughout the procedure, vital signs (e.g., body temperature, expiratory CO₂, etc.) were monitored and maintained at values indicative of deep anesthesia. A craniotomy provided access to the SC and a head-holder was attached to the skull using stainless steel screws and orthopedic cement. Postoperative care (i.e., analgesia, antibiotics) was provided in consultation with veterinary staff.

After 7–12 days of recovery, animals were prepared for recording. They were anesthetized [ketamine (induction: 25 mg/kg im; maintenance: 5–10 mg · kg^–1 · h^–1 iv)] and paralyzed [pancuronium bromide (2 mg · kg^–1 · h^–1 iv)] to prevent ocular drift, and infused with fluids [lactated Ringer (8–20 ml/h iv)]. Microelectrodes were advanced into the SC, and single-unit neural activity was recorded, amplified, and routed to an oscilloscope, audio monitor, and computer. Experiments lasted from 6–10 h and were conducted at weekly intervals.

Each SC neuron was classified according to the modality(ies) to which it responded. Receptive fields were mapped using conventional methods and transferred to standardized representations of visual, auditory, and somatosensory space. Receptive field size comparisons were matched by SC location. Receptive field overlap was calculated for visual-auditory neurons and operationally defined as the proportion of the smaller receptive field encompassed by the larger receptive field.

A multisensory neuron was defined as one responding to stimuli from more than one sensory modality or the responses of which to a stimulus in one modality were significantly altered by a stimulus from another modality. Once a neuron's modality selectivity (e.g., visual-auditory) was established, its responses to each modality-specific and cross-modal stimulus combination were quantitatively evaluated. Stimuli were presented 8–20 times at 8- to 12-s intervals in an interleaved fashion. A multisensory interaction was operationally defined as a significant (2-tailed t-test, P < 0.05) change in the response to the combined stimuli when compared with the response to the most effective single-modality stimulus. The magnitude of this interaction was calculated as: (CM – SM_max)/(SM_max) x 100 = % interaction, where CM is the mean response evoked by the combined-modality stimulus and SM_max is the mean response evoked by the most effective single-modality stimulus.

The depth of each recorded neuron was noted in each penetration and electrolytic lesions (5- to 10-µA DC current; 10 s) were made at several locations. Following the final experiment, the animal was killed (sodium pentobarbital, 100 mg/kg iv) and perfused transcardially with saline followed by formalin. The midbrain was blocked stereotaxically, removed, and placed in sucrose overnight. Frozen coronal sections (50 µm) were taken and counterstained (neutral red) to facilitate tissue reconstruction.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
Rearing animals in an environment in which brief spatially disparate visual and auditory stimuli were simultaneously presented had little disruptive effect on the distribution of the different sensory-responsive neuronal types in the SC. A significant sample of both unisensory and multisensory neurons was obtained, and the incidence of the various modality convergence patterns was not substantially different from controls (Table 1). Furthermore, although the visual responses of neurons in disparity-reared animals were neither as vigorous nor as well tuned as in controls, in many other respects (e.g., latency, response duration, etc.), they appeared to be similar. They were readily activated by stationary and moving bars of light and had similar variability of response and adaptation characteristics. Outside of an enlargement of receptive fields (see following text), the auditory and somatosensory responses of these neurons were indistinguishable from controls.

However, in 39% of the visual-auditory population in which both receptive fields were mapped (a total of 41 neurons), these receptive fields did contract to within 50% of normal adult size. Many of these receptive fields were oddly shaped and exhibited an unusual azimuthal elongation. In contrast to control animals, in which visual receptive fields were only slightly elliptical [mean azimuth:elevation (i.e., aspect) ratio of 1.3:1], these receptive fields had aspect ratios as large as 4:1 and a mean (2.2:1) that was significantly larger (P < 0.01). Although less evident, auditory receptive fields showed a similar azimuthal bias (controls = 1.2:1; experimental animals = 1.6:1, P <. 05). No such bias was evident in somatosensory receptive fields.

The most striking rearing effect was seen in multisensory neurons the receptive fields of which had contracted and become misaligned (Fig. 1, A and B). Receptive field misregister, almost never seen in normally reared cats (Meredith and Stein 1983; Stein and Meredith 1993; Stein et al. 1973; Wallace and Stein 1997), was specific to visual-auditory neurons in spatial-disparity-reared animals. The misregister matched the animal's early sensory experience with auditory receptive fields consistently displaced to the right of the visual receptive field. In the 41 visual-auditory neurons in which both receptive fields could be adequately mapped, the average degree of receptive field overlap was degraded from the control value of 81% to 38% in the experimental group (Fig. 1C; P < 0.01), with a near-continuous distribution of varying degrees of overlap in the experimental population (Fig. 1D). Whereas in controls auditory receptive fields centers were on average 16.0° peripheral to visual receptive field centers, in the spatial-disparity-reared group, the difference was 46.5° (Fig. 1, E and F; P < 0.01). This 30° difference is at the upper end of the range of disparities possible in the rearing environment. Again, these effects were restricted to visual-auditory neurons with visual-somatosensory (81 vs. 79% in controls) and auditory-somatosensory (80 vs. 74% in controls) neurons showing normal overlap.

View larger version (29K): [in this window] [in a new window]   FIG. 1. Visual-auditory receptive field overlap is altered in spatial-disparity-reared animals. A and B: 2 examples of receptive fields of visual-auditory neurons in disparity-reared animals. Note the lack of overlap of the visual (blue shading) and auditory (green shading) receptive fields. Receptive fields are plotted on a representation of visual and auditory space, with each 10° shown as a concentric circle and the horizontal and vertical lines indicating 0° azimuth and 0° elevation, respectively. The half circle to the right represents right caudal space (i.e., space behind the interaural axis). C: average amount of visual-auditory receptive field overlap in control (yellow) and spatial-disparity-reared (brown) animals. Error bars represents the SE. D: population plot showing the distribution of visual-auditory neurons with varying degrees of receptive field overlap for the control and experimental populations. E: grand average visual and auditory receptive fields (shading), along with the location of the center of the receptive fields (V, A), for control (left) and spatial-disparity-reared (right) animals. Each concentric circle represents 10°, and the horizontal line shows the horizontal meridian (interaural plane). Grand averages were calculated by taking the horizontal (i.e., azimuthal) and vertical (i.e., elevation) borders of each receptive field, adding these together and dividing by the total number of receptive fields. F. Plot of the displacement of the center of the auditory receptive field relative to the center of the visual receptive field for individual neurons in control (green dots) and spatial-disparity-reared (red dots) animals. Stars show the mean displacement for the two populations. Note the lateral (i.e., peripheral) shift in this index for the experimental population.

View larger version (30K): [in this window] [in a new window]   FIG. 2. The auditory topography in the superior colliculus (SC) appears shifted by spatial-disparity rearing. Left: schematic representation of a dorsal view of the left SC. Numbers within circles correspond to the location of recording penetrations designed to sample from a large extent of the SC. Right: representation of auditory space (conventions are the same as in Fig. 1), with the numbers representing the average auditory receptive field centroid for the recording locations shown on the left for both normally reared (green) and spatial-disparity-reared (red) animals. A, anterior; P, posterior; M, medial; L, lateral; N, nasal; T, temporal; S, superior; I, inferior.

In contrast, many neurons with receptive fields that did contract developed the ability to engage in multisensory integration. Now, however, the visual and auditory stimuli had to be in different locations to fall within the neuron's (misaligned) receptive fields (Fig. 3). To examine the possibility that receptive field contraction serves as a marker for multisensory integrative maturity, the probability of multisensory integration was calculated as a function of relative receptive field size (corrected for SC location). All 13 neurons having receptive fields >200% of control size failed to exhibit multisensory integration; neurons with receptive fields 150–200% of control values had a low probability (P = 0.15, 2/13) and those with receptive fields <150% of controls had a high probability (P = 0.71, 10/14).

View larger version (45K): [in this window] [in a new window]   FIG. 3. The spatial constraints of multisensory integration appear dependent on the experiences received during development. Shown are data from 2 neurons, 1 from a control animal (left) and 1 from a spatial-disparity-reared animal (right). Top: receptive fields (visual, blue shading; auditory, green shading) of these neurons, along with the locations of the stimuli used to assess multisensory integration. Conventions are the same as in Fig. 1. Middle: responses of these neurons to visual, auditory, and combined visual-auditory (i.e., multisensory) stimulation. Peristimulus time histograms depict the summed neural responses for a total of 15 trials for each condition with the ramps and square waves at the top showing the timing of the visual and auditory stimulus, respectively. Summary bar graphs at the bottom show the mean responses for each condition, along with the magnitude of the multisensory enhancement (brown bars) and the predicted sum of the visual and auditory responses.

View larger version (32K): [in this window] [in a new window]   FIG. 4. The development of multisensory integration is dictated by the sensory experiences received during postnatal life, as indexed by changes in receptive field architecture. A: plotted are the percentages of the visual-auditory SC population that exhibit multisensory integration as a function of receptive field overlap for control (left; yellow bars) and spatial-disparity-reared (right; brown bars) animals. In control animals, almost all neurons have a high degree of spatial overlap between their visual and auditory receptive fields, and a high percentage of them exhibit multisensory integration. In contrast, in spatial-disparity-reared animals, many neurons have poor or no receptive field overlap, and it is these neurons that exhibit a high incidence of multisensory integration. Numbers in parentheses above the bars show the total number of neurons for that group. B: plot of receptive field overlap as a function of receptive field size for all sampled neurons in the spatial-disparity-reared population. Each neuron is plotted and categorized as to its modality specificity and integrative capacity (see legend). Note the presence of 2 distinct clusters of integrating neurons (highlighted in gray shading)—on the left visual-auditory neurons with small receptive fields and little or no overlap and on the right nonvisual-auditory neurons with small receptive fields and good overlap.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

The present results suggest that the primary factor in multisensory maturation is the temporal synchrony of different sensory inputs. Presumably, experience with temporally coincident cross-modal stimuli leads to a physiological binding of these stimuli that is reflected in the SC through the creation of spatially overlapping receptive fields and the integrative capabilities of its multisensory SC neurons. In normally reared animals, the most frequent temporally linked cross-modal events are produced by the same events, thereby producing a spatial overlap of the receptive fields of activated multisensory neurons and constraining multisensory enhancement to stimuli that are coincident in space and time (Stein and Meredith 1993). Apparently, this is only one possible developmental outcome. Why some neurons appeared to be insensitive to the particular visual-auditory experiences provided here is unknown. It is, however, quite likely that the stimulus set employed here provided a conservative estimate of the plasticity inherent in the developing SC multisensory representation and that a far greater proportion of its multisensory neurons would have been altered by a contextually richer set of stimuli and/or a stimulus set with immediate behavioral relevance.

Nevertheless, it is interesting to consider the implications of the apparent primacy of temporal synchrony in the maturation of multisensory processes in SC neurons. Multisensory integration changes the physiological salience of an event based on the spatial properties of its cross-modal stimuli: as noted in the preceding text, spatial coincidence among the cross-modal stimuli is normally required for enhanced responses—spatially disparate cross-modal stimuli are either not integrated or result in response depression (Kadunce et al. 1997; Meredith and Stein 1996). Yet the maturation of this integrative capability appears to be an indirect consequence of a neural priority that is entirely unrelated to space. Thus the seemingly odd spatial requirements for multisensory integration in disparity-reared animals are best viewed as the equally direct and logical maturational products of their rearing condition. Whether the temporal synchrony of cross-modal cues proves to be a unique factor for coupling other sensory modalities during early maturation, and/or is as powerful a determinant of multisensory integration in other brain areas, remains to be determined.

It is also important to note that there were no compensatory auditory receptive field shifts seen here, such as those seen after rearing animals with deviations of the visual axis (King et al. 1988; Knudsen and Brainard 1991, 1995). These studies have shown that the auditory system uses vision to calibrate its map in the SC and that spatial discrepancy between the maps can be minimized by shifting the auditory receptive fields to better realign with their visual counterparts. As a result, the misalignments among the different receptive fields of multisensory neurons are also minimized. Although not tested in these previous studies, the present data strongly suggest that multisensory integration would develop in these circumstances and would accurately reflect the realignment of receptive fields. There is little reason to doubt that cross-modal temporal coincidence is responsible for this compensatory process as well. Presumably, however, the compensatory shifting of auditory receptive fields also requires feedback from the motor acts involved in dealing with the initiating events. These instructive signals were unavailable to the brain in the present experiments, thereby revealing an underlying cross-modal process that was not "corrected" by the initiation of other adaptive processes. Whether interactive experience with aligned cross-modal cues during adulthood could bring the receptive fields into register and adapt the multisensory integrative capabilities of these multisensory SC neurons remains to be determined.

	GRANTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
This work was supported by National Institutes of Health Grants MH-63861 and NS-36916.

	ACKNOWLEDGMENTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
We thank N. London for editorial assistance.

	FOOTNOTES

 
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Present address and address for reprint requests and other correspondence: M. T. Wallace, Vanderbilt Kennedy Center for Research on Human Development, Dept. of Hearing and Speech Sciences, Vanderbilt University, 465 21st Ave. S., Nashville, TN 37232 (E-mail: mark.wallace{at}vanderbilt.edu)

	REFERENCES

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
Buonomano DV and Merzenich MM. Cortical plasticity: from synapses to maps. Annu Rev Neurosci 21: 149–186, 1998.[CrossRef][ISI][Medline]

Calvert GA, Spence C, and Stein BE. The Handbook of Multisensory Processes. Cambridge, MA: MIT Press, 2004.

Feldman DE. A new critical period for sensory map plasticity. Neuron 31: 171–173, 2001.[CrossRef][ISI][Medline]

Kadunce DC, Vaughan JW, Wallace MT, Benedek G, and Stein BE. Mechanisms of within- and cross-modality suppression in the superior colliculus. J Neurophysiol 78: 2834–2847, 1997.[Abstract/Free Full Text]

King AJ, Hutchings ME, Moore DR, and Blakemore C. Developmental plasticity in the visual and auditory representations in the mammalian superior colliculus. Nature 332: 73–76, 1988.[CrossRef][Medline]

King AJ, Parsons CH, and Moore DR. Plasticity in the neural coding of auditory space in the mammalian brain. Proc Natl Acad Sci USA 97: 11821–11828, 2000.[Abstract/Free Full Text]

Knudsen EI and Brainard MS. Visual instruction of the neural map of auditory space in the developing optic tectum. Science 253: 85–87, 1991.[Abstract/Free Full Text]

Knudsen EI and Brainard MS. Creating a unified representation of visual and auditory space in the brain. Annu Rev Neurosci 18: 19–43, 1995.[CrossRef][ISI][Medline]

Li Y, Fitzpatrick D, and White LE. The development of direction selectivity in ferret visual cortex requires early visual experience. Nat Neurosci 9: 676–681, 2006.[CrossRef][ISI][Medline]

Meredith MA and Stein BE. Interactions among converging sensory inputs in the superior colliculus. Science 221: 389–391, 1983.[Abstract/Free Full Text]

Meredith MA and Stein BE. Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75: 1843–1857, 1996.[Abstract/Free Full Text]

Rauschecker JP. Cortical map plasticity in animals and humans. Prog Brain Res 138: 73–88, 2002.[ISI][Medline]

Simons DJ and Land PW. Early experience of tactile stimulation influences organization of somatic sensory cortex. Nature 326: 694–697, 1987.[CrossRef][Medline]

Singer W. Development and plasticity of cortical processing architectures. Science 270: 758–764, 1995.[Abstract/Free Full Text]

Stein BE, Labos E, and Kruger L. Sequence of changes in properties of neurons of superior colliculus of the kitten during maturation. J Neurophysiol 36: 667–679, 1973.[Free Full Text]

Stein BE and Meredith MA. The Merging of the Senses. Cambridge, MA: MIT Press, 1993.

Wallace MT, Perrault TJ, Jr, Hairston WD, and Stein BE. Visual experience is necessary for the development of multisensory integration. J Neurosci 24: 9580–9584, 2004.[Abstract/Free Full Text]

Wallace MT and Stein BE. Development of multisensory neurons and multisensory integration in cat superior colliculus. J Neurosci 17: 2429–2444, 1997.[Abstract/Free Full Text]

Wallace MT and Stein BE. Sensory and multisensory responses in the newborn monkey superior colliculus. J Neurosci 21: 8886–8894, 2001.[Abstract/Free Full Text]

Wiesel TN and Hubel DH. Comparison of the effects of unilateral and bilateral eye closure on cortical unit responses in kittens. J Neurophysiol 28: 1029–1040, 1965.[Free Full Text]

This article has been cited by other articles:

				B. N. Carriere, D. W. Royal, T. J. Perrault, S. P. Morrison, J. W. Vaughan, B. E. Stein, and M. T. Wallace Visual Deprivation Alters the Development of Cortical Multisensory Integration J Neurophysiol, November 1, 2007; 98(5): 2858 - 2867. [Abstract] [Full Text] [PDF]

				B. Roder, A. Kusmierek, C. Spence, and T. Schicke Developmental vision determines the reference frame for the multisensory control of action PNAS, March 13, 2007; 104(11): 4753 - 4758. [Abstract] [Full Text] [PDF]

				A. J. King Coordinating Different Sensory Inputs During Development. Focus on "Early Experience Determines How the Senses Will Interact" J Neurophysiol, January 1, 2007; 97(1): 3 - 4. [Full Text] [PDF]

This Article Abstract Full Text (PDF) All Versions of this Article: 97/1/921    most recent 00497.2006v1 Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (8) Citing Articles via Google Scholar Google Scholar Articles by Wallace, M. T. Articles by Stein, B. E. Search for Related Content PubMed PubMed Citation Articles by Wallace, M. T. Articles by Stein, B. E.