Ventral Intraparietal Area of the Macaque: Congruent Visual and Somatic Response Properties

Jean-René Duhamel, Carol L. Colby, Michael E. Goldberg


Duhamel, Jean-René, Carol L. Colby, and Michael E. Goldberg. Ventral intraparietal area of the macaque: congruent visual and somatic response properties. J. Neurophysiol. 79: 126–136, 1998. In a previous report, we described the visual response properties in the ventral intraparietal area (area VIP) of the awake macaque. Here we describe the somatosensory response properties in area VIP and the patterns of correspondence between the responses of single neurons to independently administered tactile and visual stimulation. VIP neurons responded to visual stimulation only or to visual and tactile stimulation. Of 218 neurons tested, 153 (70%) were bimodal in the sense that they responded to stimuli that were independently applied in either sensory modality. Unimodal visual and bimodal neurons were intermingled within the recording area and could not be distinguished on the basis of their visual response properties alone. Most of the cells with a tactile receptive field (RF) responded well to light touch or air puffs. The distribution of RF locations principally emphasized the head (85%), with approximately equivalent representations of the upper and lower face areas. The tactile and visual RFs were aligned in a congruent manner, with the intersection of the visual vertical and horizontal meridian having its tactile counterpart in the nose/mouth area. Small foveal visual RFs were paired with small tactile RFs on the muzzle, and peripheral visual RFs were associated with tactile RFs on the side of the head or body. Most cells showed a strong sensitivity to moving stimuli, and the preferred directions of visual and tactile motion coincided in 85% of bimodal cells. In some cases, bimodal responses patterns were complementary: cells responding to motion in depth toward the monkey had on responses, whereas cells responding to motion in depth away form the monkey had off responses. Other forms of bimodal response congruence included orientation selectivity, and on, off, and on/off response types. The large proportion of bimodal tactile and visual neurons with congruent response properties in area VIP indicates that there are important functional differences between area VIP and other dorsal stream areas involved in the analysis of motion. We suggest that VIP is involved in the construction of a multisensory, head-centered representation of near extrapersonal space.


Organisms moving about their environment generate a continuous, changing flow of stimulation on their sensory epithelia. Neural maps receiving converging input from different sensory modalities, such as vision and somatic sensation, are believed to contribute to the analysis of the spatial relations between the body and the immediate surrounding and to generate an integrated representation of external events (Stein et al. 1976). Such maps have been described in many brain areas, including the superior colliculus (Drager and Hubel 1975, 1976; Meredith and Stein 1983), putamen (Graziano and Gross 1994), parietal (Hyärinen and Poranen 1974; Leinonen and Nyman 1979), and premotor cortex (Rizzolatti et al. 1981).

The posterior parietal cortex contains several distinct areas that are part of the dorsal stream visual pathway (Ungerleider and Mishkin 1982). One of these is the ventral intraparietal area (VIP), located in the deepest portions of the intraparietal sulcus, which receives a major source of visual input from the middle temporal area (MT) in the superior temporal sulcus (Maunsell and Van Essen 1983; Ungerleider and Desimone 1986). Consistent with this input, we have shown that neurons in area VIP display response properties that are similar to those observed in MT, such as a sensitivity to the speed and direction of motion of visual stimuli (Colby at al. 1993). Most neurons are tuned to the direction of a spot of light moving along a path in the frontoparallel plane, whereas other neurons respond best to motion in depth toward or away from the animal's head or show activity that is gated by the object's distance.

Anatomic data indicates that the intraparietal sulcus of the macaque monkey receives input from somatic cortical areas (Jones and Powell 1970; Pandya and Kuypers 1969; Pandya and Selzer 1982; Selzer and Pandya 1980), suggesting the existence of regions of intersection between the visual and somatosensory processing hierarchies (Van Essen et al. 1990). Specifically, the fundus of the intraparietal sulcus, which contains area VIP, receives somatic input through different pathways originating from the primary somatosensory cortex (Seltzer and Pandya 1986). In the present study, we examined somatosensory and visual response properties of neurons in the physiologically defined area VIP. We found that a majority of neurons had tactile and visual receptive fields and response properties that were matched in an orderly manner. The body representation in VIP strongly emphasized the face region, and neurons often responded to the same stimulus features, such as motion direction, in the two sensory modalities. Preliminary reports of some of these finding have been published (Colby and Duhamel 1991; Duhamel et al. 1989, 1991).


Surgical and physiological methods

Three rhesus monkeys (Macaca mulatta) were prepared for chronic neurophysiological experiments by surgical implantation of ocular search coils, head-holding device, and recording chambers under general anesthesia following previously described techniques (Goldberg 1983). Recording chambers were centered over the intraparietal sulcus and tangentially to the skull. All experimental protocols were approved by the National Eye Institute Animal Care and Use Committee and certified to be in compliance with the guidelines set in the Public Health Service Guide for the Care and Use of Laboratory Animals.

Recordings were made with flexible tungsten microelectrode inserted in stainless steel guide tubes, which were themselves maintained in place by a nylon grid fitted in the recording chamber (Crist et al. 1988). The electrodes were introduced into the brain by means of a hydraulic microdrive. Responses properties were systematically assessed on both banks of the intraparietal sulcus and in the fundus at adjacent sites spaced 1-mm apart. After completion of the experiments, monkeys were anesthetized with an overdose of pentobarbital sodium and perfused. The method employed for the histological reconstruction of the recording sites and the conclusions regarding the anatomic location of area VIP have been reported in a previous paper (Colby et al. 1993).

Behavioral methods

Behavioral monitoring, unit sampling and on-line data analysis were controlled by a PDP-11/73 computer (Goldberg 1983). The data reported here were collected in separate blocks of visual and somatosensory trials. For visual testing, the monkey gazed for 2,000–3,000 ms at a fixation spot rear-projected straight ahead on a translucent tangent screen located 57 cm away. The monkey was rewarded with a drop of water for maintaining his eye within a 2° × 2° electronically defined window. Visual responses usually were evoked with computer-controlled spots of light generated by a light-emitting diode and oriented bars generated by an adjustable slit-maker that could be positioned and moved anywhere on the screen by a pair of servo-controlled mirror galvanometers. Most neurons were tested with the tangent screen at the fixed 57 cm distance. Depth sensitivity also was tested by presenting stimuli on a smaller projection screen that could be placed at variable distances from the eyes. Different hand-held stimuli (random-dots, ophthalmoscope, small objects) also were used as needed to test for whole-field motion, distance, and motion-in-depth selectivity.

For somatosensory testing, the monkey also was required to fixate a central target. The tactile stimulus was moved into position either from behind the monkey's head or from the side, beyond the monkey's field of view. When the receptive field (RF) was within the field of view, the eyes were patched and the monkey was rewarded periodically simply for remaining still. Tactile responses were evoked with mechanical stimuli including air-puffs, light pressure applied with the tip of a stationary or moving cotton applicator, and individual joint rotations. The monkeys habituated easily to this and remained quiet throughout the procedure. We did not have fine control over the somatosensory stimuli. The purpose of this study was to establish that a region previously described as a visual area had a high number of visual neurons that were also sensitive to somatosensory stimuli. Our main intent was to describe the location and extent of tactile receptive fields and estimate cutaneous and joint directional selectivity. We used the opening and closure of a manual switch to signal the beginning and end of somatosensory stimuli. This enabled us to estimate the amplitudes of somatosensory responses but not their latencies.


Location and distribution of tactile receptive fields

Of 218 neurons independently tested for the presence of somatosensory and visual activity, 153 (70%) responded to both sensory modalities. The remaining 65 cells were activated by visual stimulation only. None responded to tactile stimulation only. The location and contours of the tactile and visual RFs were mapped for 106 of 153 cells. The other cells either had bimodal RFs with unclear boundaries or were recorded only long enough to demonstrate the existence of visual and tactile responses and identify the effective stimuli in each sensory modality. Figure 1 shows several examples of cells with somatosensory and visual RFs. A large majority of cells with somatosensory RF responded well to passive superficial stimulation of the hair and glabrous skin surface (95%) using light touch or air puff. Few cells could be driven only by joint rotation of the hand and upper limb. As all experiments were performed with the monkey's head fixed, responses to neck rotation were not tested. Tactile RFs varied in size from quite small (1–8 cm2; e.g., Fig. 1, top row) to large (whole body side). The large majority of tactile RFs were located on the head (85%), with the upper and lower face areas being represented in approximately equivalent manner. Most RFs were located on the contralateral side of the body (72%), but were also found along the midsagittal line (18%) and on the ipsilateral side (10%). A systematic relationship was found between the location and the size of the tactile RF. Small RFs were always located near the lips and nose, and large RFs were located on the top, side, or back of the head, on the neck. This is illustrated in Fig. 2, which shows that the size of the RFs increased with radial distance of their center from the muzzle. Therefore the underlying body representation in VIP appears to be organized as to include a higher resolution region around the muzzle and a coarser peripheral region reaching the back of the head and the upper body. This pattern also can be seen in Fig. 1, looking from the top to the bottom row.

Fig. 1.

Schematic representation of somatosensory and visual receptive fields (RFs) of 11 representative bimodal ventral intraparietal area (VIP) neurons mapped in alert monkeys. Tactile responses were elicited with the monkey blindfolded, and visual responses were elicited with the monkey looking at a central fixation target on the tangent screen. Shaded areas on the monkey and screen represent tactile and visual RF surfaces. Note that for some visual RFs, the shaded area extends beyond the limits of the screen. This indicates that the RF was very peripheral, and cells were best driven by hand-held stimuli or large random dot patterns projected on the room ceiling or walls.

Fig. 2.

Relation between tactile RFs size and location for the population of bimodal VIP neurons. Height of each column represents the number of cells in each of 9 possible categories. Location of RFs is represented as the radial distance of its center from the tip of the nose.

Spatially congruent tactile and visual responses

The bimodal RFs of VIP neurons were organized in an orderly manner. Figure 3 shows the relationship between RF sizes in two sensory modalities. Tactile and visual RFs were measured within their own coordinate frame and are described accordingly in retinal coordinates for visual RFs and somatic coordinates for tactile RFs. Small central RFs were matched to small RFs on the muzzle, and large peripheral visual RFs were associated with large tactile RFs on the side of the head or body, providing further evidence for the center of the face as a somatosensory “fovea.” In the majority of neurons, the tactile RF showed a systematic relation to the main axes of the visual field. The spatial alignment of RF locations in the two sensory modalities is summarized for all recorded neurons in Fig. 4. Tactile RFs on the contralateral body side were associated with visual RFs in the contralateral visual field (Fig. 4 A), and tactile RFs on the upper part of the face were associated with visual RFs in the upper visual field (Fig. 4 B). It can be inferred from the two plots that the intersection of the retinal vertical and horizontal meridian has its somatosensory counterpart in the nose/mouth area.

Fig. 3.

Size of tactile and visual RF are related to each other.

Fig. 4.

Location of tactile and visual RFs. A: location of tactile RF center with respect to the body's midsagittal plane is in register with the location of the corresponding visual RF with respect to the vertical meridian. Majority of bimodal RFs are contralateral. B: location of tactile RF center with respect to the body's horizontal plane is in register with the location of the corresponding visual RF with respect to the horizontal meridian. There is an approximately equivalent representation of the upper and lower portions of the face and of the upper and lower visual fields.

Bimodal direction selectivity

A major feature of visual activity in VIP is a strong selectivity for speed and direction of moving stimuli in the frontoparallel plane. In the current sample, many cells with planar direction selectivity in the visual modality displayed a congruent directional sensitivity for moving tactile stimuli. Tactile direction tuning was ascertained by applying upward, downward, leftward, and rightward stimuli across the cell's RF. The method of stimulation did not allow us to measure velocity sensitivity. Preferred directions of visual and tactile stimuli coincided in the majority (83 of 97, 85%) of the cases. A typical example is shown in Fig. 5. The left panels show the response of a cell to a tactile stimulus moving in two opposite directions across the RF. The preferred direction for tactile stimulation matched the preferred direction for a moving spot of light projected on a tangent screen in one of eight possible directions at constant velocity (40°/s). The right panels only show the two directions of visual stimulation that matched those of the tactile stimulus. This neuron gave a sustained response to a tactile stimulus moving in the ipsiversive direction and responded briskly to the onset of visual motion in the same direction.

Fig. 5.

Cell with direction selective responses to tactile and visual stimulation. Unit was recorded in the left hemisphere. Cartoons (left) indicate that a tactile stimulus (a cotton tip applicator) moved from left to right (top) and right to left (bottom) across the cell's tactile RF while the monkey was blindfolded. Single trial rasters and poststimulus time histograms (PSTHs) below the cartoons are synchronized on stimulus movement onset (long vertical line). Neuron responded selectively to the leftward moving stimulus. Neuron was most active for the same direction of motion of a spot of light moving across the visual RF at 40°/s. Vertical calibration bar (left) corresponds to 100 impulses/s; tick marks on horizontal axis are 200 ms apart.

For most neurons, visual direction tuning was measured with computer controlled stimuli producing motion in the frontoparallel plane. In some cases, the optimal driving stimulus was not a light spot on a tangent screen. Many VIP neurons had RFs in the far periphery. For these neurons, hand-held stimuli were used to test for responses to motion beyond the limits of the screen. Figure 6 shows a neuron that had a very peripheral visual RF, best driven by a large object moved in the front-to-back direction on the right side while the monkey maintained fixation on the tangent screen. The cell also responded preferentially to a tactile stimulus moving front to back on the right side of the head. In the population tested, all directions of motion were represented. Parallel visual and tactile direction selective responses were recorded both in cells with central or peripheral RFs (Fig. 7). Some cells that responded to limb joint rotation also showed direction selective responses to passive arm movement (units 63L74 and 63K45 in Fig. 7).

Fig. 6.

Cell with direction selective responses to tactile and visual stimulation. Conventions as in Fig. 5. Cell responds best to a tactile stimulus moving front to back on the side of the head and to a large stimulus (hand-held stick) moving in the same direction in the far contralateral visual periphery. Vertical calibration bar (left) corresponds to 100 impulses/s; tick marks on horizontal axis are 200 ms apart.

Fig. 7.

Examples of cells with matched tactile and visual RFs and preferred direction of motion. →, preferred direction of motion of the RF stimulus.

Most bimodal VIP neurons showed congruent direction selectivity in the two sensory modalities (n = 83). Table 1 summarizes the frequency of the different patterns of visual and tactile responses to moving stimuli. Cells that did not display parallel directionalities fell into one of three categories. In a few isolated cases, the preferred directions of motion in the two modalities were not precisely coaligned but could be shifted by as much as 90° (n = 3). None of the cells in our sample showed direction-selective responses to tactile but not to visual stimuli, but some cells showed the reverse pattern (n = 11). In most cases, these were neurons that responded optimally, not to a spot of light moving in the frontoparallel plane, but to a three-dimensional object moving in depth, as described below. The remaining neurons were not directionally tuned (n = 26). Such cells could respond better to moving than stationary stimuli, without having any clear directional preference, or could respond equally well to stationary or moving objects.

View this table:
Table 1.

Correspondence between visual and somatosensory responses for bimodal neurons that were tested for direction selectivity in the two modalities

Other forms of complementary tactile and visual responses

We found VIP neurons that responded best to moving or stationary three-dimensional objects in near extrapersonal space and responded poorly or not at all to visual stimuli on the distant tangent screen. A subset of those near neurons (n = 11) gave strong responses to movement in depth toward (8 cells) or away (3 cells) from the animal. Sensitivity to motion in depth has no direct somatosensory analogue, and cells with this form of selectivity did not have selectivity for tactile motion. Instead they had nonmotion-selective receptive fields that included the point of impact of the object moving toward the head in space. In these cells, visual responses were a function of the projected point of impact of the object on the body rather than the retinal vector of the movement: the visual stimulus elicited a discharge when it was moving toward the location corresponding to cell's tactile RF but not when it was moving toward another part of the body. Analysis of the somatosensory responses in neurons responding to such patterns of visual motion showed that the match between the two modalities is best regarded as complementary. For instance, cells responding to motion in depth toward the monkey responded to the onset of a tactile stimulus (on response; Fig. 8, unit 64079). Conversely, when the effective visual stimulus was one that moved away from the monkey's face, the tactile response was triggered by the removal of the stimulus (off response; Fig. 8, unit 64068).

Fig. 8.

Complementary responses to tactile and visual stimulation for 2 bimodal VIP neurons. Both cells had a tactile RF on the upper right side of the head and a visual RF in the upper right visual periphery. Although both neurons could be driven by a stimulus presented on the tangent screen, the most vigourous responses were obtained with a real object moving in depth. These 2 cells did not show direction selective tactile responses. Cell 64079 responded transiently to the onset of the tactile stimulation and was activated more strongly by an approaching than by a receding object. Cell 64068 did not respond to the onset of a tactile stimulus. A transient response was evoked by removing a stimulus already touching the head, and the cell was activated more strongly by a receding than by an approaching object. Vertical calibration bar (left) corresponds to 100 impulses/s; tick marks on horizontal axis are 200 ms apart.

Neurons with no motion or direction sensitivity often had corresponding somatosensory response properties. In some of these cases, in addition to RF size and position congruence, the cells showed matching discharge patterns and stimulus-feature selectivity. For instance, some VIP neurons have robust responses to the appearance and disappearance of a stationary visual stimulus (n = 5). These neurons were likewise strongly responsive to the onset and offset of a stationary tactile stimulus applied to the somatosensory RF (Fig. 9). A small number of cells responded selectively to the orientation of a visual stimulus. Visual orientation selectivity was tested with a stationary white bars, and tactile orientation selectivity was tested by applying a short stationary bar on the RF. In the visual modality, a slit maker was used to generate bars oriented at intervals of 45°. For tactile stimulation, we applied the stimulus at the same orientation as the cell's preferred visual orientation and at the orthogonal orientation. Because the tactile stimuli were only applied with the head upright, the frames of reference for orientation were equivalent in the visual and tactile modalities. Again, bimodal VIP cells showed congruent response selectivity when oriented tactile stimuli were applied in their RF (n = 3; Fig. 10).

Fig. 9.

Cell with transient on and off responses to application and removal of a tactile stimulus and to the appearance and disappearance of a visual stimulus. Spike trains are synchronized on stimulus onset. Vertical calibration bar (left) corresponds to 100 impulses/s; tick marks on horizontal axis are 200 ms apart.

Fig. 10.

Cell with orientation selective responses in the somatic and visual modalities. Spike trains are synchronized on stimulus onset. Vertical calibration bar (left) corresponds to 100 impulses/s; tick marks on horizontal axis are 200 ms apart.

Characteristics and location of bimodal neurons versus purely visual neurons

Pure visual and bimodal visual-tactile neurons were not distinguishable on the basis of their visual response properties. The distribution of visual RF locations, the proportion of cells with selective responses to the direction of motion and to the distance of visual stimuli, did not differ significantly in the two populations. Furthermore, histological analysis of the distribution of neurons with and without somatosensory responses showed complete intermingling of the two cell classes. Therefore bimodal VIP neurons do not constitute an anatomically distinct subpopulation of neurons in area VIP, and the only apparent difference between visual-only and bimodal neurons was indeed the presence of sensitivity to somatosensory stimulation.

No clear topographical organization was found that would suggest the existence of a retinotopically or somatotopically organized map. Neurons recorded in the same penetration frequently had adjacent and/or overlapping receptive fields. However, tactile RF could progress from contralateral to ipsilateral and back to contralateral or from upper face to lower face and back to upper face in series of adjacent penetrations. Because VIP extends on both the lateral and medial banks of the sulcus, these apparent abrupt jumps could be due to the fact that adjacent penetrations did not enter adjacent zones of area VIP. In two different hemispheres, we plotted the information from electrode penetrations records on a flattened map reconstruction of the physiologically defined borders of area VIP as identified by marking lesions. Even with this method, the abrupt discontinuities remained evident, and no systematic progression of receptive field locations was observed with respect to the anterior/posterior or medial/lateral extent of area VIP.


VIP as a physiologically and anatomically distinct area in the intraparietal sulcus

We have studied neurons located in the ventral portion of the intraparietal sulcus and have found two major response characteristics: spatially aligned visual and tactile RFs and direction-selective responses to moving stimuli in both modalites. This distinguishes VIP from adjacent area lateral intraparietal area (LIP) on the lateral bank of the sulcus where neurons do not respond to tactile stimuli and are not strongly selective for the direction and speed of moving stimuli (Colby and Duhamel 1991). The response properties of neurons in the fundus of the intraparietal sulcus also differ from those recorded on its medial bank in the medial intraparietal area (MIP). Area MIP contains bimodal cells but their receptive fields and response properties are different from those of VIP in several respects: the tactile RF are usually located on the upper limbs or hands, as in area 5 on the superior parietal lobule convexity; although some cells may respond to cutaneous stimulation, most prefer joint rotation and are best activated during active arm reaching or pointing movements in darkness; visual responses in area MIP are not as frequent or as brisk as in area VIP and are not direction selective (Colby and Duhamel 1991).

Consistent with this physiological distinctiveness, the fundus of the intraparietal sulcus has a unique pattern of cortical connections. VIP is a middle-level area in the cortical visual processing hierarchy (Felleman and Van Essen 1991). Quantitative analysis of the laminar pattern of connections within the extrastriate cortex shows that VIP is one of the rare areas with a fixed relative position in this hierarchy, located at the same level as area MSTd (Hilgetag et al. 1996). It originally was defined anatomically as the MT projection zone in the intraparietal sulcus (Maunsell and Van Essen 1983; Ungerleider and Desimone 1986). Although both VIP and LIP are connected to area MT, VIP forms visual connections exclusively with dorsal stream areas (MT, MST, FST, PO), whereas LIP has a much more extensive set of connections with other dorsal and ventral stream areas (V2, V3, V4, MDP, DP, TEO, TE) (Baizer et al. 1991; Blatt et al. 1990; Ungerleider and Mishkin 1982).

Somatic input to area VIP

Several sources of input could provide a route for somatic information into VIP. Like the visual system, the cortical somatosensory system is organized hierarchically. In SI cortex, information is channeled from area 3 to areas 1 and 2 and is subsequently relayed to SII, area 5, and area 7b in the parietal lobe (Jones and Powell 1970, 1973). The fundus of the intraparietal cortex receives projections from convexity area 5, the medial bank of the intraparietal sulcus, and the rostral inferior parietal lobule or area PF (Jones and Powell 1970; Neal et al. 1990a,b; Pandya and Kuypers 1969; Pandya and Selzer 1982; Seltzer and Pandya 1986). The presence of a population of bimodal neurons in this region is therefore consistent with the prediction, initially formulated by Seltzer and Pandya (1980) on anatomic grounds, of a visual and somesthetic convergence zone in the fundus of the intraparietal sulcus and suggests that VIP is an early point of intersection between the visual and somatosensory processing hierarchy (Van Essen et al. 1990).


The most distinctive feature of somatosensory activity in VIP is its directional selectivity. In studying tactile directional selectivity, one must always worry that differences in the responses of cortical neurons arises not from neural processing but from the different orientation of the receptors themselves. This is unlikely to be the case in these experiments because the same cutaneous receptive fields had cells with orientation selectivity in all four quadrants of direction. We therefore are confident that the directional selectivity is a function of neural processing.

Directional selectivity has been convincingly demonstrated by several groups (Constanzo and Gardner 1980; Hyvärinen and Poranen 1978) in areas 1 and 2 of SI and in SII (Whitsel et al. 1972). It also has been found in areas 5 (Sakata et al. 1973) and in 7b (Graziano and Gross 1994;Leinonen and Nyman 1979; Leinonen et al. 1979; Robinson and Burton 1980). In these higher-order areas, the response patterns can be more complex than in lower-order areas, involving multijoint configurations or active instead of passive movement. It is not clear which of these somatosensory areas provides VIP with directional information. Area 5 mostly contains cells with RFs located on the limbs and trunk (Sakata et al. 1973), whereas the somatic representation in VIP largely is confined to the head. In area 7b, a rough somatotopic organization is present. Directionally tuned activity has been found mostly in the lateral convexity of area 7b, where the majority of cells have RFs located on the limbs or are large and bilateral, often covering the whole body (Leinonen et al. 1979) as in area 5. More medially, at the tip of the intraparietal sulcus and near the 7b/SI border, Leinonen and Nyman (1979) identified another region containing a representation of the face analogous to that found in VIP that they termed “associative face area.” However, direction-selective cutaneous responses are not reported. Most cells in this subregion of area 7b are active during passive stimulation of the face and inside the mouth or during active mouth and lips movements (Leinonen and Nyman 1979).

Bimodal visual-tactile response patterns in VIP

The proportion of bimodal, visual-tactile neurons in VIP was ∼70%. These neurons have spatially aligned visual and tactile RFs the size and location of which were related to each other in an orderly manner. Specific parts of the visual field were matched with specific parts of the head, with the central area of the visual field being associated consistently with the region of the muzzle. The most frequent response pattern in VIP is characterized by congruent preferred directions for moving visual and tactile stimuli. VIP may be unique in containing a majority of such bimodal direction-selective neurons. Cells showing this type of response could be driven with spots of light projected on the distant tangent screen or by a real object moved by hand in the far periphery of the visual field when the RF extended outside the limits of the screen. The paired tactile RF varied in size, but it usually covered a circumscribed portion of the face and, more rarely, the trunk or the arm. The presence of limb directional-selective somatosensory responses to limb joint rotation might reflect an involvement of such units in the analysis of hand movements made around or toward the face and in hand/mouth coordination. We previously reported that about half of visual neurons tested in VIP showed some kind of selectivity for the stimulus distance (Colby at al. 1993). Some of these neurons responded to frontoparallel motion of a visual stimulus on a distant tangent screen but gave a stronger response to the same moving retinal stimulus presented closer to the monkey. These neurons showed the same pattern of congruent direction selectivity for visual and tactile stimuli as neurons with no sensitivity to stimulus distance. Other neurons responded best to motion in depth or preferred stationary near (20–40 cm) or ultra-near (10 cm) visual stimuli to more distant ones. The response evoked in the corresponding tactile RF was usually not direction selective.

Multisensory convergence in other brain areas

Populations of neurons that respond to both visual and somatic stimuli have been found in several brain regions, with similarities and differences with respect to the findings reported here for area VIP. Multisensory activity has been described in the mouse (Drager and Hubel 1975) and cat superior colliculus (Stein et al. 1975). However, few neurons in the superior colliculus are directionally selective, (Clemo and Stein 1987), with the exception of vibrissae-driven neurons in the mouse (Drager and Hubel 1975). Stein and his colleagues have demonstrated the convergence of visual, auditory, and/or somatosensory input on single neurons and have described the spatial and temporal interactions between different stimulus modalities (Clemo and Stein 1987; Meredith and Stein 1983; Meredith et al. 1987). The present study was not designed to assess eventual facilitatory or inhibitory effects of multimodal stimulation because we always applied an optimal stimulus in each modality during independent blocks of visual and somatosensory trials.

The macaque superior temporal polysensory area (STP) contains neurons responding to visual, tactile, and auditory stimuli alone or in combination (Bruce et al. 1981; Desimone and Gross 1979; Hikosaka et al. 1988). Bimodal visual-tactile neurons in area STP represent only a small subset of the multisensory cells. The somatic representation differs from that found in VIP in that all body parts are present and the RFs are usually large and bilateral (Bruce et al. 1981). STP also contains neurons responding to complex “biological” visual stimuli, such as faces and body parts, and is likely to have a different functional role than area VIP (Bruce et al. 1981; Perrett et al. 1985, 1989; Wachsmut et al. 1994).

Three other areas contains bimodal, visual/somatosensory neurons with response properties similar to those found in VIP. In area 7b, about one-third of neurons respond to both visual and tactile stimulation, a minority are purely visually driven, and the remaining neurons are somatosensory or somatomotor (Graziano and Gross 1994; Hyvärinen 1981).Thus in contrast to area VIP, somatic sensation is the prevalent modality in area 7b. Outside of parietal cortex, the inferior premotor cortex (Gentilucci et al. 1983, 1988; Graziano et al. 1994; Rizzolatti et al. 1981) and the putamen (Graziano and Gross 1994) contain bimodal neurons that also display a close resemblance to those found in VIP. As in area 7b, the prevailing sensory modality is somatic, although in both areas a subset of cells are purely visual. The putamen has a complete somatotopic map of the body, including a large proportion of bimodal neurons with tactile RFs on the head (Gaziano and Gross 1994). Similarly, in the F4 subdivision of inferior premotor cortex, more than half of the neurons have a tactile RF on the face (Fogassi et al. 1996). Areas VIP, 7b, F4, and the putamen therefore might form afunctional network in which VIP represents an early cortical stage for the integration of visual and somatic information.

The pattern of parallel directional selectivity for visual and tactile stimulation described here for area VIP has been observed in a small subset of area 7b neurons (Leinonen et al. 1979). The other main response pattern found in VIP, visual motion in depth and nondirection-selective tactile RFs, is by far the most consistently reported one in area 7b and in the premotor cortex, where most bimodal cells discharge to an approaching visual stimulus with a projected point of impact that matches to the location of the tactile RF (Fogassi et al. 1996; Gentilucci et al. 1983; Graziano and Gross 1994a; Leinonen and Nyman 1979; Leinonen 1980). Unlike VIP, the majority of neurons in 7b and F4 could not be driven with small spots of light on a distant screen and responded effectively only to visual stimuli located within a short distance (10–40 cm) from the animal, in the range of the near and ultra-near neurons described in VIP (Colby et al. 1993; Duhamel et al. 1989). This suggests that the functional homology between VIP and areas 7b and F4 may be limited to the second subclass of bimodal neurons.

Graziano and Gross (1994b) have shown that the visual RF of such “approach” neurons in the putamen and premotor cortex, but not area 7b, remains anchored to the matched tactile RF on the hand regardless of the hand's position in space, thereby encoding the trajectory of the incoming object in “body parts coordinates.” In area F4, cells responding to an object moving toward the head have been studied using a well-controlled visual stimulus mounted on a robot arm (Fogassi et al. 1996). The main variable affecting the neurons' discharge was found to be the direction of the approaching stimulus with respect to the tactile RF, similarly to VIP neurons responding to motion in depth. Furthermore, the visual RF of area F4 cells also had a limited extension in depth, and their sensitivity to the distance of the stimulus varied as a function of approaching movement speed, suggesting that these neurons could convey predictive information about time to collision.

Role of multisensory neurons in VIP

It has been suggested that multisensory convergence contributes to interpreting the functional significance of stimuli linked by common causality (Meredith et al. 1987). In a natural environment, motion-sensitive VIP neurons that have congruent directional selectivity would become active in two kinds of circumstances: during a movement of an external object (or the animal's own hand) occurring close enough to the face as to produce a concomitant tactile stimulation and during self-motion when head displacements produce congruent optical and tactile flow, as is the case when the animal navigates in dense vegetation. VIP is highly similar to area MT in terms of the prevalence and degree of direction selectivity for visual motion (Colby at al 1993). Also, as in area MST (Duffy and Wurtz 1991, 1995; Tanaka and Saito 1989), a subset of VIP neurons are sensitive to optic flow patterns simulating expansion and contraction of the visual field that are produced by self-motion (Bremmer et al. 1997). The addition of tactile input to such neurons may serve, as suggested by Leinonen et al. (1979) “[to analyze] the direction of a stimulus moving in one sensory system using another sensory system as reference” (p. 299).

In one likely scenario, the somatic modality is used as reference frame for coding visual information. This seems to be the case for VIP neurons that respond to motion in depth. Because most VIP neurons have a tactile RF on the head, the reference frame for visual input can be identified by moving an object along a fixed path in space while the head-restrained animal fixates in one of several possible directions. VIP approach neurons were tested in this manner and found to discharge in relation to the projected point of impact of the object irrespective of its particular retinal vector (Colby et al. 1993). Conversely a similar set of retinal trajectories directed at a point outside the RF were not effective. Another argument for a contribution of VIP to a head-centered representation of space is the presence of cells that respond to ultra-near visual stimuli presented near the matched tactile RF regardless of eye position. A different possibility is that somatosensory stimuli are encoded in a visual frame of reference. This would be reflected, for instance, in tactile RFs that move on the body in conjunction with changes in gaze direction or, if it is located on a limb, by a compensatory displacement of the RF when the limb is displaced. In VIP, most RFs are located on the head, and because our method of somatosensory testing did not have the requisite accuracy to measure the stability of RF contours as a function of gaze shift, our data did not address directly this possibility.

It is becoming increasingly clear that the representation of space in parietal cortex is not a single monolithic function. It is achieved by several specialized areas that use different encoding formats and participate in different types of sensorimotor transformations. For example, the involvement of the LIP in saccadic eye movements is supported by both anatomic and electrophysiological data. LIP is connected to the superior colliculus (Lynch et al. 1985) and frontal eye fields (Andersen et al. 1990). Many LIP neurons have saccade-related activity and visual information appears to be encoded in eye-centered coordinates (Colby et al. 1995; Duhamel et al. 1992; Gnadt and Andersen 1988; Goldberg et al. 1990). Although activity in LIP can be dissociated easily from actual eye movements (Colby et al. 1996), it is clear that the oculomotor system uses information from LIP for the guidance of eye movement. It may be that LIP's role in the analysis of space is limited to the range of distant space unique to eye movements. In contrast, anatomic evidence suggests that area VIP is connected with F4, a region of the inferior premotor cortex implicated in the control of head and mouth movements (Lewis and Van Essen 1996; Matelli et al. 1994). The present results suggest that visual information in VIP may be represented at the single cell level in a somatic reference system centered around the face. Recent findings obtained with quantitative visual RF mapping experiments showed that although some VIP cells encode the retinal coordinates of a stimulus, another subset of neurons encode the azimuth and/or elevation of a visual stimulus in head-centered rather than retinal coordinates (Duhamel et al. 1997). Finally, many VIP neurons recently were found to respond to head rotation in total darkness (Bremmer et al. 1997), thus raising the possibility that this area contributes to yet a third, inertial reference frame for representing spatial information. These findings taken together raise the interesting possibility that neurons in a single cortical area may contribute to multiple representations of space.


Anatomic and electrophysiological data indicate that VIP is involved in the multisensory analysis of stimulus motion. The presence in area VIP of cells that are sensitive to near stimuli and the convergence of somatic and visual information point to a key difference between VIP and other extrastriate areas involved in motion processing. The analysis of the relations between the movements of an observer and those of external objects likely depends on several interrelated cortical areas. The current evidence suggests that the third dimension of space is not represented in a single continuous map and that different areas may be specialized for processing information in different portions of space, VIP being particularly involved in the analysis of near extrapersonal space. Area VIP stands out in both its multisensory stimulus encoding and in its contribution to multiple spatial representations.


We are grateful to Dr. James Raber for veterinary care, T. Ruffner and N. Nichols for machining, M. Smith for histology, A. Hayes for computer support, and J. Steinberg and B. Harvey for facilitating.

This work was supported by the National Eye Institute and the Centre National de la Recherche Scientifique.


  • Address for reprint requests: J.-R. Duhamel, Laboratoire de Physiologie de la Perception et de l'Action, CNRS-Collège de France, 11, Place Marcelin Berthelot, 75005 Paris France.


View Abstract