In order to elucidate the neural mechanisms involved in the perception of the three-dimensional (3D) orientation of a surface, we trained monkeys to discriminate the 3D orientation of a surface from binocular disparity cues using a Go/No-go type delayed-matching-to-sample (DMTS) task and examined the properties of the surface-orientation–selective (SOS) neurons. We recorded 57 SOS neurons from the caudal part of the lateral bank of the intraparietal sulcus (area CIP) of three hemispheres of two Japanese monkeys (Macaca fuscata). We tested 29 of 57 SOS neurons using the square plate of a solid figure stereogram (SFS) and random-dot stereogram (RDS) without perspective cues; almost all of the tested neurons (28/29) showed surface orientation selectivity for the SFS and/or the RDS without perspective cues. Eight of these 28 neurons (28.6%) showed selectivity for both the RDS and SFS, 7 (25.0%) were dominantly selective for the RDS, and 13 (46.4%) were dominantly selective for the SFS. These results suggest that neurons that show surface orientation tuning for the RDS without perspective cues compute surface orientation from the gradient of the binocular disparity given by the random-dot across the surface. On the other hand, neurons that show surface orientation tuning for the SFS without perspective cues may represent surface orientation primarily from the gradient of the binocular disparity along the contours. In conclusion, the SOS neurons in the area CIP are likely to operate higher order processing of disparity signals for surface perception by integrating the input signals from many disparity-sensitive neurons with different disparity tuning.
Stereopsis is the ability to perceive a three-dimensional (3D) object based on the differences between its images projected on the two retinae. Although there are monocular cues for representing the 3D shape of an object, such as texture, shading, and linear perspective, binocular disparities are crucial for building a real 3D representation of an object in a viewer-centered frame of reference. Marr (1982) postulated in his theory of vision that an important step in visual information processing for 3D representation of an object is the description of the geometry of the visible surface, especially that of the surface orientation and curvature, based on the disparity gradient. Furthermore, many kinds of disparity cues for the representation of surface orientation have been demonstrated from psychophysical studies (Howard and Rogers 1995). Integrative processing of these different types of disparity signals is necessary to build a neural representation of the surface orientation in depth. Recently, we identified a group of neurons in the parietal cortex of alert monkeys that discriminated the surface orientation of a stereoscopic stimulus and designated them as surface-orientation–selective (SOS) neurons (Shikata et al. 1996). Most of these SOS neurons were sensitive to binocular disparity cues; however, it is not yet known what specific disparity cues activate the SOS neurons. In the present study, we trained monkeys to discriminate the 3D orientation of a surface by binocular disparity cues using a Go/No-go-type delayed-matching-to-sample (DMTS) task and attempted to specify the critical cues for the representation of surface orientation by SOS neurons in the caudal part of the lateral bank of the intraparietal sulcus (area CIP, designated by us as area cIPS in previous papers).
Two Japanese monkeys (Macaca fuscata) were used in the present study. Throughout the experiments, the monkeys were treated in accordance with the National Institute of Health Guide for Care and Use of Laboratory Animals. This project was approved by the Ethical Committee of Nihon University School of Medicine.
All stimuli used in this study were generated by a graphics computer (SGI, Indigo2) and presented on a 21-in. display (1,240 × 1,024 pixels) with a liquid crystal polarized filter (Tektronix, SGS610). The computer programs for stimulus presentation were developed in collaboration with Solidray (Yokohama, Japan). The display was placed 44 cm in front of the monkey at eye level. The filter was switched at 120 Hz synchronized with each frame of the display, whereby 60 frames/s of stimulus were presented to each eye. The monkey wore polarized glasses to view these stimuli stereoscopically.
Figure 1 A shows the samples of binocular pairs used in the present study. Neither the solid figure stereogram (SFS, top row) nor the random-dot stereogram (RDS, bottom row) had perspective cues in the fused image. The size of the frontoparallel square plate stimulus was 6.3 × 6.3°. In the regular recording session, the fixation point and the center of the stimulus were presented without disparity 44 cm away from the monkey. The binocular subtense (convergence angle) of the fixation point was 3.9°. Pure red was chosen as the color of the stimuli to prevent ghost stimuli from appearing inappropriately in the eye when the filter was switched. All stimuli used in this study were rendered without shading or texture. The square-plate SFS was of minimum thickness (1 dot on the display, 0.0385 × 0.0385°), so that only the edges provided depth cues. In the RDS, an abrupt change in disparity provided the contours of the square plate. The dot size of the RDS was 0.0385(w) × 0.077(h)°, and the density was 50%. The background of the RDS was correlated random-dot with 2.3° uncrossed disparity.
Because the size of the stimulus was 6.3 × 6.3° and the stimulus was inclined and/or slanted by 45°, the disparity of stimulus ranged from −0.16 to 0.14° (crossed and uncrossed disparity ranges, interocular distance was 3 cm), when the fixation point was on the screen. The position invariance of the response of the SOS neurons in depth was tested by varying the distance of the fixation point instead of changing the disparity of the stimulus. The binocular subtenses of the convergent and divergent fixation points were 4.4 and 3.5°, respectively, corresponding to 5 cm in front of and behind the display. The fixation of the near spot brought the stimulus entirely in the uncrossed disparity region (ranging from 0.34 to 0.64°), whereas that of the far spot brought the stimulus entirely in the crossed disparity region (ranging from −0.26 to −0.56°), and both regions were outside the range of the regular stimulus.
We used a Go/No-go–type DMTS task (Fig. 1 B) and analyzed the neural response to the sample stimulus only in successful trials. When a small fixation point (0.2°, FP) appeared, the monkey pressed the key (KEY) and fixated on the spot. The monkey had to fixate on the spot until it released the key at the end of the trial. The presentation time of the sample (sample) and matching (match) stimuli was 1 s or 750 ms, and the delay period was 2 s. If the surface orientation of the sample stimulus was the same as that of the matching stimulus, the monkey had to release the key as soon as possible after the color of the fixation point changed (Go trial); however, if the surface orientation was different, the monkey had to release the key after the fixation point was turned off to obtain the reward (No-go trial).
One recording session consisted of five blocks, each of which included nine trials (45 trials per session). In one block, a stimulus in a set of nine orientations (Fig. 2,A and B) was presented as the sample stimulus in random order. The matching stimulus was selected so that half of the trials became Go trials and the remaining half became No-go trials in one session. Throughout one session, either the SFS or RDS was used as the stimulus.
Before recording, a stereotaxic magnetic resonance image (MRI) of the brain of each monkey was made. These images included the markers that indicate the stereotaxic AP level of that image. For head fixation, a halolike metal ring was implanted in each monkey's skull under pentobarbital sodium anesthesia. After the recovery from the surgery, the monkeys were retrained for the DMTS tasks under the head fixating condition at least for 4 wk, then a microelectrode recording chamber was stereotaxically implanted in the opening of the skull over the parietal cortex under pentobarbital sodium anesthesia.
Extracellular single-unit recordings were made in the lateral bank of the intraparietal sulcus using tungsten microelectrodes (Fig.1 D). We made probe penetrations to map the location of the intraparietal sulcus (IPS) at an early stage of the recording and compared it with the estimated location of IPS from the stereotaxic MRI map to make corrections in the position of the chamber. Thus the penetrating track of the electrode could be nearly superimposed onto the stereotaxic MRI brain map. During the electrode penetration, we carefully checked the extent of the gray and white matter based on the wave form of single units (spikes of fibers are monophasic and positive) and the background multiunit activity (less active in the white matter) and compared with the MRI map to estimate the electrode track and the depth position of the electrode tip. We also used the physiological criteria to assign neurons to area CIP. Neurons in V3A, which is one of the adjacent areas of area CIP, had smaller receptive fields and were more easily activated by simple visual stimuli than the CIP neurons. On the contrary, many neurons in area LIP, which is anterior to area CIP, were eye movement related (either saccade or fixation) and difficult to activate by the stimulus sets we used in the present study.
Receptive fields of the neurons were tested by presenting a small white square (1 × 1°) on the display while the monkey fixated on the small spot at the center of the screen. The approximate position of the areas in which the neuron responded were hand-plotted. In a few neurons, the receptive field did not include the fovea. In this case, we moved the position of the stimulus up to 3.5° from the fixation point so that the entire stimulus was in the receptive field.
We monitored the position of the right eye routinely during unit recording using an infrared eye movement recording system (sampling rate 250 Hz, RMS, Hirosaki, Japan) to confirm that the monkeys made stable fixation during the task (Fig. 3,E and F).
To confirm that the monkey made accurate vergence movement especially when we varied the fixation distance, the detailed measurements of eye position were made in the second monkey after the unit recording (Fig.1 C). These measurements were carried out while the monkey performed the DMTS task as during unit recordings, and the fixation distance was changed by changing the binocular subtense of the fixation spot (see legend of Fig. 1 C for more detail). We measured the left and right eye position separately in different sessions; however, we set the same task conditions for the measurement of both eye positions. We could estimate the vergence angle from these data, because the shifts of left and right eye positions for a particular binocular subtense of the fixation point were nearly symmetrical.
Preferred orientation: The orientation tuning curve of SOS neurons could be described by a sinusoidal function, according to the following equation where d i is the discharge frequency with orientation θi, θo is the preferred direction, andb 0 andc 1 are regression coefficients. For details of the calculation see Georgopoulos et al. (1982).
The SFS-RDS index is as follows where R SFS is the neural response to the square plate of the SFS without perspective cues at the best orientation and R RDS is the neural response to the square plate of the RDS without perspective cues. If one response was 60% of the other, the index was ±0.25. If the neuron responded selectively only to the SFS,R RDS was 0, resulting in an SFS-RDS index of 1, and if it responded only to the RDS, the index was −1.
We recorded 134 neurons from 44 penetrations into the caudal part of the lateral bank of the intraparietal sulcus (area CIP) of 3 hemispheres of 2 Japanese monkeys (Macaca fuscata). Of these, 63 neurons responded to the SFS with perspective cues. The statistical significance of the tuning was tested, and 57 of the 63 neurons showed selectivity for surface orientation (Rayleigh test,P < 0.05) (Mardia 1972); these neurons were therefore designated as SOS neurons (Table1 A). We plotted the recording sites of these 57 SOS neurons on a stereotaxic MRI brain map (Fig.1 D). The SOS neurons were located in the caudal part of the lateral bank of the intraparietal sulcus, between area LIP and area V3A. This area was designated as area cIPS in a previous paper (Sakata et al. 1997); however, we use area CIP to represent this area in this paper. The preferred orientations of 57 SOS neurons calculated from a regression equation (see methods) were distributed randomly (Rayleigh test, P > 0.10) and covered almost all directions. Most of the SOS neurons tested had relatively large receptive fields (10 × 10° to more than 30 × 30° beyond the size of the screen), which included the fovea. Thirty-nine of the 57 (68.4%) SOS neurons showed significantly greater response to the SFS than to the solid figure (SF) without disparity (Student's t-test, P < 0.05), suggesting that perspective cues of the SF do not have much effect and that it is the binocular disparity cues that are predominantly effective for the discrimination by SOS neurons of surface orientation in depth. These results confirm our previous study (Shikata et al. 1996).
We further analyzed the properties of 29 of the 57 SOS neurons using stimuli consisting exclusively of binocular disparity cues (Fig.1 A). Almost all of the tested neurons (28/29) responded to the RDS and/or the SFS without perspective cues (Table 1 B). Figure 2 shows examples of the responses of an SOS neuron to a set of these stimuli in nine orientations. This neuron showed high selectivity for the surface orientation of the SFS without perspective cues (Fig.2 A), suggesting that surface orientation was represented by the binocular disparities of the contours (Howard and Rogers 1995; Sakata et al. 1998). This neuron also showed orientation selectivity in response to the RDS (Fig.2 B). The only cue for surface orientation in the RDS without perspective cues was the gradual change in horizontal binocular disparity across the square plate. Thus the surface orientation seemed to be computed from the disparity gradient, as postulated byMarr (1982) on the basis of the psychophysical theory of perception proposed by Gibson (1950). Figure 3,A–C, shows the tuning curves of neurons that responded to both the SFS and RDS without perspective cues (A), mainly to the RDS (B), and mainly to the SFS (C). To compare the response to the SFS with those to the RDS, we calculated the SFS-RDS index (see methods). Seven neurons responded predominantly to the RDS (SFS-RDS index < −0.25) and were designated as the “RDS-dominant” type. Thirteen responded predominantly to the SFS (SFS-RDS index >0.25) and were designated as the “SFS-dominant” type. Eight were an intermediate type showing similar responses to both the SFS and RDS (SFS-RDS index between −0.25 and 0.25). However, because the SFS with perspective cues was used to select the SOS neurons (57/63), there was a possibility that the number of “RDS-dominant” neurons was underestimated. In the neurons that showed significant responses to both the SFS and RDS (n= 17), the preferred orientations for both stimuli were almost the same (r = 0.74, P < 0.01, circular correlation) (Batschelet 1981).
Because a slanted surface covers a wide range of horizontal disparity, it was necessary to exclude the possibility that the SFS and RDS stimuli simply hit or missed the 3D receptive field of putative SOS neurons depending on the orientation. Therefore we examined the effect of the stimulus position in depth by changing the vergence angle of the fixation point (see also methods). Figure 3, Band C, shows typical neurons whose activities were independent of the depth of the stimuli. The neuron in Fig.3 B was an “RDS-dominant” type. Whether the entire RDS was located closer than the fixation point, i.e., in the crossed disparity region, or farther than it, i.e., in the uncrossed disparity region, the orientation tuning curve of this neuron was almost the same as the original one. The neuron in Fig. 3 C was an “SFS-dominant” type. The orientation tuning curve of this neuron was also unaffected by the position of the fixation point. Six of the eight neurons tested (from 2 monkeys) showed no change in their orientation tuning with change in stimulus depth relative to the fixation point.
Eye position traces taken during the recording session of each neuron indicated that there was no abrupt change of eye position even when the stimulus was presented at a different depth position from the fixation point (Fig. 3, E and F), confirming the monkey's stable fixation on the fused fixation point. In almost all conditions of eight neurons, the mean eye position during the stimulus presentation (200–700 ms after the stimulus onset) did not change from that before the stimulus presentation (300 ms before the stimulus onset; P > 0.10, paired t-test). Only in one condition (Fig. 3 F, middle) was the mean eye position during and before stimulus presentation significantly different (P < 0.05); however, the eye position shift was very gradual and toward the opposite direction to the stimulus location. Furthermore, the measurements of eye position in relation to the change of fixation distance indicated that the shift in position of both left and right eyes was linearly correlated to the shift of binocular subtense of the fixation point (r = 0.992 and −0.987, respectively, with the SFS, Fig. 1 C;r = 0.981 and −0.952, respectively, with the RDS, not shown). These results suggested that both eyes accurately converged or diverged according to the position of the fixation point.
The major finding of this study is that the majority of SOS neurons in area CIP responded preferentially to the surface of the RDS in a particular orientation that had no other cues for surface orientation than a gradient of binocular disparity. Most of the 28 tested neurons (n = 20) showed orientation selectivity for the surface of the RDS without perspective cues, suggesting that these neurons compute surface orientation purely from the gradient of the binocular disparity. This finding provides strong evidence in support of Marr's hypothesis (Marr 1982) in his computational theory of vision, that surface orientation can be computed in the visual system from the gradient of binocular disparity across the surface. The hypothesis was based on the psychophysical theory of surface perception proposed by Gibson (1950), which postulates that gradients of binocular disparity, as well as those of texture density, are critical cues for the perception of surface orientation in depth. Early psychophysical studies byAmes (1935) and Ogle (1938) showed that the magnification of one of the two retinal images in the horizontal direction with a meridional lens induces the perception of a slant of a surface textured with small irregular dots. Recent psychophysical studies have demonstrated that the disparity gradients in an RDS cause the perception of slanted or inclined surfaces (Cagenello and Rogers 1993; Gillam et al. 1988; Gillam and Ryan 1992). The correspondence between the response property of the SOS neurons and the psychophysical estimation of surface orientation strongly suggests that the activity of the SOS neurons may correspond to the perception of surface orientation.
The second finding of this study is that many SOS neurons in the area CIP responded preferentially to the surface of the SFS in a particular orientation that had no other cues for surface orientation than the disparities along the contours. One possible mechanism of the selectivity of those neurons is that they computed the surface orientation from the orientation disparity and/or width disparity of the contour. This agrees with the classical theories of stereopsis, which have been proposed since Wheatstone's invention of the mirror stereogram (Wheatstone 1838), that the orientation and width disparities of contours allow the perception of inclination and slant of lines and surfaces in depth (Howard and Rogers 1995). From the theoretical point of view, however, the orientation disparity and the width disparity of the contour could not be segregated from the gradient of the binocular disparity along the contours of SFS. The orientation disparity can be explained by a gradient of disparity along the vertical or diagonal edges, and the width disparity can be explained by a gradient of disparity along the horizontal edges (Howard and Rogers 1995). Therefore the evidence from the SFS-dominant neurons in this study is ambiguous about whether the surface orientation was computed from the gradient of the binocular disparity along the edges or from the orientation and/or width disparity of the edges.
The selectivity for the surface orientation of the RDS-dominant neurons may be a result of higher-order processing of binocular disparity beyond the prestriate cortices. Neurons selective for horizontal disparity were first identified in the striate cortex of the cats (Barlow et al. 1967; Nikara et al. 1968), and later in the visual cortical areas V1, V2, V3, and V3A of the monkey (Hubel and Livingstone 1987; Poggio et al. 1985, 1988) as well. The neurons in these areas had relatively small receptive fields with retinotopic organization and narrow ranges of disparity tuning. Area CIP is adjacent to area V3A, and wheat germ agglutinin—horseradish peroxidase (WGA-HRP) injections to area V3A have shown corticocortical connections from this area to area CIP (Adams 1997). Because the V3-V3A complex contains plenty of disparity-sensitive neurons, it is plausible that the RDS-dominant SOS neurons may integrate the signals of a set of V3A neurons with different disparity tuning to compute a gradient of binocular disparity. By analogy with the vector field hypothesis for the optic-flow–sensitive neurons in area MSTd that were selective to the planar, circular, or radial optic flow fields (Duffy and Wurtz 1991), position invariant responses of SOS neurons in area CIP may be explained by their unique sensitivity to the distributed properties of the disparity gradient across large receptive fields. However, any concrete model of the neural circuit to compute the disparity gradient is a matter of speculation.
Neural mechanisms to detect the orientation and width disparity are currently less plausible. Neurons that respond to orientation disparity were reported in the striate cortex of the cat (Blakemore et al. 1972) and the monkey (Hänny et al. 1980) but are rather rare. Moreover, no neuron in the visual areas has ever been found to be sensitive to width disparity. Nevertheless, the possibility still remains that the signals of orientation disparity may contribute to the computation of surface orientation, as suggested by some psychophysical studies (Cagenello and Rogers 1993;Ninio 1985).
What is the functional role of SOS neurons in the parietal cortex? Neurological studies have shown that 3D constructional apraxia occurs in patients with right parietal lobe lesions (Critchley 1953; De Renzi 1982). These patients show abnormalities in assembling blocks according to a 3D model, and the drawings of these patients also show a characteristic lack of 3D perspective. Furthermore, an impairment of stereopsis has also been reported in patients with parietal lobe lesions (Holmes and Horrax 1919; Riddoch 1917; Rothstein and Sacks 1972). Recently, in a patient with severe damage to the ventral visual pathway, binocular viewing was found to be crucial for the grasping of objects as well as the matching of surface orientation in depth (Dijkerman et al. 1996). These clinical studies suggest that the parietal cortex plays a crucial role in linking depth perception to the visual control of hand action (Sakata and Taira 1994). Our previous studies on hand manipulation neurons in the anterior part of the lateral bank of the intraparietal sulcus (area AIP) demonstrated that many cells in this area were visually sensitive to the axis and surface orientation of objects as well as to their shape (Murata et al. 2000; Sakata et al. 1995; Taira et al. 1990). These neurons are likely to receive information regarding axis and surface orientation from area CIP neurons. Thus the SOS neurons are likely to provide viewer-centered representation of 3D surfaces for the manipulation of objects. It is also likely that SOS neurons represent an intermediate stage in the hierarchical processing of the stereoscopic system, before the final stage of representation of 3D shape in depth.
Part of this study was supported by a Grant-in-Aid for Scientific Research on Priority Areas (07244103, 06680817, 10680768) from the Ministry of Education, Science, Sports and Culture, a Grant-in-Aid for Target-Oriented Research and Development in Brain Science from the Science and Technology Agency, and a grant from the Hayao Nakayama Foundation for Science. MRI images of the monkey brains were taken at the Laboratory for Magnetic Resonance Imaging and Spectroscopy, National Institute for Physiological Science. We thank Solidray Co., Ltd. for help in developing the computer programs for our experiment.
MRI images of the monkey brains are available on the Web athttp://www.med.nihon-u.ac.jp/department/physiol1/
Address for reprint requests: M. Taira, First Department of Physiology, Nihon University, School of Medicine, 30-1 Ohyaguchi-kamimachi Itabashi, Tokyo 173-8610, Japan.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2000 The American Physiological Society