Performing a visually guided reach requires the ability to perceive the egocentric distance of a target in three-dimensional space. Previous studies have shown that the parietal reach region (PRR) encodes the two-dimensional location of frontoparallel targets in an eye-centered reference frame. To investigate how a reach target is represented in three dimensions, we recorded the spiking activity of PRR neurons from two rhesus macaques trained to fixate and perform memory reaches to targets at different depths. Reach and fixation targets were configured to explore whether neural activity directly reflects egocentric distance as the amplitude of the required motor command, which is the absolute depth of the target, or rather the relative depth of the target with reference to fixation depth. We show that planning activity in PRR represents the depth of the reach target as a function of disparity and fixation depth, the spatial parameters important for encoding the depth of a reach goal in an eye centered reference frame. The strength of modulation by disparity is maintained across fixation depth. Fixation depth gain modulates disparity tuning while preserving the location of peak tuning features in PRR neurons. The results show that individual PRR neurons code depth with respect to the fixation point, that is, in eye centered coordinates. However, because the activity is gain modulated by vergence angle, the absolute depth can be decoded from the population activity.
The parietal lobe plays an important role in spatial perception and movement coordination. Lesions in this area of cerebral cortex can result in deficits that range from hemifield neglect to diminished ability to execute goal-directed reaches (Karnath 1997; Perenin and Vighetto 1988). Visually guided reaches require an accurate estimate of the target location in three-dimensional space that necessitates the integration of depth cues. Damage to the parietal lobe can selectively disturb the representation of depth in the visuomotor control of reaches (Baylis and Baylis 2001). Movement planning activity in the parietal reach region (PRR) has been shown to encode the direction of an intended reach movement to eccentric targets on a single frontoparallel plane in an eye centered reference frame. Knowledge of the egocentric distance of the target is necessary to end the reach at the appropriate depth. It is not known whether or how neural activity in PRR reflects the egocentric distance to reach targets.
It is possible to represent egocentric distance by encoding the absolute depth of the target directly. This method of encoding target depth is independent of fixation depth. Alternatively, the egocentric distance of a target can be represented by the relative depth that is referenced to fixation depth, namely the disparity of the target. To fully specify egocentric distance in neural activity based on disparity, a representation of fixation depth, which is determined by vergence angle, is also required. Previous studies have shown that disparity tuning is gain modulated by vergence angle as early as V1 during stimulus presentation (Trotter et al. 1992) and during the planning activity for eye movements in depth in lateral intraparietal area (LIP) (Genovesio and Ferraina 2004; Gnadt and Mays 1995).
We examined whether PRR neurons encode the disparity of the reach target and vergence angle or instead encode the absolute depth of the reach target during movement planning. To observe the neural representation of movement plans, we studied spiking activity in the trial after cue offset and before the reach to the remembered target location (see Fig. 1A, planning period). Each neuron was tested with a memory reach task that required binocular fixation using separate reach and fixation targets that were modulated solely in depth. All reaches were initiated from the same location, and the position of both hands remained constant until the cue to execute the reach was presented. This ensured that neural activity was not modulated by the distance of the hands from the body but reflected changes in egocentric target distance only. Reach and fixation targets were arranged in two configurations (Fig. 1, B and C) to investigate how egocentric distance is encoded and to examine the tuning properties of PRR neurons as the target depth and vergence angle varied.
Memory reach task
Two rhesus macaques were trained to perform memory reaches with binocular fixation. The stimuli were presented in a dark chamber, and reach target and fixation stimuli were physically isolated from each other to prevent cross contamination of luminance. Trials were initiated when the hands were placed on two capacitive touch sensors located at hip level 10 cm in front of the face. Both hands were required to remain stationary until the cue to perform the reach movement was given. A fixation stimulus appeared after trial initiation, and the monkey was required to fixate this stimulus within a spherical window with 1° radius for the entire trial. The monkey maintained fixation on the fixation stimulus for all epochs, including during the reach movement and presentation of feedback after a successful reach. Trial epochs are shown in Fig. 1A. After 1,000–1,200 ms of fixation (fixation), the reach stimulus flashed for 300 ms (cue), and a memory period of 1,200–1,400 ms followed during which the monkey maintained hand position and gaze (planning). The cue to perform the reach movement was signaled by the dimming of the fixation stimulus to half its previous luminance (go). The monkey had 1,200 ms to perform a reach with the right hand to a clear touchscreen centered on and in front of the reach stimulus (reach). If the touch position occurred within a 2.5 cm radius around the target for 250 ms, the reach stimulus reappeared for 250 ms, during which the touch position had to be maintained (success). On success, a juice reward and auditory tone was delivered. Reach and fixation targets were arranged in two experimental configurations. In experiment 1, reach targets were coupled with fixation targets such that the same target disparities were sampled at different vergence angles (Fig. 1B, coupled reach targets; Fig. S11 ). Because coupled reach targets with crossed disparities had to be placed within the monkey's reach span, the distances of fixation stimuli were constrained. The disparity tuning of PRR neurons, the effect of vergence angle, and the interaction between disparity and vergence angle by gain modulation, and disparity-tuning shifts were explored with coupled reach targets. Reach targets in experiment 2 were fixed in space and were decoupled from fixation targets, so that all reach targets were tested with each fixation target (Fig. 1C, decoupled reach targets; Fig. S2). Decoupled reach targets tested different ranges of disparity at each vergence angle and hence did not constrain the depth of the fixation stimulus to be within the monkey's reach span. A fixation stimulus was placed at a “far” depth (1 m) in addition to the fixation stimuli in experiment 1, and many targets appeared highly diplopic due to their large disparity values at the different vergence angles. This configuration allowed us to examine the encoding of reach targets with constant absolute depth but with different disparities due to changes in vergence angle.
Fixation stimuli were red circular light-emitting diodes (LEDs, 2 mm diam), centered in azimuth and placed in elevation at eye level at 15, 20, 30, and 100 cm from the monkey's face (see Fig. 1, B and C). The interocular distance for both monkeys was 34 mm, thus the fixation stimuli corresponded to 13, 9.7, 6.5, and 1.9° of vergence angle. The reach stimulus was a square green LED (5 mm side). The reach stimulus and touch screen were mounted to a three-axis Cartesian robot (Samsung FARA RCM-4M, with SRC+ controller; Suwon City, Kyungki-Do, Korea). They were centered in azimuth, fixed in both azimuth and elevation, and modulated solely in depth at the distances designated in Fig. 1, B and C. Stimulus presentation was computer controlled, and touch sensor (Omron Electronics, Schaumburg, IL), touchscreen (Microtouch; 3M Touch Systems, Methuen, MA), and eye-position signals (CNC Engineering, Seattle, WA) were monitored in real-time using custom software written in Labview (National Instruments) and C++. The position of the reach stimulus was changed during intertrial intervals only. The time duration that the motors of the Cartesian robot were powered to change the position of the reach stimulus was kept constant to offset possible auditory cues from the manipulation of the depth of the reach stimulus. Reach and fixation stimuli were randomized in blocks of five trials. The experiments were performed in the dark except for the intertrial intervals between blocks when the lighting in the chamber was briefly flashed to prevent dark adaptation.
Animal preparation and recording procedure
Monkeys were implanted with head posts for reach training. Both monkeys performed reaches with the right limb. After initial training, the monkeys were implanted with scleral search coils (Baer Wire) in each eye to monitor the position of both eyes (Judge et al. 1980). Structural MRIs were performed to determine the stereotaxic location of the intraparietal sulcus for subsequent recording chamber placement (Fig. 1E). Surface normal recording chambers were implanted above the posterior parietal cortex to allow access to the medial wall of the intraparietal sulcus in the left hemispheres of both monkeys. Following the implant surgeries, the monkeys were trained to perform the memory reach task with binocular fixation. All surgical procedures were done under general anesthesia and in accordance with National Institutes of Health guidelines. The Caltech Institutional Animal Care and Use Committee approved all protocols.
Extracellular neural signals were recorded using a three-channel microdrive with quartz insulated platinum-tungsten electrodes with impedances of 1.0–2.5 mΩ (Thomas Recording, Giessen, Germany). Signals were preamplified in the microdrive by 20 times, an eight-pole low-pass filter (<20 kHz) by 7x, and then band-passed and amplified 50 times to isolate spiking neural activity (Plexon; Dallas, TX). Spike waveforms were viewed and sorted on-line (Sort Client; Plexon) and subsequently digitized and recorded to hard disk. Spike waveforms were inspected off-line and further sorted to isolate single units and assess stationarity (off-line sorter; Plexon). The signals from the capacitive touch sensors, the touchscreen, and horizontal and vertical positions of both eyes were recorded via the Plexon system onto hard disk. Preliminary single-unit recordings were conducted using a memory reach task to locate reach related, planning period activity in the recording chambers. Recording chambers were mapped to identify a region of interest based on the functional definition of PRR (Snyder et al. 1997), with the center located at ∼9P, 5.25L (Horsley-Clarke coordinates). The data from the recording chamber mapping procedure were not included for analysis in this study. All subsequent single-unit recordings made after chamber mapping were included in the analysis regardless of activation for the task or planning period.
The spike rates of 137 single units (90 from monkey G; 47 from monkey T) were analyzed from successful reach trials. Results were qualitatively the same from both monkeys and are presented jointly. Spike rates for the planning period were considered 200 ms after the reach stimulus cue offset to ensure that activity from visual processes related to the cue presentation, onset, and offset did not contaminate the planning period spike rate. Data collected for all units that were analyzed (n = 137) included all fixation and reach target configurations in both experiment 1 (n = 20 total configurations) and experiment 2 (n = 20 total configurations), with a minimum of five trials per condition. The disparities of coupled reach targets in experiment 1 ranged from −1.5 to +1.5° in 0.5° steps with respect to the fixation stimuli at 15 and 20 cm, and disparities ranged from −1.5 to +1.0° in 0.5° steps with respect to the fixation stimulus at 30 cm (n = 20 total configurations; see Fig. S1). For all analyses comparing disparity responses at two vergence angles, only disparity values that were sampled at both vergence angles were used in the calculations. The disparities of decoupled reach targets in experiment 2 had different values and ranges with respect to the fixation stimuli (n = 20 total configurations, see Fig. S2). Coupled and decoupled reach target disparities were considered separately in all but the following analyses: to assess whether cells were disparity sensitive overall and for the population disparity tuning shown in Fig. 3A (red bars), where disparities from both experiments were grouped by vergence angle and tested for significant modulation. Modulation by disparity or vergence angle was considered significant by ANOVA (P < 0.05), and yielded very similar results using Kruskal-Wallis in all tests (P < 0.05).
The disparity-tuning classification procedure was based on the classes previously described in V1 (Poggio 1995; Poggio and Fischer 1977). Neurons with peak features for zero disparity were classified in the “tuned-zero” category and included both excitatory and inhibitory tuning. Neurons with excitation for a large range of crossed disparities (−1.5 to −0.5°) or a peaked response (excitatory or inhibitory) for crossed disparity were classified as “near + tuned-near”. Neurons with excitation for a large range of uncrossed disparities (+0.5 to +1.5°) or a peaked response for uncrossed disparity were classified as “far + tuned-far”. Neurons with multimodal tuning, or broad tuning that included both crossed and uncrossed disparity were classified as “complex” (Trotter et al. 1996; Uka et al. 2000).
Disparity tuning index
Disparity tuning index (DTI) is a measure of modulation by disparity at a single level of vergence We calculated DTI for planning and cue period responses without subtracting baseline firing rates to compare it with the DTI observed in other cortical areas (Uka et al. 2000). The DTI was measured using coupled reach targets in experiment 1 and compared at different vergence angles across the population of neurons. We performed several nonparametric tests to detect differences in DTI across the entire population at different vergence angles in addition to those reported on the disparity tuned population. The Wilcoxon rank-sum test for independent population medians confirmed the results from Kruskal-Wallis test (P > 0.66), yielding the same medians between vergence angles (P > 0.36 for 13 and 9.7°, P > 0.53 for 13 and 6.5°, P > 0.87 for 9.7 and 6.5°). In addition, we tested whether there was a difference in the median of paired observations for each neuron (DTI of a neuron at 2 levels of vergence) with the Wilcoxon signed-rank test and found no significant differences (P > 0.57 for 13 and 9.7°, P > 0.09 for 13 and 6.5°, P > 0.14 for 9.7 and 6.5°).
Separability analysis and vergence gain model
The separability of neural responses to disparity and vergence angle was examined using singular value decomposition (SVD) analysis. A two-dimensional (2D) response matrix M was formed from the mean planning period responses across trials. A reconstruction of M was computed as N, which was the sum of an offset term α and a multiplicative component where S is a diagonal matrix containing singular values, and U and V are orthogonal matrices containing singular vectors. The offset α was found iteratively by computing the mean squared error between M and N for values of α that ranged from the minimum to maximum firing rates found in M in steps of 1 Hz (Mulliken et al. 2008; Pena and Konishi 2001). Values for α were highly correlated with the mean of M (ρ = 0.953). The fractional energy contained in the first singular value was computed as A permutation test was performed to determine whether neuronal responses M were significantly separable. Trials were randomly selected with replacement to form a resampled 2D matrix of mean responses, and the procedure in the preceding text was repeated. Separability for a neuron was considered significant (P < 0.05) if the first singular value (FSV) was >95% of resampled FSVs (Pesaran et al. 2006).
A gain model was employed to determine whether vergence acted on disparity as a linear operator. This was done at two levels of vergence, and one disparity-tuning curve was regressed onto the other using linear least squares where Y and X are disparity-tuning curves at different fixation depths, b is the multiplicative gain, and c is the additive gain by vergence angle. We performed a bootstrap analysis (Efron and Tibshirani 1993) where trials were selected with replacement to form resampled disparity-tuning curves. We used 500 resamples in the bootstrap analysis. Vergence gain was considered significant for a given pair of disparity-tuning functions if the bootstrap yielded a significant correlation coefficient (r value) from the regression (95% confidence interval using percentiles).
Absolute target depth and disparity modulation
Index A was a measure of the modulation of planning period firing rate in individual neurons where vergence angle and disparity varied for reach targets at constant absolute depth Index A was measured at three absolute target depths (15, 20, and 30 cm) and vergence angles (13, 9.7, and 6.5°). These three vergence angles created three different disparities for each reach target at a constant absolute depth. Figure S3 shows the vergence angle and reach target sampling for Index A, where three groups of targets with constant absolute depths but varying disparity and vergence angle are circled. Index A from the reach targets at the different depths were subsequently averaged for each neuron to obtain a single index for comparison to Index B. Index B was a measure of the modulation of planning period firing rate in individual neurons where vergence angle and the absolute depth of reach targets varied for targets at constant disparity Index B was measured using the same reach targets and vergence angles as Index A. The target and vergence angle configurations created three different reach target disparities (0, −3.2, and +3.2°), and the responses were averaged across disparity for each neuron. Figure S4 shows the vergence angle and reach target sampling for Index B, where three target groups with constant disparity are circled. The modulation was computed for each disparity group and then averaged across disparity groups to obtain a single value for each neuron.
Shifting responses were measured by cross correlating pairs of disparity-tuning curves from coupled reach targets in experiment 1 for individual neurons. The tuning curves were mean subtracted before computing the cross-correlation where CC(j) is the cross-correlation value at lag j, and X[n] and Y[n] are disparity-tuning curves from two different fixation depths. The maximum lag in the cross-correlation was j = ±3 (±1.5° disparity) to ensure that all cross correlations included at least three points. A permutation test was employed to determine the significance of cross correlations. Trials were shuffled, and disparity-tuning curves at each level of vergence were calculated from the shuffled data (preserving the number of trials for each condition obtained in the experimental session) and cross correlated. Shifting responses between disparity-tuning curves at different vergence angles were considered significant if they exceeded 95% of shuffled cross-correlation values (P < 0.05). A histogram of significant shift values from all pairs of disparity-tuning curves was calculated (Fig. 5B).
Simulated receptive fields
Spatial receptive fields were simulated using Gaussian functions where d is depth (cm), a is the height of the Gaussian peak, μ is the location of the peak in depth (cm), and σ is the width of the Gaussian function. The peak location and width for the functions were randomly selected from uniform distributions where the μ ranged from 15 to 35 cm in 1-cm increments, and σ ranged from 1 to 5 cm in 0.1-cm increments. The choice of a (peak height of the function) did not affect the computation. Samples of these Gaussian tuning functions were taken at the spatial locations corresponding to the disparity and vergence samples taken in experiment 1 for coupled reach targets. For example, Gaussian tuning functions were evaluated at the spatial locations of disparity samples [−1.5°:0.5°:1.5°] for coupled reach targets for 13° fixation at d = 13.4, 13.9, 14.4, 15, 15.6, 16.3, and 17 cm. These samples formed a set of simulated disparity-tuning curves from a neuron with a Gaussian-tuned spatial receptive field. These simulated disparity-tuning curves were then cross correlated to measure shifting responses in an identical manner as described in the preceding text. Similarly, significant shifts were obtained by permutation tests and compiled in a histogram. Simulations of populations of absolute depth encoding neurons (n = 500) were performed 100 times to estimate the histogram, with the average percentage of shifts (and SD) shown in Fig. 5D.
Figure 1D shows vergence eye position in a session from experiment 1 for the three fixation targets and all coupled reach target disparities. Vergence angle did not vary with the disparity of the reach target during movement planning (ANOVA; P > 0.33) or change due to presentation of the stimulus (ANOVA; P > 0.31). We compared the vergence eye position across the planning period for reach targets placed at zero and nonzero disparities to determine whether fixation depth changes were similar in magnitude to changes in reach target disparity. The smallest reach target disparity tested in all experiments was 0.5°, whereas the median of the difference in vergence eye position between zero and nonzero target disparities across sessions was 0.13°. Ninety-two percent of vergence eye positions during movement planning had a <0.33° difference due to nonzero target disparity, which did not change for fixation targets (ANOVA; P > 0.67). We looked at vergence eye position to see if the presentation of the cue to the reach target caused changes in fixation depth. Vergence eye position across sessions had a median difference of 0.09° before and after the presentation of the stimulus with 99% of all vergence eye positions with <0.33° difference. Although changes in vergence angle were much less than the smallest reach target disparity tested, we tested the possibility that they systematically varied with disparity and found a very low correlation (mean r2 = 0.02 across sessions). Last, we studied whether changes in reaction time could have a role in modulating planning period responses and found that reaction time did not systematically vary with disparity (mean r2 = 0.02) or target depth (mean r2 = 0.02). These behavioral data show that vergence angle and reaction times did not vary with the presentation of the reach target or its disparity to modulate the neural response.
Neural response to disparity
Figure 2A shows the firing rate response of a neuron at 13° of vergence angle for two coupled reach target disparities in experiment 1. There is a significant increase in firing during movement planning for a crossed disparity of −1.5° that is not present when the reach target is at 0° disparity. The neuron exhibits significant modulation and nonlinear tuning for the disparity of the reach target (ANOVA, main effect; P = 9.7e-6; for 13° vergence, P = 5.0e-3; for 9.7° vergence, P = 1.3e-2; Fig. 2B) during the planning period, which is a typical feature of disparity tuning in this neural population and in other cortical areas. Disparity is similarly encoded at two vergence angles and demonstrates a preference for near targets (crossed disparity). Coupled reach target disparities in experiment 1 significantly modulated planning period activity in 56% (76/137) of the neural population (n = 137), whereas the wide range of disparities included from both experiments significantly modulated 82% (112/137) of the population. The large number of sensitive neurons signifies the strength of modulation by disparity because reach targets were in a fixed location in azimuth and elevation. Because the frontoparallel location of both the fixation stimulus and reach target were fixed, neurons were not necessarily probed in the preferred 2D response fields for either the fixation or the reach target. Disparity sensitivity emerged in a larger proportion of the PRR population in experiment 1 for movement planning (56%, 76/137) than for the cue period during the presentation of the reach target (34%, 46/137; ANOVA P < 0.05). 67% (92/137) of PRR neurons had significant modulation during movement execution at a constant vergence angle; however, this modulation may also reflect motor-related signals such as motor efference or proprioceptive feedback. The shape of disparity tuning from the cue period was reflected in the planning and movement period responses in PRR neurons and suggests that information from visual responses during stimulus presentation influences neural activity during planning and executing the reach movement (see Fig. S5).
Figure 3A shows the distribution of significant disparity-tuning responses during movement planning across vergence angle for coupled reach targets (blue), decoupled reach targets (black), and both coupled and decoupled target disparities at the vergence angles in common (red). The proportion of disparity tuning in the population does not vary with fixation distance for coupled reach targets (Χ2; P > 0.8) or for the different ranges of disparity at each vergence angle for decoupled reach targets (Χ2; P > 0.6) or when including both (Χ2; P > 0.9). Taken together, the population response suggests that the network level planning activity in PRR is sensitive to reach target disparity evenly across fixation depth. The number of neurons with significant modulation for the small disparities tested in experiment 1 and for the large disparities in experiment 2 at the same vergence angle (independent of significant differences between small and large disparities tested between both experiments) was similar across vergence angle (n = 15/137 for 13° vergence, n = 17 for 9.7°, and n = 16 for 6.5°). Many neurons that were tuned to the large crossed disparities (−3.6 to −11°) of reach targets when viewing the farthest fixation stimulus (vergence angle = 1.9°; n = 45; ANOVA; P < 0.05) were also tuned to the small disparities tested in experiment 1 at other vergence angles (n = 30/45; ANOVA; P < 0.05). The sensitivity to both small and large disparities indicates that PRR plays a general role in planning movements to targets that may require either fine or coarse depth discrimination.
Disparity-tuning classification and modulation strength
Significantly tuned coupled reach target disparity responses (experiment 1) were used to classify disparity-sensitive neurons (76/137) subjectively into tuned-zero, near + tuned-near, far + tuned-far classes (see methods) and occurred with the same frequency (Χ2; P > 0.5). Neurons with multimodal or broad tuning that span the above categories were classified as complex (see e.g., Fig. S6). Figure 3B shows the distribution of classification, with a similar proportion of tunings across all categories (Χ2; P > 0.15). The large number of complex-tuned neurons shows that there exists a substantial nonclassical component in encoding disparity. This suggests that disparity tuning in PRR does not form discrete classes, which is a feature shared in other extrastriate cortical areas such as the middle temporal cortex and inferior temporal cortex (DeAngelis and Uka 2003; Uka et al. 2000).
Modulation by coupled reach target disparity at each vergence angle was calculated using the disparity tuning index (DTI, see methods), where DTI = 1 indicates a maximal response modulation occurs (with no spiking at the minimum response); DTI = 0 indicates that disparity does not modulate the response; and DTI = 0.33 indicates that the maximum disparity response is double the minimum response. The mean of the maximum planning period DTI from each neuron in the population (n = 137) is 0.4322 ± 0.2374 with a median DTI of 0.3638 and indicates that the modulation of planning period firing rate by target disparity was large. The DTI of disparity-sensitive neurons does not differ by the classification of the tuning (Kruskal-Wallis; P > 0.19). Based on the disparities at which the maximum and minimum responses were obtained at each vergence angle for the DTI, we find that the difference between these disparity values as a measure of disparity tuning width does not change with vergence angle across the PRR population (Kruskal-Wallis; P > 0.99). The histogram of planning period DTIs for all neurons at each level of vergence is shown in Fig. 3C, and the histogram of cue period DTIs is shown in Fig. S7A for comparison. Cue period and planning period DTI is correlated across the population (r = 0.75 across all vergence; r = 0.80 at 13°, r = 0.71 at 9.7°, and r = 0.74 at 6.5° vergence angle; all P < 1e-5; Fig. S7C), and although DTI is higher for the population during the cue period (Kruskal-Wallis; P = 0.0011), the average reduction in DTI is only 13% during the planning period.
The average planning period DTI of the population does not change with fixation depth (Kruskal-Wallis; P > 0.66), and testing for differences in the distributions of DTI across the population between different vergence angles shows that the DTI distribution is the same at each vergence angle (Komolgorov-Smirnov test; P > 0.54 for 13 and 9.7° vergence; P > 0.75 for 13 and 6.5°; P > 0.54 for 9.7 and 6.5°; Fig. 3C). There is an equal incidence of disparity tuning for coupled reach targets at each level of vergence (Fig. 3A, blue); however, this might arise if neurons were encoding the absolute depth of reach targets with a distribution of tuning across space. If neurons were directly encoding the absolute depth of reach targets in planning period firing rates, or the distance between reach and fixation targets, the response modulation measured by the planning period DTI should increase with fixation depth due to the larger distance between reach targets (e.g., intertarget distance changes by a factor of 3 between 13 and 6.5° of vergence angle). Instead the DTIs for the significantly tuned responses at different fixation depths reveal that the response modulation does not increase with larger intertarget distances (Kruskal-Wallis; P > 0.56; Wilcoxon rank sum P > 0.78 for DTI of tuned responses at 13 and 9.7°, P > 0.27 for 13 and 6.5°, P > 0.51 for 9.7 and 6.5°). In fact, the DTIs from different vergence angles are paired and shown for individual neurons in Fig. 3D and are correlated at different fixation depths across the population (r = 0.70 for 13 and 9.7° vergence; r = 0.76 for 13 and 6.5° vergence; r = 0.76 for 9.7 and 6.5° vergence; P < 1e-5 for all r). The constancy of DTI despite large changes in intertarget distance implies that PRR neurons do not directly reflect the absolute depth of the reach target or the distance between reach and fixation targets. The results suggest that the planning activity encodes disparity with similar strength across vergence angle.
Vergence angle and gain modulation
Downstream cortical areas can obtain an estimate of the egocentric distance of a reach target if a representation of vergence angle can be recovered in conjunction with target disparity from PRR neurons. Vergence angle significantly modulated planning period activity in 74% (101/137) of the population for coupled reach targets with constant disparity (ANOVA; P < 0.05). Fifty-eight percent (44/76) of neurons that were sensitive to coupled reach target disparity (n = 76) had a main effect of vergence angle (ANOVA; P < 0.05). A large proportion (52%, 32/61) of disparity insensitive neurons (61/137) was also significantly modulated by vergence angle (ANOVA; P < 0.05); these neurons directly represent fixation depth during planning. It remains possible that this disparity-insensitive population of neurons is sensitive to disparity as well as vergence in other frontoparallel locations (e.g., see Fig. 4, A and B). A vergence tuning index (VTI), similar to DTI, was based on response modulation by vergence angle for reach targets at a constant disparity and computed for PRR neurons. The mean of the maximum planning period VTI from each neuron in the population (n = 137) is 0.4326 ± 0.2348 with a median VTI of 0.3612 and is nearly identical to and correlated with the DTI (see Fig. S8). VTI is likely underestimated due to the fact that only three samples (13, 9.7, 6.5° vergence angle) were obtained at each disparity.
The nature of target disparity and vergence angle interaction was explored by examining the separability of the joint receptive field during planning. A separable receptive field suggests that the neural response encodes disparity and vergence angle independently by a multiplicative encoding mechanism. SVD analysis was used to decompose receptive fields into a basis set of singular vectors and compare the receptive field to a reconstruction made by the multiplication of the first singular vectors (see methods). Forty-six percent of disparity-sensitive neurons (35/76) separably encoded disparity and vergence angle (P < 0.05). One measure of the degree of separability is the fractional energy (FE) of the first singular value (FSV), and the distributions are shown in Fig. 4C, where the mean FE for all PRR neurons (n = 137) is 0.79 ± 0.1404, and 0.9160 ± 0.0425 for the separable, disparity-sensitive population (n = 35). The results indicate that a population of PRR neurons multiplicatively encode target disparity and vergence angle. To investigate specifically if vergence angle gain modulated disparity tuning, we employed a nonparametric approach of transforming disparity responses using vergence as a linear operator for each neuron. This method did not constrain disparity tuning to any functional form and instead regressed on disparity tuning curves from different fixation depths to produce a model of the disparity response with multiplicative and additive gain by vergence angle. Figure 2C shows the firing rate response for a neuron at 6.5° of vergence angle for a near and far coupled reach target in experiment 1, and Fig. 2D shows this neuron has “far” disparity tuning for reach planning with a significant multiplicative gain on disparity tuning by vergence angle (R2 = 0.48, P < 0.05; multiplicative gain b = 1.65 between 13 and 6.5° of vergence angle; see methods). Twenty-six percent (20/76) of disparity-sensitive neurons (experiment 1, n = 76) exhibit a significant gain model with an average R2 = 0.35 (σ = 0.1; P < 0.05) and average multiplicative gain of b = 1.77 (σ = 0.63; P < 0.05), and all gain modulated neurons had separable receptive fields (20/20). From the population of neurons that had significant disparity tuning at multiple vergence angles (experiment 1, n = 23), 65% (15/23) exhibit a gain model (P < 0.05) where the interaction between disparity tuning curves can be accounted for by vergence acting as a linear operator.
Absolute target depth and disparity modulation
In experiment 2, we explored whether PRR neurons directly encode absolute depth by using decoupled reach targets that were fixed in space but represented different values of disparity at each level of vergence. We would expect that an absolute depth encoding neuron would not change firing rate for a reach target at a constant absolute depth. We found that 70% (96/137) of neurons have a significant change in firing rate during movement planning, and 80% (110/137) during movement execution, for a reach target at the same absolute depth but at different disparities and fixation depths (ANOVA; P < 0.05). We examined the remaining population of neurons without modulation in movement planning (n = 41) to see whether these neurons encoded the absolute depth of the reach target. Forty-one percent (17/41) of this subpopulation was modulated by the absolute depth of the reach target at any level of vergence (ANOVA; P < 0.05). An absolute depth-encoding neuron would encode the absolute depth of the reach target independent of fixation depth; however, none of the neurons in this subpopulation (0/17) demonstrated target depth modulation across all vergence angles tested. We observed that for the nine neurons that were modulated at >1 vergence angle (n = 7 for 2 vergence angles, n = 2 for 3 vergence angles), most (6/9) came close to significant modulation (ANOVA P < 0.1) for a reach target at constant absolute depth and likely do not encode the absolute depth of the reach target. This suggests that PRR encodes target location in an eye centered reference frame, and this change in firing rate reflects both target disparity and eye position.
The disparity of the reach target, vergence angle, and the absolute depth of the reach target are related in three ways: A) disparity and vergence angle covary for a reach target at a constant absolute depth, B) absolute depth of the reach target and vergence angle covary for a target at a constant disparity, and C) disparity and absolute depth covary when vergence angle remains constant. The reach targets in experiment 1 were used to measure relation (B) as the VTI at six levels of disparity, and the last relation (C) as the DTI at three levels of vergence angle. In experiment 2, several decoupled reach targets were used to test the covarying relations 1 and 2 with the same vergence angles (13, 9.7, 6.5°) during reach planning. In other words, it is possible to compare the strength of modulation by vergence when target disparity changes (A) to when it remains constant (B). We measured the firing rate modulation for three decoupled reach targets that were fixed in depth but with changing disparity and vergence angle (A). This measured modulation by the same vergence angles for each decoupled reach target, however each target had different ranges of disparity (Fig. S3). Interestingly, the strength of modulation in PRR neurons when absolute target depth remained constant (A) is similar to when it varied (C, DTI). The median of the maximum modulation from the three targets with changing disparity and vergence angle was similar to the DTI for coupled reach targets when vergence angle was held constant (0.33 vs. 0.36). We subsequently averaged the modulation from the three decoupled reach targets for each neuron (Index A) to compare with other quantities.
In contrast to Index A, we used decoupled reach targets in experiment 2 to measure the change in planning period firing rate when target disparity remained constant (B). A measure of modulation due to changing reach target depth and vergence angle for targets at the same disparity was calculated (Fig. S4). This modulation was computed at 3 different disparity levels and averaged for each neuron (Index B). We subtracted Index B from Index A for each neuron to determine if planning period firing rate was influenced more by changing target disparity (A) or vergence angle (B). A value of zero indicates that the modulation by vergence angle is unaffected by changes in the disparity of the target; negative values indicate that vergence angle modulation has more influence than disparity, whereas positive values indicate that changes in disparity affect planning period firing rate more than changes in vergence angle. Overall, 65% (89/137) of neurons have a difference between Indices B and A that is >0. The distribution of the difference between the Indices A and B for all neurons is unimodal and concentrated to the right of zero (Fig. S9) with a population average indicating a larger Index A than Index B. If the encoding of egocentric distance in an eye centered reference frame is achieved with divergent populations of vergence and disparity encoding neurons, we would expect a bimodal distribution with peaks at negative and positive differences corresponding to the two populations. PRR contains a homogenous gradient where the average response is influenced more by disparity than fixation depth. In summary, most PRR neurons have changes in planning activity for a reach target at a constant absolute depth due to different target disparities and vergence angles. The strength of modulation during reach planning was greater by disparity and vergence angle for a decoupled reach target fixed in depth (A) than by targets at different absolute depths (B; Wilcoxon signed-rank test, P = 5.86e-6).
Disparity tuning shifts
A shift in disparity tuning due to vergence angle is a nonseparable and nonlinear interaction that is exclusive of a gain model. We examined whether PRR neurons exhibited disparity tuning shifts from one level of vergence to another. Shifting responses were measured using cross-correlations for coupled reach target disparities, where neurons with a significant cross-correlation between a pair of disparity tuning curves (3 possible pairings/shifts) were considered to have measurable shift(s) in disparity tuning by vergence angle. We identified 120 neurons which had at least one shift in disparity tuning, with a total of 219 shifts in the population. The most frequent significant shift in the population was 0°, indicating that disparity tuning profiles at different vergence angles were precisely aligned (Fig. 5B). The distribution of significant shifts from the disparity-sensitive population (n = 76, experiment 1) is similar to that for the entire population shown in Fig. 5B (Komolgorov-Smirnov test; P = 0.88). Figure 5A shows the planning period response of a neuron with a significant cross-correlation of shift 0° for tuned-zero disparity responses at 13 and 9.7° of vergence angle. Fifty-eight percent (128/219) of all shifts were in the range of ±0.5°, whereas large shifts of ±1.5° in disparity tuning were less frequent (<20% of shifts). We examined the disparity-sensitive (P < 0.05), nonseparable (P > 0.05) encoding population of neurons (n = 41) and found that 88% of these neurons (36/41) had nonzero shifts in disparity tuning that accounted for their receptive fields being nonseparable. The average value from the unimodal distribution of all shifts (n = 219) was −0.03 ± 0.87°. These results suggest that modulation by vergence angle often preserves the location of peak features in the disparity tuning of PRR neurons across fixation depth.
We compared the PRR shifting responses to shifts that would be observed in a hypothetical population of absolute depth encoding neurons. It has been proposed that a network of absolute depth tuned neurons with Gaussian spatial receptive fields could serve to encode the egocentric distance of reach targets (Pouget and Sejnowski 1994). Coupled reach target disparities in experiment 1 probed neural responses in adjacent, nonoverlapping regions of space (Fig. 1B). A neuron encoding absolute target depth has a receptive field corresponding to a region of space instead of disparity. Figure 5C (top) shows an example of a simulated Gaussian spatial receptive field, where the disparity tuning curves that would have been sampled from it using coupled reach targets are shown in different colors and plotted as a function of absolute depth. The same simulated responses from different vergence angles are paired and plotted as a function of disparity in Fig. 5C (bottom). These disparity tuning curves were cross correlated to measure shifts by vergence, which resulted in a single significant shifting response of +1.5° between disparity tuning at 9.7 and 6.5° of vergence angle (Fig. 5C, bottom far right). We simulated receptive fields in space for populations of absolute depth encoding neurons using Gaussian functions with different means and widths. Figure 5D shows a histogram of the proportions of disparity tuning shifts observed in these simulated populations. The vast majority (>75%) of simulated absolute depth encoding neurons exhibit shifts that have a large value of ±1.5°. The large difference in shift distributions between the simulated neurons and those obtained from PRR neurons indicates that the PRR population does not encode absolute target depth (Komolgorov-Smirnov; P = 1.92e-4). The distribution of shifts shows most PRR neurons exhibit the alignment of disparity tuning consistent with an eye centered reference frame that is gain modulated by vergence angle.
The alignment of disparity tuning curves and gain modulation by vergence angle during movement preparation illustrates that neural activity encodes egocentric distance and implicates PRR in the early stages of reach planning. Psychophysical studies on human depth perception are more consistent with the encoding of egocentric distance using a network model with units tuned to vergence angle and target disparity than a model with units tuned to the absolute distance of targets (Pouget and Sejnowski 1994). The processes underlying visual perception and action in visually guided reach movements require the coordination of information in multiple reference frames across the cerebral cortex. Our perceptual experience when scanning a visual scene and selecting a goal from multiple targets may occur in a world- or object-centered reference frame, where target distances are referenced to each other or a landmark in space. Once a goal is selected, its spatial parameters must be estimated with reference to the subject to prepare and execute a movement. The egocentric distance of the goal is represented in many ways throughout the reaching network at various stages of processing. The egocentric distance could represent the distance of the target from the hands in a limb centered reference frame in cortical areas that are involved in the final stages of processing. The distribution of the preferred directions of neurons in the motor cortex during the execution of arm movements in 3D space contain an enhanced representation for the forward and backward directions and shows a specialization in motor control for reaches in depth (Naselaris et al. 2006). Stimulated ensembles of neurons in primary motor cortex encode arm postures that can place the hand at proximal or distal locations from the trunk that are suited for reaching to targets in depth (Graziano 2006). Intermediate stages in processing use mixtures of limb and eye centered reference frames, such as in area 5 in the posterior parietal cortex (Buneo et al. 2002) and in dorsal premotor cortex (Pesaran et al. 2006). We have shown that reach plans encoded in PRR contain a representation of egocentric distance that is eye centered, modulated by vergence angle, and reflect the initial processing stages in goal directed reaching in depth.
Spiking activity in PRR neurons represents egocentric distance by encoding the disparity of the reach target and fixation depth in movement plans. PRR neurons often have a visual response during stimulus presentation modulated by target disparity and vergence eye position that is likely to be strongly associated with the activity in the visual cortices through feedforward and feedback connections [34% (46/137) disparity sensitive in experiment 1; 44% (60/137) vergence main effect; ANOVA P < 0.05]. In this study, we examined neural activity after the visual response from the stimulus and before movement execution to assess whether PRR neurons represented the depth of the target in the formation of movement plans. These signals emerge during movement preparation more frequently than during the presentation of the reach target stimulus [34% (46/137) are disparity sensitive during stimulus presentation versus 56% (76/137) during planning]. The shapes of significantly tuned disparity tuning profiles during movement planning in PRR are diverse, and a number of functional forms (e.g., linear, Gaussian, sine, gabor, polynomials ≤3rd degree, etc.) were explored to fit the disparity responses in the population and found to be insufficient (R2 > 0.2; data not shown) in summarizing the disparity tuning shapes for a significant number of neurons (n > 15). The large number of neurons in the complex-tuned group in this data indicates that a coarser approach of subjective disparity response classification into previously described discrete classes does not play a significant role in separating the functions of PRR neurons. It remains possible that a finer sampling of disparity could reveal a functional form that models the disparity tuning as well as reveal a role for discrete classes of disparity tuning. Reach target disparity tuning during the cue, planning, and movement epochs from experiment 1 were similar; however, it is difficult to distinguish disparity responses during the movement because they are confounded by motor-related signals from motor efference and/or proprioception. The design of reach and fixation target configurations in experiment 2 enabled the testing of the motor and visual components separately during movement execution by keeping the motor component (target depth) constant and varying the visual component (target disparity and vergence angle). Neural activity during the execution of the reach movement is strongly modulated by vergence angle for reaches to targets at constant absolute depth in 80% (110/137) of the population (ANOVA P < 0.05) and indicates that PRR neurons do not represent pure proprioceptive or motor commands (efference or feedback), but a visuomotor representation of hand guidance.
Fixation depth is recoverable directly from vergence angle and is a veridical depth cue, influencing neural activity in the oculomotor pathways extending from the midbrain through neocortex (Akao et al. 2005; Judge and Cumming 1986; Kurkin et al. 2003) as well as visual cortical areas beginning at the level of primary visual cortex (Rosenbluth and Allman 2002; Trotter et al. 1992). It has been shown that neurons in Area 7a have modulation for azimuth, elevation, and the depth of a fixation stimulus, and indicate 3D eye position during fixation (Sakata et al. 1980). Vergence angle strongly affected the response of the majority of PRR neurons for coupled reach targets at constant disparity during movement planning; however, because the fixation target was fixed in azimuth and elevation, it is likely that the optimal response to vergence was not examined for many of the neurons. In addition, the sparse sampling of vergence angle taken at constant target disparity for experiment 1 probably also underestimates the degree of modulation by vergence angle (VTI) in the population. The DTI of the population, however, does not change with fixation depth and is well correlated between vergence angles, indicating that the strength of modulation by disparity is preserved across fixation depth in the population activity. The DTI of PRR neurons during movement planning is notably similar to that seen in the visual responses to various stereoscopic stimuli in area V1, IT, and V4 (see Table S10) (Hinkle and Connor 2005; Uka et al. 2000). Most PRR neurons were sensitive to both disparity and vergence angle, and the correlation between DTI and VTI in experiment 1 and the unimodal distribution of (Index A –Index B) from experiment 2 indicate the PRR population is not subdivided into discrete and separate subpopulations of vergence- and disparity-sensitive neurons.
Previous studies have found separable encoding of 2D frontoparallel eye and target positions in PRR during movement planning (Buneo and Andersen 2006; Pesaran et al. 2006) and separable spatiotemporal encoding in the PPC during movement execution (Mulliken et al. 2008). A population of disparity-sensitive neurons was found to separably encode disparity and vergence in PRR during planning. These separable encoding neurons contained significant fractional energy (>0.90) in their SVDs; this suggests that the receptive field was constructed from a multiplicative response to disparity and vergence angle. We examined multiplicative responses further by modeling the multiplicative and additive gain modulation on disparity by vergence angle explicitly between pairs of disparity tuning curves. Vergence angle was sampled in >3° steps, and a population of neurons exhibited gain modulation of disparity tuning by vergence angle acting as a linear operator. More neurons may express this relationship for smaller changes in vergence than those tested in this study. In addition, because vergence angle and disparity tuning was not necessarily probed at the optimal azimuth and elevation for either fixation or reach target, the number of neurons exhibiting linear gain modulation is potentially underestimated. Nonlinear gain mechanisms based on fixation depth may play a role in transforming disparity responses as well. Two-dimensional limb position gain fields have been found in reach related areas in the posterior parietal cortex (PPC) that underlie the coordinate transformation process. Initial hand location on a frontoparallel plane is encoded in an eye centered reference frame in PRR (Buneo et al. 2002), and it is likely that a similar gain mechanism may exist to encode the initial egocentric distance of the hand during movement planning. If so, the population activity in PRR could contain more than the goal of a reach in planning activity, but a complete representation of a movement plan from initiation to target acquisition for reaches in three dimensions in visual coordinates.
Previous studies have shown the influence of depth cues during visuospatial tasks on neural activity throughout the intraparietal sulcus (IPS) (Sakata et al. 1997). For example, neurons in the anterior region of the lateral bank of the IPS, area AIP, have been shown to have a role in grasping and vary their response to the presentation of 3D disparity defined shape of target objects of manipulation (Murata et al. 1996). The caudal region of the IPS, area CIP, has been shown to be sensitive to different surface orientations in depth as prescribed by diverse depth cues such as binocular disparity, texture, and linear perspective (Sakata et al. 2005). Area LIP has been shown to have modulation by stimulus blur, vergence angle, and binocular disparity during the planning of eye movements in depth (Genovesio and Ferraina 2004; Gnadt and Mays 1995). Human subjects have the ability to perceive and estimate the depth of targets with large disparities, which is necessary for performing reaches without foveating the target (or fixating at the same distance as the target) and for planning eye movements to locations far outside of the plane of fixation (Westheimer and Tanzman 1956; Ziegler and Hess 1997). LIP neurons support large vergence eye movements in behavior with sensitivity to large disparities (ranging from −7 to +10°) during saccade planning (Genovesio and Ferraina 2004; Gnadt and Mays 1995).
Reaches can be performed with steady decoupled fixation throughout the movement, or more commonly planned with decoupled fixation to first bring the hand to the “ballpark” of the object before it is foveated and grasped (Heath and Binsted 2007; Johansson et al. 2001). Both scenarios initially require the representation of the depth of a reach target that is highly decoupled or “far” from fixation depth to plan the reach movement. The majority of PRR neurons (69%, 95/137) that were tested with decoupled reach targets at constant vergence angle in experiment 2 during reach planning are sensitive to large target disparities. The functional similarity between PRR and LIP responses (Snyder et al. 2000) and anatomical connectivity (Lewis and Van Essen 2000) suggests that a parallel neural architecture evolved in both areas to coordinate saccades and reaches in the same reference frame. Damage to the parietal cortex is known to cause limb specific deficits in decoupling a reach movement from gaze (Jackson et al. 2005). In experiment 2, 40% (45/112) of the disparity-sensitive population (including target disparities from both experiments 1 and 2) encoded the disparity of highly decoupled targets (experiment 2) when fixation depth was 1 m. These results show that movement planning activity in PRR contains a neural correlate of decoupled reaching that supports hand-eye coordination.
Damage to the parietal cortices can create specific depth related visuomotor deficits that cause errors in visually guided reaching movements. Baylis et al. tested a patient with bilateral parietal lobe damage using a delayed reach task with targets at different azimuthal locations and depths (Baylis and Baylis 2001). The patient exhibited significantly more errors in depth than direction during visually guided reaching with either arm. This depth specific deficit in reaching vanished when reaches were guided without vision based on verbal instructions. These findings suggest that damage to the parietal lobe can disturb the representation of depth specifically for reaches planned using vision. Visual form agnosia patient D.F. was tested with a reach to grasp and perceptual distance estimation task in depth (Carey et al. 1998). D.F.'s performance in reaching to targets in depth under visual guidance was indistinguishable from normal subjects; however, D.F.'s verbal estimates of target depth exhibited an above normal error rate, suggesting that the neural mechanism for visuomotor control in depth was intact and separate from those required in making perceptual judgments of depth. This may reflect the fact that the neural substrates for perceptual processes that use a world or object centered frame of reference to estimate target depth are disturbed. Taken together, the implications of these studies are that the parietal cortex is essential in the coordinate transformations that create early movement plans in an eye centered reference frame used in planning reaches in depth. The integration of depth cues central to representing the egocentric distance of a target place PRR at the early foundations of planning and making reaches in a 3D world.
This work was funded by National Institutes of Health and Office of Naval Research.
We thank G. Mulliken, E. Hwang, and H. Cui, for helpful comments on the manuscript; Z. Nadasdy for useful discussions; K. Pejsa, N. Sammons, L. Martel, J. Baer, and C. Lindsell for help with animal handling and veterinary assistance; V. Shcherbatyuk for computer support; T. Yao for administrative assistance; and R. Panagua and M. Walsh for laboratory equipment construction. We also thank the reviewers for comments and suggestions.
↵1 The online version of this article contains supplemental data.
- Copyright © 2009 the American Physiological Society
- Akao 2005.↵
- Batista 1999.
- Baylis 2001.↵
- Buneo 2006.↵
- Buneo 2002.↵
- Carey 1998.↵
- DeAngelis 2003.↵
- Efron 1993.↵
- Genovesio 2004.↵
- Gnadt 1995.↵
- Graziano 2006.↵
- Heath 2007.↵
- Hinkle 2005.↵
- Jackson 2005.↵
- Johansson 2001.↵
- Judge 1986.↵
- Judge 1980.↵
- Karnath 1997.↵
- Kurkin 2003.↵
- Lewis 2000.↵
- Mulliken 2008.↵
- Murata 1996.↵
- Naselaris 2006.↵
- Pena 2001.↵
- Perenin 1988.↵
- Pesaran 2006.↵
- Poggio 1995.↵
- Poggio 1977.↵
- Pouget 1994.↵
- Rosenbluth 2002.↵
- Sakata 1980.↵
- Sakata 1997.↵
- Sakata 2005.↵
- Snyder 1997.↵
- Snyder 2000.↵
- Trotter 1992.↵
- Trotter 1996.↵
- Uka 2000.↵
- Westheimer 1956.↵
- Ziegler 1997.↵