Recently, using event-related functional MRI (fMRI), we located a bilateral region in the human posterior parietal cortex (retIPS) that topographically represents and updates targets for saccades and pointing movements in eye-centered coordinates. To generate movements, this spatial information must be integrated with the selected effector. We now tested whether the activation in retIPS is dependent on the hand selected. Using 4-T fMRI, we compared the activation produced by movements, using either eyes or the left or right hand, to targets presented either leftward or rightward of central fixation. The majority of the regions activated during saccades were also activated during pointing movements, including occipital, posterior parietal, and premotor cortex. The topographic retIPS region was activated more strongly for saccades than for pointing. The activation associated with pointing was significantly greater when pointing with the unseen hand to targets ipsilateral to the hand. For example, although there was activation in the left retIPS when pointing to targets on the right with the left hand, the activation was significantly greater when using the right hand. The mirror symmetric effect was observed in the right retIPS. Similar hand preferences were observed in a nearby anterior occipital region. This effector specificity is consistent with previous clinical and behavioral studies showing that each hand is more effective in directing movements to targets in ipsilateral visual space. We conclude that not only do these regions code target location, but they also appear to integrate target selection with effector selection.
To plan an action, the brain needs to select the object to act on and choose the optimal effectors (i.e., body parts) to perform the action. Most studies have focused on the first issue: understanding the mechanisms underlying target selection (see e.g., Schall 2003 for review). However, to generate a movement, the location of the target needs to be integrated with the selection of an effector (Buneo et al. 2002; Mascaro et al. 2003). For example, to pick up a target object, say, a cup of coffee, we could use either the right or the left hand. How does the brain decide on how to respond to a particular target? Presumably, a critical factor in this decision-making process is some assessment of the cost of the movement by each effector to the selected target—a cost that depends on the relative position of the effector to the target. For example, the kinematics of human reaching movements are dependent on whether the effector is contra- or ipsilateral to the targets (Fisk and Goodale 1985). In the example in the preceding text, the brain may choose to move the hand closest to the cup. Of course, additional factors like handedness (accuracy demands) and effector availability likely play a role in effector selection.
Recent studies have implicated both the posterior parietal cortex (PPC) and the premotor areas in target-specific effector selection. That is, when planning a forthcoming action, both regions have been shown to encode activity related to integrating information about target position and the end effector (Buneo et al. 2002; Carlton et al. 2002; Dickinson et al. 2003; Hoshi and Tanji 2000; Kertzman et al. 1997). This is supported by clinical studies in patients with unilateral lesions in the superior parietal lobe (Perenin and Vighetto 1988; see Battaglia-Mayer and Caminiti 2002 for review) that show that the deficits in producing reaching movements are dependent on both target and hand position.
A significant complication to understanding these processes is that in the respective early sensory areas, the position of the target and the position of the effector are initially coded in different reference frames. Proprioceptive information about the position of end effectors (body parts) is initially encoded along the posterior bank of the postcentral gyrus, somatotopically as relative joint angles. In contrast, there is good evidence in the monkey (Batista et al. 1999) and human (see following text) that target location is coded in eye-centered (retinal) coordinates in the early stages of movement planning. This might require visual information to be transformed into body coordinates before comparing it to effector information (Flanders et al. 1992; McIntyre et al. 1997). However, a recently proposed alternative is that this comparison is done at an early stage of visuomotor processing, within the posterior parietal cortex, in eye-centered coordinates (Buneo et al. 2002). This scheme would require that initial hand position, as derived from proprioceptive information, be transformed “backward” into eye coordinates, using eye position and other signals. This simplifies the reference frame aspect of the problem by placing early spatial information about the target in the same frame as the effector, but the question remains: how does the human PPC integrate spatial information for the purpose of effector selection?
Recently, using event-related functional magnetic resonance imaging (fMRI), a bilateral region in the human posterior parietal cortex was located that shows a contralateral left-right topography for both saccade and pointing targets and updates this information in eye-centered coordinates whenever the eye moves (Medendorp et al. 2003a, Merriam et al. 2003; Sereno et al. 2001). In the present study, we examined, using fMRI, whether the activation of this region (which we will refer to as retinotopic IPS, or retIPS) is dependent on how it is used by different effector systems (eye/left hand, right hand) and compared the results to other regions involved in movement control. Six subjects were instructed to fixate centrally and to point with either the right or left hand toward remembered peripheral targets in either the right or left visual field. Regions showing activation related to both the effector used and the visual field pointed to likely integrate spatial and effector information in planning hand action.
Subjects and ethics approval
Details about the setup and methods as well as the general paradigm have been described in Medendorp et al. (2003a). All procedures were approved by the York University Human Participants subcommittee and the University of Western Ontario Ethics Review Board. All subjects gave informed consent to participate in the experiments. Pilot testing was performed on four subjects. After our test paradigms were finalized, six subjects were tested. The results from these subjects (1 female), aged between 20 and 33 yr, are documented in the present report. Each subject extensively practiced all tasks before imaging to ensure that the tasks were performed correctly. Moreover, kinematic recordings were taken to confirm correct behavior, as described in the following text.
MRI scanning and data analysis
Data were collected with a 4.0 Tesla Varian Siemens whole-body imaging system. Stimuli were presented using a NEC VT540 LCD projector (refresh rate: 70 Hz) with custom optics projected onto the ceiling of the magnet bore. All subjects were right-handed. Pointing movements were made using either the right or the left hand. During the experiments, 17 contiguous slices were used to image the entire parietal cortex using a quadrature RF surface coil centered on the posterior parietal lobe. The location and orientation of these slices are shown in Fig. 1, which assured that the functional volume coincided with the parietal regions of interest and also covered parts of occipital and frontal cortex. Functional data were obtained using navigator echo corrected T2*-weighted segmented gradient echoplanar imaging (TE = 15 ms; FA = 45°; FOV = 19.2 × 19.2 cm; TR = 2 s; in-plane pixel size = 3 × 3 mm; THK = 4 mm). Functional data were superimposed on high-resolution inversion prepared three-dimensional T1-weighted anatomical images of the brain (typically 128 slices, 256 × 256, FOV = 19.2 × 19.2 cm, TE = 5.5 ms, TR = 10.0 ms) using a phase reference image that corrected for high-field geometric distortions. In separate sessions, subjects were rescanned using a birdcage-style head coil to obtain full brain anatomical images. A high-resolution inversion prepared three-dimensional (3D) T1-weighted sequence was used (FA = 15°; voxel size: 1.0 mm in-plane, 256 × 256, 164 slices, TR = 0.76 s; TE = 5.3 ms). Analysis was performed using Brain Voyager 4.8 software (Brain Innovation, Maastricht, The Netherlands) and Matlab software (The Mathworks). Surface coil images were aligned manually to head-coil images. Anatomical images for each subject were segmented at the gray/white matter boundary, rendered and inflated for visualization purposes only. For functional data analysis, we excluded any scans in which motion artifacts were observed. Time courses within each voxel were corrected for linear drift. Anatomical and functional images were transformed to Talairach space to obtain coordinates for the regions of interest (Talairach and Tournoux 1988).
During the experiments, subjects fixated on a central letter, S, L, R or F, and were instructed to make a delayed-saccade (S), a delayed right-hand pointing movement (R), or a delayed left-hand pointing movement (L) or to maintain central fixation (F) (see Fig. 2) (see also Batista et al. 1999; Sereno et al. 2001; Snyder et al. 1997). Then a brief peripheral dot was presented for 250 ms either to the left or right at random horizontal eccentricities from the continuous interval between 10 and 25°. This target location was not presented in fixation (F) mode. Subsequently, a band of distractors (70° horizontal × 8° vertical, dot's eccentricity: 0.8°, density: 0.14 dots/deg2) blinked (at 5 Hz) for 2.5 s during which the subjects maintained central fixation (and pointed to the central letter when in the pointing mode). Then at distractor offset, 3 s after target presentation, subjects made either a saccade or a pointing movement to the remembered target location and immediately returned to center. Subjects made no movement in the fixation (F) task. During the pointing task, they were instructed to maintain central fixation of the eyes at all times. Pointing movements consisted of wrist rotations such that the index finger pointed to the remembered target location (DeSouza et al. 2000). During pointing, each hand was held a few centimeters ipsilateral to the body midline. The subject's view of the hand was occluded with black cardboard during pointing (DeSouza et al. 2000). Thus the visual input during the saccade and pointing tasks was the same. The time between successive movements was 5 s.
We incorporated this task in a block-design paradigm so that within blocks the memory delay interval was maximized and the motor delay interval was minimized. Scans to determine the movement-related activation maps comprised 17 blocks (each 20 s) in which either saccade and right-hand pointing blocks (each including movements to four different target locations) or right- and left-hand pointing blocks were alternated with fixation blocks. Scans for contralateral topography comprised 12 blocks (each 20 s): first one block of fixation, then 10 blocks in which four leftward targets were alternated with four rightward targets; and finally a fixation block that concluded the scan. Typically, three to four scans for each task were obtained, and these scans were averaged to improve the signal-to-noise level. Data were analyzed using a standard general linear model (GLM) with separate predictors for each state, convolved with the hemodynamic response function (modeled using a gamma function with a tau of 2.5 s and a delta of 1.5 s) and contrasts between the predictors to determine activated and topographic (left vs. right) voxels along the dorsal stream networks. GLM data could be analyzed across the whole group (in stereotaxic space) or for individual subjects. To further quantify our results, we computed the BOLD percentage signal change in each of our regions of interest for each condition (saccades, left hand, or right hand). This was computed by taking the average of the signal at the last half of each block (which excluded the hemodynamic rise and fall) relative to the defined baseline. These were then averaged across all blocks and runs within a subject and then across subjects.
During imaging, we did not record either eye or hand movements. In three subjects, however, we did record these movements outside the scanner. In this behavioral experiment, eye movements were recorded at 250 Hz with an Eyelink II gazetracker (SR Research); finger and wrist movements were measured at 100 Hz using an OPTOTRAK 3020 system (Northern Digital). The head was stabilized, and the eyes were at a distance of 25 cm from the stimulus presentation screen. We used the same set of stimuli as in the fMRI recordings.
Figure 3 shows that fixation was maintained while generating left and right hand pointing movements in one subject. The same was found in all subjects; on average fixation was broken in <3% of the trials. The figure also shows some variation in the amplitude of the generated pointing movement. Pointing amplitude correlated well with the eccentricity of the location of memorized target (r > 0.88 for all subjects). The same was found in eye movement trials (r > 0.91 for all subjects). This indicates that subjects did not just generically make an eye or pointing movement to the left or to the right without regard for amplitude. On average, saccade and pointing accuracy were 4.9° and 2.2°, respectively. Together, our behavioral data confirm that subjects were following the instructions and were able to keep fixation while making pointing movements (see also Medendorp et al. 2003a).
We first identified regions of the cortex that were activated when making movements of the eye or right hand to targets presented in either visual hemifield when compared with the fixation control. Figure 4 (left) provides an overview of these results in data based on Talairach averaged group results (P < 0.01), which for clarity were rendered onto an inflated representation of the cortical surface of one subject. The majority of the regions that were activated during saccades were also activated during pointing movements, including occipital, posterior parietal, and prefrontal cortex. The regions within the left postcentral sulcus, corresponding to the primary somatosensory cortex, and within the left central sulcus, corresponding to primary motor cortex (M1) (Picard and Strick 2001; Yousry et al. 1997), were exclusively activated by movements of the right hand.
Figure 4 (right) provides an overview of the areas that were hand-selective. As expected, the regions that were more active for hand movements than for eye movements (somatosensory and motor cortex) were also clearly selective for which hand was used. Furthermore, as one moved more posteriorly along the IPS, activation for left and right hand became co-mingled, as was the case in areas more anterior to the central sulcus, i.e., within the precentral gyrus and sulcus, which correspond to the dorsal premotor areas (PMd) (Picard and Strick 2001; Wise et al. 1997). Hand-specific activations were also observed in the higher visual areas of occipital cortex.
Contralateral topography of target representation
We next focused our analysis on those regions that showed a preference for remembered targets in either the left or right visual hemifield, as in Medendorp et al. (2003a). The results of this analysis in two subjects are depicted in Fig. 5. Yellow regions indicate a stronger activation for remembered target locations in the left visual field than in the right, whereas blue voxels represent the opposite pattern. As reported previously (Koyama et al. 2004; Medendorp et al. 2003a; Sereno et al. 2001), this analysis identified a bilateral region in the human IPS that shows contralateral topography.
More precisely, this region was located at a medial branch of the intraparietal sulcus. Mean Talairach coordinates (x, y, z) across six subjects for this region were –25, −59, 41 for the left IPS and 20, −58, 42 for the right. These coordinates are consistent with previous studies (Koyama et al. 2004; Medendorp et al. 2003a; Sereno et al. 2001). We will refer to this region as retIPS. In addition, for most subjects we also found analogous bilateral topographical regions in frontal and occipital cortex. The topographical area in the frontal cortex (left: −31, −10, 52; right: 26, −11, 49) was close to the intersection of the superior frontal and precentral sulcus and may correspond to the human frontal eye fields (FEF) (Astafiev et al. 2003; Bermann et al. 1999; Brown et al. 2004; Corbetta et al. 1998; Koyama et al. 2004; Paus 1996). The region in the occipital cortex (left: −26, –79, 20; right: 28, −76, 20) was located anteriorly to V3A, perhaps overlapping V3A's superior half as well as V7 (Tootell et al. 1998), in agreement with the findings of Sereno et al. (2001). It is this region that showed activation during pointing movements of the hand (Fig. 4) and during saccades. We will refer to this region as anterior occipital cortex (aOC).
Thus besides our earlier reported region retIPS, we found other regions in the brain that were co-activated during our movements tasks, some of which have topographical representations of remembered target location, like aOC and FEF, some of which have not, like M1 and PMd. Across subjects, the average Talairach coordinates (in mm), the extent of the activation maps (in mm3) and their corresponding statistical thresholds (t-value) of all five regions are presented in Table 1. The following provides a more quantitative analysis of the degree of left-right topography in our regions of interest.
The degree of contralateral visual field preference for memory-guided eye movements (Figs. 4 and 5) is quantified in Fig. 6 for the brain areas that were identified as having left-right topography in our preliminary analysis. Here the percentage BOLD signal changes for targets in the left and right visual hemifield are compared. These data show that the contralateral bias that is present in bilateral retIPS is also observed in FEF and aOC (paired t-test, P < 0.05). Figure 6 demonstrates again that M1 is not involved in eye movements (see also Fig. 4). Furthermore, the PMd region is activated in relation to eye movements, although its activity is not spatially tuned as indicated by the lack of laterality (paired t-test, P > 0.05). Because the quantitative contralateral bias for the hand may be affected by which hand is used, we next explored the effector specificity within the topographically organized regions identified.
How effector-specific are these topographic regions? A comparison of Figs. 4 and 5 shows that the topographical regions retIPS, aOC and FEF were activated by movements of all three effectors, that is, by the eyes or by the left or right hand. To examine the degree of effector-specificity in each of these regions in more detail, we looked for systematic differences in activation related to effector choice across both target conditions. The results of this analysis are presented in Fig. 7, showing a systematic pattern of effector dependency across these brain areas.
Figure 7, left and middle, shows the results for each hemisphere. The right-most column presents the data averaged across both hemispheres. Consistent with the qualitative picture presented in Fig. 4, all regions were activated for both eye and pointing movements [eye movements: P < 0.05, t-test, hand (left or right) movements: P < 0.05, t-test], except for M1 which was not activated for eye movements (P = 0.27, t-test). AOC and FEF regions showed no significant differences in activation across effector choice (paired t-test, P > 0.05). The topographical area, retIPS, was activated more strongly for saccades than for pointing movements (paired t-test; P < 0.05). Furthermore, enhancement of activity seems to occur in all areas identified here, and it occurs most in the hemisphere contralateral to the hand used to reach (Fig. 7, right), and most strongly for M1, as would be expected given the results depicted in Fig. 4. This effect was significant for M1 and PMd (t-test, P < 0.05). In the more posterior brain areas aOC and retIPS, the trend remained, but below statistical significance (paired t-test; P > 0.05).
Contralateral topography and effector specificity
A comparison of Figs. 4 and 5 shows that the topographical regions retIPS, aOC, and FEF were activated by movements of all three effectors, that is, by the eyes or by the left or right hand. Similarly, some effector-specific regions, like PMd, did not show significant left-right target topography. However, some topographic regions, like retIPS, showed an effector preference in our separate effector analysis. Moreover, it is possible that some effector-specific or topography-specific results (including those which were below statistical significance in the preceding separate analyses) were masked by interactions between these two factors. To test for this, we investigated the interactions between hand specificity and topographic specificity in each of our regions of interest.
In Fig. 8 we have illustrated the activation of each area in four different conditions defined by either the left or right hand pointing to either leftward or rightward targets. There were no target location effects in the PMd and M1 regions (P > 0.05). Furthermore, the regions that showed topography for delayed-saccades (aOC, retIPS, and FEF) also showed lateralized responses in the delayed-pointing task. Strikingly, for our parietal region (retIPS), as well as for the aOC region (which failed to show significant effects in the previous lumped comparison), the activation was significantly greater when pointing with the unseen contralateral hand than with the ipsilateral hand to remembered target location in the contralateral visual field (paired t-test, P < 0.05). For example, activation was observed in the left retIPS and aOC regions when pointing to targets on the right with either hand, but the activation was significantly greater when using the right hand. The mirror symmetric effect was observed in the right retIPS and aOC regions (paired t-test, P < 0.05). In summary, the parietal and occipital regions, retIPS and aOC, show significant modulations related to both effector-hand and target location.
A crucial first step in the planning of a goal-directed action is integrating information about the target's location with information about the selected effector. Primate studies have implicated both the posterior parietal cortex and the premotor areas in this process. In line with these findings, the present human fMRI study also observed activation along the parietal-frontal network when subjects were planning hand actions. Except for the M1, all regions were activated for movements using either hand. The premotor area, PMd, demonstrated clear hand-specific modulations irrespective of the visual field where the target was presented. Its location corresponds to previous studies also showing this frontal area to be involved in preparatory set for finger and upper limb movements (see Picard and Strick 2001; for review). The putative FEF region is modulated by target location, but is effector nonspecific—its response for planning the eye movement was similar to the response for planning left or right hand movements (Astafiev et al. 2003; Connolly et al. 2000; but see Lawrence and Snyder 2002).
The main finding of this study was the spatial organization of the posterior regions retIPS and aOC. These regions, which encode target location in topographical maps, are activated for movements of the eyes or either hand, but these modulations are dependent on the effector selected to act on the targets. In this respect, our results for these regions are inconsistent with the notion of a generic salience spatial map (Colby and Goldberg 1999). Our results imply that these regions also play a role in effector selection. The regions retIPS and aOC appear to be unique because each is selective for remembered targets in the contralateral visual field, but the activation is also dependent on whether the left or right hand is used. It is somewhat surprising to find hand specificity in aOC, an area thought to primarily receive visual inputs. Its activation cannot simply be explained as being the visible hand moving in some part of the visual field because the subject's view of the hand was occluded. Moreover, no such responses were observed in the early visual areas (V1–V3).
There may be a saccade-to-pointing gradient that begins at retIPS and then extends inferior and medial to areas that no longer show a preference for contralateral targets. For example, Connolly et al. (2003) and Astafiev et al. (2003) reported effector specificity (for the hand over saccades) in the PPC, in an area that is located medial to retIPS.
Taking our findings a step further, area retIPS (and aOC) can be regarded as a point where information from a body map, representing the locations of the body parts, becomes integrated with information from remembered target locations. Why would a topographical map be useful for target-effector integration? An argument can be made that a topographical map provides an efficient way of organizing object locations in the outside world such that each of the body's multitude of effector systems can easily access and analyze the map. That is, such topography can provide a common spatial map for both target salience and effector relevance, particularly in determining which effector is optimally located to interact with the target. This is not to say that other factors like handedness (in terms of accuracy demands) and effector availability do not play a role in effector selection; these factors could bias some of the interactions within these target-effector salience map.
Figure 9 outlines the hypothetical flow of signals to the retIPS region. Information about target location, as gathered by the eyes, travels along an anteriorly directed stream into retIPS and is remapped whenever the eye or head are displaced (Henriques et al. 1998; Medendorp et al. 2003A; Medendorp and Crawford 2002; Medendorp et al. 2003B; Merriam et al. 2003). Information about the effectors originates in the postcentral sulcus, with each hemisphere predominantly representing the contralateral body parts in a somatotopical fashion. From here, a posteriorly directed stream flows to the retIPS and converts relative joint angles into the position of the hand relative to the body and then into the position of the hand relative to the eye to match the target location information. Each retIPS, and aOC as well (not shown), contains a topographical map of target location in the contralateral visual field, and superimposed on this is a map of possible effectors with those initially coded in the contralateral cortex having the weaker representation.
It is perhaps this difference in strength that facilitates the selection of the right hand for targets on the right and visa versa. This notion (depicted by the size of the arrows in Fig. 9) is also consistent with clinical and behavioral studies showing that each hand is more effective in directing movements to targets in the visual space ipsilateral to the hand (Di Pelligrino et al. 1997; Fisk and Goodale 1985; Perenin and Vighetto 1988). It is known, from our previous study, that the topography of retIPS is eye-centered (Medendorp et al. 2003a). To facilitate the integration of target and effector information, common coordinates are required. One way of achieving common coordinates is to code the information about the effector in retIPS in eye coordinates as well, which would be consistent with recent findings by Buneo et al. (2002). Note, however, that the results of our study do not disambiguate the representation of the hand from the selection of the hand. This would require the use of different starting locations of the hand.
If the function of the PPC is to select targets for action and the effectors to perform these actions (Snyder et al. 1997), then computations must occur in a common coordinate frame. But why would this coordinate frame be eye-centered? Recently, Buneo et al. (2002) suggested that this could facilitate a so-called direct visuomotor transformation (but see Crawford et al. 2004). Another advantage of an eye frame may be to simplify the orchestration of the eyes and hand when they move to the same target (Andersen et al. 1997; Cohen and Andersen 2002). Even reaches to sounds seem encoded in an eye-centered reference frame (Cohen and Andersen 2000). A final reason for an eye-centered coordinate frame is related to the spatial resolution of the coordinate frames, each linked to its own sensory modality. When the eye is directed toward the target, the target is represented on a high-resolution scale (fovea-resolution) in the eye-centered coordinate frame. On the other hand, it appears that directing the eyes on an unseen hand when executing reaches improves endpoint accuracy (Newport et al. 2001), and this could be interpreted as the map of initial arm position being retinotopic, but under-represented for regions distant from the fovea. Either way, this retinotopic resolution will degrade when information is transformed into body coordinates.
In summary, the integration of visual spatial information with hand proprioceptive information and the use of this information to generate commands for arm movement is a complex process. To plan and execute a motor action, information in different frames of reference must be integrated and controlled simultaneously. The results of this study suggest that topographic regions within the posterior parietal cortex play a crucial role in the integration of target and effector information for the planning of action and that this process likely operates in eye-centered coordinates.
This work was supported by grants from the Canadian Institutes of Health Research and the Canadian Natural Sciences and Engineering Research Council. W. P. Medendorp is supported by the Human Frontier Science Program and the Netherlands Organization for Scientific Research. J. D. Crawford is supported by the Canada Research Chair Program.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2005 by the American Physiological Society