HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) All Versions of this Article: 99/6/2968    most recent 90225.2008v1 Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (2) Citing Articles via Google Scholar Google Scholar Articles by Sorrento, G. U. Articles by Henriques, D. Y. P. Search for Related Content PubMed PubMed Citation Articles by Sorrento, G. U. Articles by Henriques, D. Y. P.

Reference Frame Conversions for Repeated Arm Movements

York University, School of Kinesiology and Health Science, Bethune College, Toronto, Ontario, Canada

Submitted 4 February 2008; accepted in final form 3 April 2008

	ABSTRACT

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
The aim of this study was to further understand how the brain represents spatial information for shaping aiming movements to targets. Both behavioral and neurophysiological studies have shown that the brain represents spatial memory for reaching targets in an eye-fixed frame. To date, these studies have only shown how the brain stores and updates target locations for generating a single arm movement. But once a target's location has been computed relative to the hand to program a pointing movement, is that information reused for subsequent movements to the same location? Or is the remembered target location reconverted from eye to motor coordinates each time a pointing movement is made? To test between these two possibilities, we had subjects point twice to the remembered location of a previously foveated target after shifting their gaze to the opposite side of the target site before each pointing movement. When we compared the direction of pointing errors for the second movement to those of the first, we found that errors for each movement varied as a function of current gaze so that pointing endpoints fell on opposite sides of the remembered target site in the same trial. Our results suggest that when shaping multiple pointing movements to the same location the brain does not use information from the previous arm movement such as an arm-fixed representation of the target but instead mainly uses the updated eye-fixed representation of the target to recalculate its location into the appropriate motor frame.

	INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
When we describe the location of an object, we express where it is with respect to some part of the body or to some location in the world. We refer to this as coding an object in a reference frame. For example, if an object was coded in a head-fixed reference frame, then its location (such as distance and angle) would be described relative to the head (or facing direction). Any movement of the head would require that the object's location be recalculated as function of that head movement. But there would be no change in the location of this object if the eyes were to move. Objects may either be coded, stored, and converted into any one of many of egocentric frames (eye, head, hand, or body centered) (Carrozzo et al. 1999; McIntyre et al. 1998) or allocentric (world centered) frames (Elliott et al. 1990; Goodale and Milner 1992; Lemay and Proteau 2003; Lemay et al. 2004). For goal-directed movements like reaching, information about the target and the hand may arrive in different reference frames, but eventually they need to be encoded in a common frame (Carrozzo et al. 1999; Cohen and Andersen 2002; Schlicht and Schrater 2007; Soechting and Flanders 1989, 1992). This common reference frame could be centered on the eye or another part of the body such as the hand. In this study, our aim is to investigate the flow of information about multiple spatial representations by looking at how repeated pointing movements to the same location are programmed. For instance, it is assumed that by the time the movement is executed, both the hand and the target are converted to, and maintained, in an arm-motor frame. Does the brain make use of the motor-related representation to program a repeated movement to the same location?

In an earlier study (Henriques et al. 1998), we inferred the reference frame of (at least 1 form of) spatial memory by exploiting an error in the brain's computations. When subjects point in darkness to the remembered location of a light flash, they point accurately if the flash falls on their central retina, but they mispoint systematically if the flash falls on their retinal periphery; this pattern is called the retinal magnification effect (Bock 1986). We showed that people make the same pattern of pointing errors even if they first see the flash on their fovea, but before pointing they move their eyes in darkness to a position in which the flash would have fallen on the retinal periphery. This means the eye movement alters the brain's stored information about the flash location; that is, the reference frame of this information is eye-linked. And these identical errors strongly suggest that the neural representation after the eye movement is the same as if the flash had actually fallen on the periphery; in other words, the brain accurately computes the new, peripheral location of the flash relative to the retina—a process called remapping or updating. Thus these results suggested that objects are stored and updated in a reference frame fixed to the eyes at least until they become a target for an action (Henriques et al. 1998). Subsequent behavioral studies have also shown that the brain updates spatial memory in an eye-fixed frame for pointing to targets at different distances relative to the body (Medendorp and Crawford 2002) or relative to gaze following body translations (Van Pelt and Medendorp 2007), to implicit targets produced by expanding motion patterns (optic flow like stimuli) (Poljac and van den Berg 2003), and to different sensory stimuli such as proprioceptive and auditory targets (Pouget et al. 2002). So even stimuli that are initially coded in a frame relative to the head or body, such as auditory and proprioceptive stimuli, are also transformed into eye-fixed coordinates. A possible advantage of this coding is that these different sensory signals can be combined in a common frame and compared with incoming visual information, necessary for planning goal-directed movements.

But likely the brain holds not one but many representations of object locations, likely in different reference frames and in different areas of the brain. Different representations probably subserve different aspects of motion planning and execution and are important at different stages of processing. This is consistent with the electrophysiological literature that suggests that neurons in arm-related motor areas in the brain, such the posterior parietal cortex and the dorsal premotor cortex, code reach plans in different reference frames. Neurons in the medial intraparietal area, in an area known as the parietal reach region (PRR) code for visual target location or the early reach plan in eye coordinates (Batista et al. 1999; Buneo and Andersen 2006; Buneo et al. 2002; Pesaran et al. 2006) while neurons in the more dorsal area 5 encode reaching targets in both hand and eye reference frames (Buneo and Andersen 2006; Buneo et al. 2002). Neurons in the dorsal premotor area code the location of the reach plan relative to the hand or limb, as well as the eyes (Batista et al. 2007; Boussaoud and Bremmer 1999; Boussaoud et al. 1998; Cisek and Kalaska 2002; Pesaran et al. 2006) or in an eye-hand combined (Pesaran et al. 2006) or some indeterminate reference frame (Batista et al. 2007). By the time these reach-related signals arrive at the primary motor cortex, these representations become anchored to the arm, coding for more intrinsic movement variables (Georgopoulos 1987, 1988; Georgopoulos and Stefanis 2007; Kalaska et al. 1989). These electrophysiological results suggest that the posterior parietal lobe and dorsal premotor area may reflect a gradual change in the coordinates used in the formation of the motor plan (Pesaran et al. 2006).

To date, this processing pathway—from eye to arm-motor coordinates—has only been described for a single arm movement. It remains unknown where along this continuum information would be coded for programming repeated movements. One possibility is that once a location is converted to an arm-related frame, this representation persists so it can be used for subsequent actions involving that same location. Another possibility is that the target location could be retained in an eye-fixed frame and thus reconverted from an eye to an arm-motor frame every time a pointing movement is made. We tested these two possibilities by comparing pointing errors made during subsequent pointing movements made to the same remembered location with gaze in different directions. In a second experiment, we had subjects perform a similar repeated pointing task, but this time with the torso and shoulder in slightly different locations relative to the target. If subjects use the updated eye-fixed representation of the target for each arm movement, they should make errors that depend on current gaze direction (because of the retinal magnification effect), which would result in different pointing errors with respect to the body for the first and second pointing movements. In this case, a spatially updated representation of the remembered target relative to gaze, and not any preceding motor-frame representation, would be used to calculate the goal for subsequent pointing movements. We call this the reconversion hypothesis since this scheme would require that a new arm motor representation of the target must be reconverted from eye coordinates each time the eyes move, even for repeated arm movements to the same target. But if subjects convert just once for both arm movements, or use motor-related information from the preceding arm movement, then errors should not depend on current gaze direction, they should make similar errors, with respect to the torso, on first and second pointing movements. We call this the motor-memory hypothesis because it can either involve an arm-related representation of the target computed when programming initial arm movements or memory traces of the proprioceptive and/or efference copy signals from this preceding arm movement. Any or all of these sources of motor-related information could be stored and reused for subsequent repeated movements. The aim in the second experiment, where subjects point the first and second time with the torso turned from the target (but with the head stationary) was to further distinguish between the contributions of these motor-related sources.

Alternatively, subjects may use a combination of the two schemes or representations. The brain may optimally combine both the newly reconverted representation as well as the previous motor-related representation of the target to program repeated arm movements.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
Subjects

Nine right-handed human subjects participated in the first paradigm of the study; four females and five males between the ages of 19 and 30 (mean = 23). All subjects had normal or corrected-to-normal vision, and none of these subjects had any known neuromuscular deficits. Six of these subjects returned to participate in the second experiment. All subjects gave informed consent and all procedures involved in the experiment were approved by the York Human Participants Review Subcommittee.

Apparatus

The subject sat with the head fixed using a bite bar that was mounted onto a table directly in front of them. The height of the bite bar was adjusted so that the subject's right eye was vertically and horizontally aligned with the central visual stimulus. The left eye was patched to eliminate any retinal disparity and changes in eye dominances as a function of gaze (Khan and Crawford 2003). A matte sheet mounted on a Plexiglas screen measuring 178 cm in width was placed at 150 cm in front of the subject (measured from the middle of the eye to the screen surface).

All visual stimuli were generated by a red Optikon XYLP-C Laser scanner (Optikon, Kitchener, ON, Canada) and rear projected onto the screen. The stimuli consisted of an array of fixation sites (displayed as crosses) and pointing targets (diamonds), as shown in Fig. 1. Fixation crosses were 2 cm or 0.76°, while target diamonds were 1.25 cm or 0.48° in diameter. Fixation crosses were located at center and 5, 10, and 15° to the left and to the right of center. The three pointing targets were located at center and 5° left and right. The laser was controlled by a custom-designed program run from a separate CPU that controlled the laser scanner and communicated the onset and conclusion of trials to the MotionMonitor (described in the following text).

View larger version (7K): [in this window] [in a new window]   FIG. 1. Experimental setup for both body-stationary and -rotated paradigms. The 3 pointing targets (diamonds) were located directly in front of the right eye (0°), 5° left, and 5° right. The 7 fixation crosses were located 15, 10, and 5° to the left and to the right of the central fixation cross (0°). The left eye was patched, and the head was fixed by a bite bar.

Pointing performance was measured using a two-camera Optotrak Certus 3D motion tracking system (Northern Digital). These cameras recorded the 3D position of infrared-emitting diodes (IREDs 7 mm diam) and used to track the movement of body segments such as the head, upper arm, and index finger. Two single IRED markers were taped to the lateral tip of the index finger. To measure angular data from the head and right arm, we attached rigid body sensors to them consisting of either three or four IRED markers prearranged in close and fixed proximity on the surface of the flat disc (5 cm diam and 1.5 cm thick). One of the rigid body sensors was taped to the upper part of the right arm, while another was attached to the Eyelink II to monitor any change in head position or the eye-tracking headset.

For one of the two paradigms in this study, the body-rotated paradigm, additional sensors were placed on the torso, on the edge of the table in front of the subject, and two more on the rotating chair. The two chair sensors were set up 50 cm apart and orthogonal to each other: one was placed on the backrest and the other on the side of the seat of the chair facing the Optotrak cameras.

Data from the Optotrak Certus and SR EyeLink II were integrated with and controlled by The MotionMonitor or TMM (Innovative Sports Training, Chicago, IL) to ensure a common temporal and spatial reference between the two data sets. These data were compiled in TMM at a common 125-Hz sampling rate. TMM was used to jointly calibrate the EyeLink II and the Optotrak Certus equipment for eye and pointing movements and to set up and define the coordinate system for recording angular and positional data. The origin was defined when subjects were in a neutral position, looking and pointing to a central target with the arm fully extended. Neutral position was recorded prior to the experiment, and all orientation and position data recorded in the experiment were calculated relative to this position. The IRED coordinates were transformed off-line into a left-hand space-fixed coordinate system. The x-y plane was aligned with the subject's frontal plane (parallel with the screen), such that positive y axis was pointed upward and positive x axis pointed to the right of the subject along the shoulder line in the neutral arm position.

Experimental paradigms

There were two paradigms in this study, both of which involved subjects pointing to a remembered target twice. In the first paradigm, both the head and body were stationary; subjects sat comfortably in a chair facing the screen: the body-stationary paradigm. In the second paradigm, the body-rotated paradigm, which will be described in more detail later, the head remained fixed, but subjects rotated their body between the pointing movements so that their torso was in a different position for each of the movements.

In both paradigms, each trial began with the subject holding down a single-button Apple mouse in a fixed location on the table. Subjects pressed down on the mouse before and after each pointing movement so we could monitor when subjects started and finished each arm movement. If subjects lifted their hand from the mouse before a computer-generated verbal command to "point" to the remembered target location, the trial was aborted (and recycled to be performed later). In this way, we ensured that the arm was never up when the stimuli were on and that the initial position of the hand before each pointing movement was the same. Experiments were done in complete darkness so no visual feedback was available. To prevent dark adaptation, a desk lamp was programmed to turn on for 3 s after each trial. Experimenters monitored the progress of the experiment in an adjacent anteroom.

Body-stationary paradigm

In the body-stationary paradigm (Fig. 2), each trial began with subjects fixating on the pointing target (diamond) while it appeared for 1 s (Fig. 2A). After the target diamond disappeared, a fixation cross (not shown) immediately followed, and subjects directed their gaze to this new location and continued to do so after the fixation cross disappeared 1 s later (Fig. 2B). Subjects were then given the verbal command to "point." Subjects pointed to the remembered target site with the arm fully extended while still maintaining gaze toward the fixation site (Fig. 2C). After completing the pointing movement, subjects returned their hand to the start location and pressed down on the mouse. This mouse click triggered the appearance of the second fixation cross for 1 s. The cross either appeared at the same location as the first cross, or one of the possible locations on the opposite side of the pointing target (Fig. 2D). For instance, if the target was at center (0°) and the first fixation cross was 10° to its right, the second cross could be at 10° right (same as the 1st cross) or 5, 10, or 15° left (opposite side of the target, relative to initial fixation). Subjects redirected gaze a second time to the new fixation cross. When the second fixation cross disappeared 1 s later, subjects were given a verbal command to point for a second time toward the same remembered target while maintaining gaze toward the site of the second fixation cross (Fig. 2E).

View larger version (6K): [in this window] [in a new window]   FIG. 2. A typical trial performed in the body-stationary paradigm. The subjects initially fixated the target (diamond) that briefly appeared for 1 s (A), and then fixated a cross when it appeared subsequently for 1 s (B). Afterward, subjects pointed to the remembered target site while still maintaining gaze toward the location of the fixation cross (C). To illustrate the retinal magnification effect, we show the subject overshooting the target slightly to the left while gaze was directed to the right. Once the 1st pointing movement was complete, a 2nd fixation cross appeared for 1 s (D), and then the subject pointed a 2nd time to the remembered target with gaze directed toward this new fixation site (E). The dashed arms show the 2 predictions. The motor-memory hypothesis predicts that the 2nd pointing movement should resemble the 1st movement (same leftward pointing error as in C). The reconversion hypothesis predicts that the subject will overshot the remembered target site relative to the new gaze direction, so that errors for the 2nd pointing movement will fall in the direction opposite to that of the 1st movement relative to the target.

View larger version (14K): [in this window] [in a new window]   FIG. 4. Examples of trials for the body-stationary (A) and body-rotated (B) conditions. A: horizontal eye (dotted lines) and arm (solid lines) directions are plotted across time (in seconds). The body-stationary trial begins with a target at 0° (open box), then the 1st fixation light presented at 10° right (closed box on the left), followed by the 2nd fixation light 10° left (closed box on the right). The arm trace indicates 2 pointing movements to the same remembered target location (0°). The 1st pointing movement is made while gaze was to the right (toward the first fixation site), while the 2nd pointing movement is made while gaze is directed to left (toward the second fixation site. B: horizontal eye (dotted lines) and finger (solid lines) traces are plotted across time. Finger position across time is shown both in centimeters and in degrees relative to the eye-target line. The body-rotasted trial begins with the target at 0° (open box), followed by a 10° leftward fixation (solid box), and a 10° rightward fixation (2nd solid box). Between the 2 pointing movements to the same target location, subjects turned their chair and body (shown by the gray curve) from 10° left to 10° right with their feet. The 1st pointing movement is made with gaze 10° left, and then gaze shifted toward the rightward fixation light for the 2nd pointing movement. Arrows indicate the approximate time of auditory signal to point.

Pointing results for trials where subjects both viewed and pointed to the same spot served as a measure of baseline performance. Previous studies have shown that pointing in these conditions were considerably accurate (e.g., Bock 1986; Henriques et al. 1998). Pointing errors made to nonfoveated targets (i.e., when the fixation cross and pointing targets were different locations) were calculated by subtracting these pointing directions from the baseline pointing responses to the same target. The target and second fixation cross never fell in the same location except in the case where the target and both the first and second fixation stimuli were located in the same spot.

Body-rotated paradigm

The body-rotated paradigm was similar to the body-stationary paradigm except that the subjects rotated their bodies 10° to the left or right of center on a turnable chair between pointing movements so that their torso was in a slightly different position for the first and second movements. As in the original paradigm, the subject's head was fixated during the entire experiment using a bite bar. When pointing to distant targets, the extended arm is oriented so that the finger intersects the eye-to-target vector (Henriques and Crawford 2002). Because the head is stationary at all times, the horizontal finger position required to intersect the eye-target line should be the same for both aiming movements, but with the torso rotated, this same finger position requires a different angle of the arm for the two torso positions. Because of this, we used finger position rather than arm direction as a measure of pointing.

The motor-memory hypothesis predicts that the horizontal finger position should be the same across the two pointing movements. The reconversion hypothesis, however, predicts that horizontal finger position would be on different sides of the target because pointing direction should vary with current gaze direction in accordance with the retinal magnification effect. Because the body-rotated paradigm required different arm postures across the two movements for the same trial, we were able to refine the motor-memory hypothesis because efferent and proprioceptive signals of the preceding arm movement or arm posture would not be useful in this case. Only the information about horizontal finger position would be the same across the two movements, and therefore only a finger- or hand-centered representation of the target could possibly be used for programming multiple actions to the same location.

For the first trial in the body-rotated paradigm, all subjects began with their body turned 10° to the right. Subjects looked at the target (on for 1,000 ms, Fig. 3 A) until it disappeared and then looked toward the first fixation cross, which appeared for 1,000 ms (Fig. 3B). As soon as the fixation cross disappeared, subjects were given a computer-generated verbal command to point to the remembered target location while still maintaining gaze toward the fixation site (Fig. 3C). After completing the pointing movement, subjects returned their hand to the mouse. Pressing the mouse button triggered a computer-generated verbal command to "turn left." Subjects actively turned the chair (and themselves) by using their feet until the chair hit a barrier at 10° left from center. Subjects tended to take between 1.5 and 2 s to complete this 20° chair rotation. Custom-made laser sensors at the bottom of the chair triggered the trial to continue (i.e., display the 2nd fixation) only when the subject was fully rotated to the correct position. Once these sensors detected that the chair had reached the 10° leftward position, the second fixation cross was displayed for 1 s either at the same spot as the first cross or in one of three locations on the other side of the target. Again subjects looked at the fixation cross and maintained this new gaze direction for the remainder of the trial (Fig. 3D). After the second fixation cross disappeared, a verbal command instructed subjects to "point" to the same remembered target location for a second time (Fig. 3E). In summary, during the first pointing movement, the subject's torso was oriented 10° right, but for the second pointing movement, the torso was oriented to the left, while the head and initial hand position remained constant.

View larger version (7K): [in this window] [in a new window]   FIG. 3. A typical trial performed in the body-rotated paradigm. While the body was rotated to the right, head-fixed subjects looked at the target while it was on for 1 s and then looked toward the fixation cross that appeared for 1 s. (B). Afterward, subjects pointed to the remembered target location while still looking toward the fixation site (C). After the completion of the pointing movement, a verbal command instructed subjects to actively turn the chair until it hit a barrier 10° left of center, and once there, a 2nd fixation cross was displayed for 1 s (D). They then pointed a second time to the remembered target with gaze in this new direction and the body oriented to the left (E). As in Fig. 2, the dotted arms show the 2 predictions.

In the body-rotated paradigm, we needed twice as many trials: a set where the body was rotated left first and then right and a set with the body rotated right and then left. So to reduce the length of the entire block, we reduced the number of permutations from those used in the body-stationary paradigm by removing the fixation-target combinations where the distance between the fixation cross and target location was >10°. So unlike the body-stationary paradigm, which had retinal eccentricities (RE) ranging from 0 to 20° right or left, the body-rotating paradigm only had RE errors from 0 to 10° in each direction. These permutations are the bold values in Table 1 and so consisted of 13 combinations for each of the three targets, plus another three trials when the target and the two fixation crosses were all in the direction spot (italicized values) to ensure we had a sufficient number of trials for a baseline measure. So this added up to 48 permutations for each of the two body rotations, for a final total of 96 trials per session. A typical body-rotated trial lasted from 11 to 13 s with a 3-s pause between trials, and the duration of a block of trials was 30 min. Subjects completed three blocks again on separate days to prevent fatigue.

Data analysis

After data collection, all raw data were exported off-line from TMM and into a Matlab-based Graphical User Interface (GUI) where all data could be plotted and viewed across time (like those in Fig. 4). For each trial, for each of the two pointing movements (solid traces in Fig. 4), we manually selected the two corresponding points on the traces representing horizontal and vertical arm directions (or finger positions for the body-rotated paradigm) when the traces became most stable. Horizontal and vertical eye direction (dotted traces in Fig. 4) were also visually selected at this time as well as during periods when subjects foveated the target diamond and both fixation crosses. The selected data required gaze accuracy within 2–3° of the fixation location even after stimuli were extinguished. For the body-rotated paradigm, we also verified that the torso and chair were in the correct orientations at the selected times. Subjects were instructed to never look back toward the remembered target location after it was initially foveated at the beginning of the trial; trials in which this occurred were removed from further data analysis. While some trials where subjects incorrectly looked back to the target were redone, a total of 14% of trials had to be discarded because subjects did not maintain gaze within the 2–3° fixation window prior to and during pointing. We also selected the three trials for pointing calibration using the same criteria as the experimental trials.

Pointing errors for each movement were computed by subtracting arm direction (and/or finger position) during trials when the fixation and the target did not overlap from the average performance of trials where fixation and target were in the same location (baseline performance). Using the SPSS statistical package, we conducted an ANOVA to compare horizontal pointing error during the first and second pointing movement, as a function of the RE for both body-stationary and -rotated paradigms.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
Body-stationary paradigm

In this study, we wanted to investigate what information the brain uses when people make repeated pointing movements in the dark to the same remembered location. To test this, we had subjects point twice to the same remembered target, after shifting gaze to different sides of the target site. If repeated pointing movements used an arm-motor representation of the target or other related motor-memory signals from the preceding movement, then errors for both movements should be the same regardless of the intervening eye movement. But if the repeated movement instead involved reverting back to an earlier, eye-centered representation of the target before transforming its location again into motor coordinates, then the intervening eye movement should result in different, gaze-dependent, pointing errors for the two movements. Figure 5 illustrates how subjects pointed the first time (open circles) when looking 10° in one direction (open cross), then pointed a second time (closed circles) while looking 10° to the opposite side (solid cross) of the remembered target. Subjects began all trials by initially looking at the visible target (represented by the origin of the axes). Figure 5A displays 2D pointing errors (open circles) for one subject when pointing for the first time while looking to the left (open cross). On average, the subject tended to point slightly to the right of the target site (overestimating the location of the target relative to where he was looking at the time), as is consistent with previous studies (Beurze et al. 2006; Henriques et al. 1998; Poljac and van den Berg 2003; Pouget et al. 2002). Closed circles depict pointing errors for the same trials after the subject moved his eyes across to the opposite side of the target (10° right) before pointing a second time to the same target site. Errors for this second movement differ from those for the first movement; that is, the solid circles on average fall to the left of the open circles.

View larger version (24K): [in this window] [in a new window]   FIG. 5. Final 2-dimensional (2D) pointing errors for trials of a single subject (A and B) and all subjects (C and D) when looking 10° to either side of the 3 target locations for the 2 repeated pointing movements in the body-stationary condition. For the 1st pointing movement, open crosses indicate gaze direction while open circles indicate pointing errors; closed crosses and closed circles indicate gaze and errors for the 2nd movement. A: trials where the subject pointed for the 1st time while looking left (open cross) and then moved their eyes to the right (solid cross) while they pointed a 2nd time for the same trial. B: trials where the gaze was 10° rightward for the 1st pointing movement and then shifted to the left for the 2nd pointing movement. C and D: subject's average pointing errors (different colors) for each target with averages across subjects in black. The direction of gaze for these trials is the same as those for A and B, respectively.

Figure 5, C and D, illustrates the average 2-D pointing errors for each of the three targets for all nine subjects (different colors for each subject). The black circles are average pointing errors for the three targets across all subjects. As was the case for the single subject in Fig. 5, A and B, open crosses and circles indicate the first fixation light and first pointing responses, respectively, while the solid crosses and solid circles indicate those for the second pointing movement. In Fig. 5C, average pointing errors are plotted for both movements for trials where subjects looked 10° to the left after foveating the briefly presented target for the first pointing movement; for this first movement, subjects missed the target by 2.69 ± 3.28° right (mean ± SD) on average. In the same trials, they pointed again to the remembered target while looking in a rightward direction, and pointed on average 0.23 ± 2.56° left of the target for a difference of 2.92° between second and first pointing movements. Figure 5D shows the average errors for trials when subjects pointed for the first time with gaze to the right of the target (0.62 ± 3.14° left) and finished these trials by pointing a second time (2.53 ± 3.10° right) while gaze was to the left for a difference of 3.15° between pointing errors. For all subjects, pointing errors differed between the two aiming movements [1st RE = 10° left, t(76) = 10.68, P < 0.001; 1st RE = 10° right, t(85) = –9.16, P < 0.001, Bonferroni corrected pairwise t-test] following the intervening eye movements, such that errors tended to fall on opposite sides of the target site as a function of final gaze.

In some trials, the fixation cross reappeared in the same location for the second pointing movement so the eyes did not move at all between pointing movements. These trials served as a control: since gaze did not change between the first and second movement, we expect errors to fall in the same direction for both pointing movements. Figure 6 A illustrates 2D pointing errors when subjects were looking 10° to the left (gray cross) while pointing both times to the target location (different colors for each subject, with black for average across subjects for the three targets). We find that the first (open circles) and second (closed circles) pointing responses tended to both be biased to the right (2.51 ± 2.47 and 2.91 ± 3.09°) so that they overlapped each other, indicating there was no difference in how subjects pointed the second time compared with the first [t(69) = –1.61, P > 0.05, Bonferroni corrected pairwise t-test]. For trials when gaze was directed toward the target site when making both pointing movements (Fig. 6B), we again see that the open and closed circles greatly overlap each other slightly to the left of the target site [0.54 ± 2.58 and 0.06 ± 3.00°; t(986) = –1.85, P > 0.05]. When gaze was directed to the 10° to the right of the target site for both movements (Fig. 6C), errors were biased slightly to the left (0.09 ± 2.44 and 0.68 ± 2.54°) and again were not significantly different from each other [t(77) = 2.46, P = 0.05].

View larger version (18K): [in this window] [in a new window]   FIG. 6. Final 2D pointing errors for trials where subjects looked in the same direction when making the 1st and 2nd pointing movements (gray crosses). Each subject's average pointing errors for the 3 targets are shown in different colors, and the averages across subjects are shown in black. Pointing errors for the 1st movement to the 3 target locations (open circles) and those for the 2nd movement (solid circles) when gaze was 10° to the left (A), toward the target site (B) and 10° to the right (C) for both pointing movements.

View larger version (34K): [in this window] [in a new window]   FIG. 7. Horizontal pointing errors averaged for each subject (different colors) and for all subjects (black) plotted as a function of gaze direction relative to the target. The horizontal line at 0° represents accurate pointing. A: mean errors for trials when gaze was left (solid circles) or right (open circles) when pointing for the 1st time. B and C: for the same trials from A, subjects moved their eyes to the opposite side of the target site and pointed a 2nd time. Second pointing errors for these panels are plotted as a function of the direction of gaze for the 1st pointing movement (B) or as function of current direction of gaze (C). Because gaze has shifted before the 2nd pointing movement, solid circles represent errors after gaze moved right and open circles represent errors after gaze moved left, as explicitly plotted in C. D: pointing errors for movements made a 2nd time when the eyes did not move between the 1st and 2nd pointing movement. The solid circles indicate pointing errors when gaze remained to the left (as in A) and open circles are those when gaze remained to the right (as in A).

Figure 7D plots the average horizontal pointing errors for the second movement for trials where gaze remained in the same direction for both pointing movements. Pointing errors across the two movements did not significantly differ [F(1,541) = 1.77, P > 0.05], suggesting that when gaze was in the same direction for both movements, the errors for both movements were directed to the same side of the target: when gaze remained to the left, errors (solid circles) fell to the right while when gaze remained to the right, errors (open circles) fell to the left for both panels. This was expected given that gaze had not moved.

Our next step was to collapse the data in Fig. 7 into a single, direct measure of the two hypotheses. This was accomplished by plotting the mean pointing error for each combination of first fixation and target for the second movement as a function of those for the first movement when the gaze was in the opposite direction for each movement (plotting errors shown in Fig. 7B as a function of those in Fig. 7A). Figure 8 A shows the fits for these data for each subject in a different color and the grand mean regression fit in black. We also plotted the mean error for the second movement followed by a gaze shift for each combination of fixation and target as a function of the mean error for second movements when gaze did not move for the same combinations (plotting errors shown in Fig. 7B as a function of those in Fig. 7D). These fits are shown in Fig. 8B (subjects in color, overall average in black). For clarity, we normalized the intercepts to zero. If the second movement is guided entirely by the same motor-related representation as the first movement (motor-memory hypothesis), then we would expect all slopes to fall along a positive unit slope (1.0) shown by the gray dotted line because errors for the second pointing movement should be similar to the first for Fig. 8A. The same would be true for Fig. 8B; if the motor-memory hypothesis is correct, pointing errors for a second movement following a gaze shift (ordinate) should be the same as those errors for a second movement when gaze did not shift (abscissa). Conversely if the location of the remembered target site is recomputed relative to current gaze each time people pointed to that location (reconversion hypothesis), then slopes should fall along a negative unit slope (–1.0), as shown by the gray dashed line, because subjects should make different errors for the second movement compared with the first when gaze shifted across the target site (Fig. 8A) and compared with second pointing movements made when gaze did not move (Fig. 8B). If pointing errors fall along a negative slope, this would indicate that the second pointing movement is guided entirely by an updated representation of the target location (relative to gaze).

View larger version (22K): [in this window] [in a new window]   FIG. 8. Slopes for horizontal pointing errors for the second pointing movement as a function of those for the 1st (A) and as a function of errors for the 2nd pointing when the eyes are still (B) for each combination of initial fixation and target in the body-stationary condition. Individual subjects are shown by colored slopes, and the average across subjects is shown by the black slope. The predicted slope for the motor memory hypothesis is shown by the dotted line, while the predicted slope for the reconversion hypothesis is shown by the dashed line. The intercepts have been normalized to 0.

But because these slopes fell below the negative unit slope (–1.0), this suggests that the effect of gaze (or retinal magnification) was overall smaller for the second pointing movement when it was followed by an intervening gaze shift, compared with the effect of gaze on the first movement or on the second pointing movement where gaze remained in the same direction. This overall smaller gaze-dependent modulation of errors can also be seen in Fig. 7—the sigmoidal black curve representing the average errors across subjects in Fig. 7C is smaller than (only 61 and 52% of the height of) those in Fig. 7, A and D. This is not to suggest that the magnitude of the errors were smaller for the second movement—the absolute pointing errors for the two movements made in the same trial were not significantly different [t(1814) = –1.329, P > 0.05]. One possibility for this reduction may be that motor-related information from the previous movement is playing a small role in guiding the second movement and thereby reducing the magnitude of the gaze-dependent effect.

Body-rotated paradigm

In the body-rotated paradigm, we wanted to test whether making the motor related information less relevant (by changing the required arm angle needed to achieve similar horizontal finger position) would result in the second pointing movement being guided more by the updated or recomputed location of the target relative to gaze. In this paradigm, the body was turned in different directions when viewing and pointing to the target site the first time and when pointing to it again the second time. Because head position and the resulting eye-target line remained the same, the ideal horizontal finger position for accurate pointing was also equivalent, but the arm angle necessary to horizontally position the finger differed (see METHODS) (Henriques and Crawford 2002).

As in Fig. 5 for the body-stationary paradigm, Fig. 9 shows how the same subject performed in trials when he pointed while looking in opposite directions of the target (A–D) as well as averaged performance of all subjects (E–H). Trials either began with the body rotated to the left (Fig. 9, A, B, E, and F) or with the body rotated to the right (C, D, G, and H) for initial viewing and initial pointing to the target site. In Fig. 9A, while their torso was initially turned to the left, gaze was directed to the left for the first pointing movement. Like in the body-stationary paradigm, this subject seemed to overestimate the target (open circles) with respect to current gaze direction (open cross) and so erred mostly to the right. For the same trials, after the subject turned his body to the right and then looked 10° to the right of the target (solid crosses), the subject mispointed mostly to the left (solid circles, Fig. 9A). Once more, we find that the errors for the two movements were different relative to each other. Figure 9B shows trials in which the subject was rotated initially left and fixated to the right, then turned his torso right and fixated left for the first and second pointing movements. Again, errors tended to fall on different sides. Figure 9, C and D, illustrates pointing errors in trials where order of gaze direction was the same as those for A and B, respectively, except the subject's torso was initially rotated to the right during the first movement and then rotated to the left for the second pointing response. Again we see that errors for the two pointing movements (open and closed circles) differed from each other for trials where the gaze was on different sides of the target site.

View larger version (27K): [in this window] [in a new window]   FIG. 9. Final 2D pointing errors of a single subject (A–D) and average errors across all subjects for the 3 targets (E and F) when gaze was directed 10° to either side of the target location for the 2 repeated pointing movements in the body-rotated condition. Conventions are similar to those in Fig. 6. A: trials where the body was initially rotated leftward (10°) and gaze was left (open cross) when pointing for the first time (open circles). In the same trial, the body was then rotated rightward and gaze moved to the right (closed cross) and pointed a second time (solid circles). B: trials where the body also began turned leftward, with gaze to the right for the 1st pointing movement (open symbols), and then the body rotated rightward and gaze moved to the left for the 2nd pointing movement. C and D: trials are similar to A and B, respectively, only the subject began these trials with their torso rotated initially to the right for the 1st pointing movement and rotated left for the 2nd movement. E–H: average pointing errors for each subject (colors) and averaged across subjects (black) per target for trials where gaze and the body were in the same directions as those shown for A–D, respectively.

Figure 9, G and H, shows pointing errors for similar gaze directions as those in Fig. 9, E and F, respectively, but this time the body was turned to the right when pointing the first time and then moved to the left when pointing to the remembered target the second time. The results show the same pattern as that shown for the opposite body rotation in Fig. 9, E and F; errors for the second pointing movement (solid circles) tended to fall on the opposite side of the target as well as opposite to the location of the first pointing errors (open circles). The two pointing movements were significantly different with an average difference of 5.34 cm when the eyes moved from left to right [t(46) = 7.98, P < 0.001] and 7.14 cm from right to left [t(46) = –11.69, P = 0.001, Bonferroni corrected pairwise t-test]. In summary, when the required arm position was altered by the rotation of the torso between pointing movements, pointing errors nevertheless continued to follow the same pattern as in the body-stationary condition in that the errors depended on gaze direction for that movement.

Using the same format as Fig. 6, Fig. 10 shows the average 2D errors for the first and second pointing movements to the remembered target for those trials where subjects did not shift their gaze between pointing movements in the body-rotated paradigm. The circles are those errors made when the subjects pointed the first time to the remembered target with the torso turned to the left and the second time after turning their torso to the right. The diamonds show those errors when the torso began rotated right and then turned left for the two pointing movements. These trials served as controls allowing us to compare these errors with those in trials where the subject looked in different directions for each pointing movement (Fig. 9). Because the gaze direction remained the same for both the first and second pointing movements for these trials, pointing errors should be similar, and thus fall in the same general direction relative to the remembered target. When gaze remained to the left for each torso rotation (Fig. 10A), the errors fell to the right of the remembered target (3.95 and 3.11 cm for the open and closed circles, 4.02 and 5.39 cm for the open and closed diamonds) and were not significantly different [t(19) = 1.61, P > 0.05; t(19) = –2.23, P > 0.05, Bonferroni corrected pairwise t-test]. When gaze was to the right (Fig. 10C), the errors did not significantly differ and when the body rotated right to left [t(19) = –0.07, P > 0.05], they did when the body rotated left to right [t(19) = 5.14, P < 0.05] because errors for the second movement were farther to the left (6.25 cm) than that of the first movement (4.39 cm). In summary, the open and closed symbols mostly overlapped, again in the direction opposite to gaze. So when gaze remained in the same direction across the two pointing movements, there was no substantial change in the direction that subjects pointed despite the different anatomical configurations required to point when rotated either way.

View larger version (18K): [in this window] [in a new window]   FIG. 10. Two-dimensional pointing errors for trials where gaze was directed to the same spot (gray cross) during the 1st and 2nd pointing movements in the body-rotated condition: 10° left of (A), 10° right of (C), and toward (B), the target site. Circles denote when subjects pointed with their body turned to the left for the 1st pointing movement (open circles) and turned to the right for the 2nd movement (closed circles), respectively. Diamonds are pointing errors for the reverse body rotation: rotated right when pointing the 1st time (open diamonds) and then turned left when pointing a 2nd time (closed diamonds). B: when gaze remained directed toward the target site for the entire trial, only pointing errors for the 2nd movement are shown, since pointing direction for the 1st pointing movement was used as a baseline for calculating all other pointing errors when gaze was deviated from the target site.

As in the body-stationary paradigm (Fig. 7), Fig. 11 plots the mean horizontal pointing errors averaged for each subject (in different colors, with the grand mean in black) as a function of current gaze direction relative to target. The top row depicts pointing errors when the torso was rotated left for the first pointing movement (A) and then turned right for the second movement (B and C) to the same remembered target, while the bottom row depicts pointing errors when the torso was turned rightward between the first (D) and second (E and F) pointing movements. The middle panels (B and E) are for those trials where the eyes moved to the opposite side of the target prior to the second pointing movement, whereas the right panels (C and F) are for those trials when gaze remained in the same direction throughout the trial. Like in Fig. 7, solid circles represent trials where gaze was to the left of the target site for the first pointing movement (Fig. 11, A and D) and then shifted to the right (Fig. 11, B and E) or kept to the left (Fig. 11, C and F) for the second pointing movement during the same trial. Open circles represent results for the reverse gaze combinations: gaze to the right for the first movement (Fig. 11, A and D) and then shifted to the left for the second movement (Fig. 11, B and E) or kept to the right (Fig. 11, C and F) for the second movement. Figure 11, A and D, shows that subjects mispointed in the direction opposite to where they were looking both when the body was turned to the left and to the right. Like those errors made when the body was stationary, errors in the body-rotated paradigm vary systematically as a function of gaze direction relative to the target (i.e., retinal eccentricity), consistent with the retinal magnification effect. So the question is whether subjects also systematically overshot the target site relative to the second (current) gaze direction when gaze shifted to the opposite side of the target or whether the errors for the second pointing movement were similar to those of the first movement so that the closed (or open) circles in Fig. 11, B and E, should be in the same direction as the closed (or open) circles in Fig. 11, A and D. The results plotted in Fig. 11, B and E, show that errors for the second movements varied as a function of current gaze direction and so did not fall in the same direction as those for the first pointing movement when gaze was directed to the opposite side. Instead the errors tended to again fall on the opposite side of the target site (the abscissa) relative to gaze, although the amount of gaze-dependent modulation was much smaller. Consistent with this we found that pointing errors between the two movements were significantly different both when the body rotated left to right [F(1,380) = 39.43, P < 0.001] and when the body rotated right to left [F(1,359) = 15.75, P < 0.001]. Only one subject showed the opposite gaze-dependent trend following a leftward body turn (blue line in Fig. 11E) and showed hardly any gaze-dependent modulation following a rightward body turn (blue line in Fig. 11B). Yet this same subject also did not show pointing errors that were all that similar to the first pointing movement either, especially for leftward gaze directions where they consistently pointed to the right of the remembered target site. Overall, for both directions of body rotations, errors differed between repeated movements to the same target when gaze was redirected to the other side of the target, such that these errors fell on opposite sides of the target site as a function of final gaze.

View larger version (40K): [in this window] [in a new window]   FIG. 11. Horizontal pointing errors averaged for each subject (different colors) and across all subjects (black) plotted as a function of gaze direction relative to the target. Conventions are the same as those in Fig. 7. Top: pointing errors when subjects began with their body turned left for the 1st pointing movements (A) and then rotated to the right for the 2nd pointing movements made after gaze shifted to the opposite side of the target (B) and when gaze remained in the same direction (C) as that for the 1st movement (e.g., A). Bottom: errors made when the body was turned right for the 1st movement (D) and then rotated left for the 2nd pointing movements made after gaze shifted to the opposite side of the target (E) and when it did not (F). Solid circles in A and D show errors for the 1st movement made when gaze was left, while solid circles in B and E are for the same trials when gaze shifted to the right for the second pointing movement. Similarly, open circles in A and D show errors from the 1st movement when gaze was right while the same symbols in B and E are errors for the same trial after the gaze moved left for the 2nd arm movement. Solid and open circles for C and F represent the same gaze directions as those shown in A and D because the panels plot errors for the 2nd pointing movement when gaze direction did not change from the 1st movement.

Using the same conventions as the body-stationary paradigm (Fig. 8A), Fig. 12, A and C, plot fits made to final pointing error distributions of the second pointing movement as a function of the first movement for the same trials when eyes were in the opposite direction. In Fig. 12, B and D (like Fig. 8B), we plotted fits to errors made during the second pointing movement for those trials where gaze shifted (as shown in Fig. 11, B and E) as a function of the second pointing movements when gaze did not shift (as shown in C and D) for the same initial fixation direction. Individual slopes for all subjects are shown in color with the averages across subjects in black. The top panels depict trials when the body was rotated initially to the left for the first pointing movement and to the right for the second, whereas the bottom panels depict those for opposite order of body rotation. If errors for the first and second movement are the same, based on a representation of the target relative to the finger generated during the first movement, then data should fall along a positive unit slope (gray dotted line, motor-memory hypothesis). But if the target location is recalculated based on the new gaze direction, then errors should land on either side of the target, and so fall along a negative unit slope (gray dashed line, reconversion hypothesis). The slopes for five of the six subjects were negative. But as in the body-stationary paradigm, these slopes fell well <1.0: the average slope was –0.13 (Fig. 12A) and –0.17 (Fig. 12B) when the body was initially rotated to the left and –0.22 (Fig. 12C) and –0.19 (Fig. 12D) when the body was initially rotated to the right. This suggests that when determining the target location or final endpoint for the second movement, this calculation was based mostly on where the target was relative to current gaze, but not exclusively. The general result is a diminished effect of current gaze on the second pointing movements following an intervening eye movement.

View larger version (30K): [in this window] [in a new window]   FIG. 12. Slopes for horizontal pointing errors for the second pointing movement as a function of the 1st (A and C) and as a function of the 2nd when the eyes were still (B and D) for each combination of initial fixation and target in the body-rotated condition. A and B are those slopes when the body rotated left to right, and C and D are those when the body rotated right to left. The predicted slope for the motor memory hypothesis is shown by the dotted line, while the predicted slope for the reconversion hypothesis is shown by the dashed line. The intercepts have been normalized to 0.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
In this study, we explored how different representations of the pointing target location subserve different stages of movement planning and execution. We did this by investigating how the brain codes for the final endpoint location for repeated pointing movements to the same remembered location. Specifically we tested whether the brain calculates this location relative to current gaze for every arm movement (i.e., the location is reconverted from an eye to an arm frame every time) or whether it simply relies on information used from the first movement to help guide the second (i.e., a converted representation relative to the hand). Until now, studies have only looked at how the brain programs single movements to a target. These studies suggest that the brain initially stores and updates the location of the remembered targets in an eye-fixed frame and then converts these signals to arm-motor coordinates prior to the action. The results from the current study suggest that even when a second movement is made to the same location, despite existing signals related to the previous movement, the second aiming movement is initially guided by an updated representation of the target relative to gaze, which is later transformed into an appropriate motor frame for executing the movement. That is, when subjects shifted their gaze across the target site between pointing responses, they tended to make errors that depended on final gaze direction as consistent with the retinal magnification effect and so fell on different sides of the remembered target for the second movement with respect to the first. This was also the case when the torso position changed between movements. But when gaze remained in the same direction throughout the trial, errors tended to fall on the same side of the target for both the first and second movements.

So why doesn't the arm motor system simply rely on an arm-related representation of the target and/or memory signals pertaining to the previous aiming movement? For instance, if there are neurons in the premotor cortex that have converted and stored information about the target in an arm-fixed frame, it is possible that these signals could also be used for a repeated movement to the exact same location several seconds later. Or perhaps memory traces of the efferent or afferent signals could be used to guide the second movement. Darling and Miller (1993) and Adamovich et al. (1998) have shown that people can be quite accurate at reproducing pointing movements to the same location. In these studies, subjects moved their unseen hand to the same endpoint location as their previous movement within 1.2–1.8° of the target. The reaching errors to kinesthetic targets were actually similar to or smaller than those made when the target was initially visible before the movement. These studies suggest that people are considerably accurate when reproducing pointing movements to the same location based on just the proprioceptive and efferent signals from the initial movement.

In our study, we also found that subjects were reasonably good at reproducing similar arm movements under similar pointing paradigms, i.e., when gaze did not shift. The average differences in pointing responses for the repeated arm movements to a target seen once only briefly were <1° when gaze was maintained either toward the target (Fig. 7B) or in some eccentric direction (Fig. 7, A and C) during both movements. Even when subjects rotated their torso, and so shifted their shoulders to a new position before each pointing movement, the average difference in finger position along the horizontal direction across the two movements was small: an average difference of 0.6 cm, which is equivalent to 0.6° when subjects rotated right to left and was as large as 2.1 cm (equivalent to 2°) when they rotated left to right. Even this larger difference between the two repeated movements is surprisingly small given that they required very different arm directions to get the finger within this distance. If information about the previous pointing movement is used to program a second movement, then we would expect that any intervening gaze shift between the repeated pointing movements should have no effect, and so the difference in pointing responses should be <2°. But instead, we find that the differences in pointing errors are usually two to three times larger when there is an intervening gaze shift compared with when there isn't one. More importantly, when gaze did shift between the pointing movements, the resulting errors usually fell on opposite sides of the remembered target, which was not the case when gaze remained in the same direction. As mentioned in the INTRODUCTION, many studies have shown that peripherally remapped targets are subject to the retinal magnification effect so that final pointing errors tend to systematically vary as a function of gaze (Beurze et al. 2006; Henriques et al. 1998; Medendorp and Crawford 2002; Van Pelt and Medendorp 2007) even for different types of targets (Poljac and van den Berg 2003; Pouget et al. 2002) and feedback conditions (Beurze et al. 2006; Vaziri et al. 2006). This study, however, shows that the remembered target continues to be remapped even after that first arm movement and that motor-related information from the previous movement is of little consequence when programming subsequent movements to the same site. This suggests that although an arm-fixed representation of the target's location may be computed for executing the movement, this representation does not seem to be reused (at least not entirely or exclusively) when planning repeated movements. Instead the whole reference frame transformation begins again, starting with the current eye-fixed representation of the target.

Several electrophysiological studies have suggested that a good portion of the neurons in the reach-related areas of the posterior parietal cortex (PPC) represent the reaching target location in a reference frame fixed to the eye or some combination of the eye and hand (Batista et al. 1999; Buneo and Andersen 2006; Buneo et al. 2002; Pesaran et al. 2006). Even downstream areas known to be involved in the later stages of reach planning, like the dorsal premotor cortex, have some neurons the activity of which can be best explained as representing the reach plan in a frame fixed to the eyes (Batista et al. 2007; Pesaran et al. 2006). These same studies also found neurons that code reach goals in hand or limb coordinates as well or in some relative coding scheme anchored to the hand. Human neuroimaging studies have also shown that certain areas of the PPC code and update the location of remembered targets relative to gaze when subjects were only attending to the stimulus (Merriam et al. 2003) and when they were planning to move the eyes or finger toward the target (Medendorp et al. 2003). In these studies, the remembered stimuli are represented on the contralateral side of PPC relative to gaze, so that if the target was seen to the left of gaze, the right PPC is active. More importantly, if subjects then shift gaze to the opposite side of the target site, so that now the remembered target falls to the right of gaze, the activity for that memory trace is shifted across to the other hemisphere so that now the left PPC becomes more active, suggesting that the representation of the target's location has been remapped as a function of gaze.

Human lesion studies have also reported a gaze-dependency effect on reach planning. When optic ataxia patients reach to either remembered visual and on-line proprioceptive targets, the magnitude of reaching errors depends on where the target is located with respect to gaze (Blangero et al. 2007; Khan et al. 2005a,2005b). That is, errors are much larger when patients reach to peripheral targets that are viewed in their damaged visual field compared with those made to the targets seen in the intact visual field. More interestingly, when patients view the target in their intact field, but shift their gaze to the opposite side of the remembered target so that this memory trace is remapped into the damaged visual field, errors are just as large as if they had viewed the target in the damaged visual field. On the other hand, when the intervening eye movement caused the target viewed in the damaged visual field to be remapped into the intact visual field, reaching was much more accurate.

Our results suggest that the brain also refers back to a remapped representation of the target site relative to gaze when programming repeated movements. Thus the transformation of the target's location from eye-centered coordinates to arm-centered coordinates would begin anew each time a subsequent arm movement was programmed. And this is the case even when the target was only seen once. It may be that the brain continues to revert back to an eye-fixed representation of the target's location even after many repeated movements to the same site. It is likely that eventually decay in the visual memory signal and accumulating noise involved in repetitive remapping across eye movements would result in this eye-centered representation of the target's location being less reliable, whereas the amount of motor-related information amassing from multiple arm movements may become a more reliable source for representation in programming further arm movements.

We originally proposed, as part of the motor-memory hypothesis, that if the remembered proprioceptive information from the preceding movement was used to program a second movement, then errors for the second pointing movement would resemble those of the first. Yet this would only be the case if proprioceptive memory signals were not also affected by the shift in gaze. But the reaching results from optic ataxia patients suggest that proprioceptive targets, like visual ones, are also remapped as a function of gaze (Blangero et al. 2007). Similar, neurons in the PRR have been found to code for hand location, as well as remembered target locations, in eye-centered coordinates (Buneo et al. 2002). Also recent work from our lab has shown that when subjects reach with their right hand to a remembered proprioceptive target (location of their unseen left hand), they make pointing errors that systematically overshoot the remembered target relative to gaze, much as they do for remembered visual targets (Henriques 2007). The implication for the current study is that the remembered location of the unseen hand during the first movement may also have been remapped (along with the remembered visual target) as a function of gaze. For example, if the arm was directed straight at the target (0°) during the first movement when gaze was aligned with the remembered site, shifting gaze to the left afterward may result in remapping the proprioceptive trace for that first movement to the right of gaze, where it would be subject to the magnification effect. So if the second movement was programmed based on the remapped felt location of the previous pointing movement, then errors for this second movement would fall to the right of those of the first. Thus the difference in pointing responses across the two repeated pointing movements in our study could be the result of remapping the location of both the visual target and the proprioceptive position of the first movement following an intervening eye movement.

Although our results suggest that the remembered target site is remapped and reconverted from eye to motor coordinates again when programming the repeated movement to that site, it is possible that motor-related information from the preceding arm movement is also partly used in combination with the remapped information. Despite a clear pattern of overshooting with respect to final gaze direction for the second pointing movements, the magnitude of gaze-dependent reaching errors (or the retinal magnification effect) was smaller for the second movement following an intervening eye movement compared with those made when gaze did not shift and when compared with the size of the errors made during the first pointing movement (Figs. 7 and 11). This explains why our slopes fitted to the reaching errors plotted in Figs. 8 and 12 fell below a negative unit slope. If errors were about the same size across the repeated movements, the slope should have been close to –1.0 rather than around –0.2. This decrease in the overall magnitude in the pattern of error for the second movement could be the result of motor-related information from the previous movement being used in combination with the updated eye-fixed representation of the target. Optimal motor planning tends to involve integrating multiple sources of information that are each weighted based on their reliability (Brouwer and Knill 2007; Sober and Sabes 2005; 2003; van Beers et al. 2002a,2002b; Vaziri et al. 2006). Our results could be explained if programming the second pointing movement involved combining both types of representations with greater weight placed on the target representation in eye coordinates than that for the representation in arm coordinates or for the motor-memory signals. Equal weighting would have resulted in flat curves in Figs. 7B and 11, B and E, and slopes of 0 in Figs. 8 and 12. An eye-fixed representation of the target may provide a more reliable source of information than a representation of the target in arm coordinates or motor memory signals (Vaziri et al. 2006), especially given that the brain likely devotes more time and resources to representing and updating space than to transforming specific objects into a particular limb or motor-related frame for action. Memory of afferent and efferent signals of the arm movement may also decay more rapidly than memory of a previously seen target (Ren et al. 2006), again suggesting that arm-related signals may be less reliable than visual spatial-memory signals for guiding repeated movements to the same spot. However, if the reduced effect of gaze on pointing errors for the second pointing movement, and the resulting smaller slopes, were due solely to the imbalance of weights given to these different representations of targets, then we should have expected less weight be given to a motor-related representation in the body-rotated paradigm. In this paradigm, the required arm posture differed across the two pointing movements, so memory traces of the proprioceptive and efference-copy signals from the first movement could not have been used to program the second movement the way they could have been when the body remained stationary. But the slopes for the body-rotated paradigm, on average, are similar to those in the body-stationary paradigm. Nonetheless, a representation of the target relative to the hand could still provide a reliable source of information for programming the second movement in both the body-stationary and -rotated paradigms, so may have made an equivalent contribution (1 smaller than of the eye-centered representation) to programming the repeated pointing movement.

Last, we would also like to offer an alternative interpretation of our results regarding how spatial information is represented for programming movements. Most current ideas resemble the scheme in Fig. 13 A: when an object is selected as the target of an action, its location is converted from the eye-fixed frame of spatial memory into a suitable motor frame, such as an arm-fixed frame. In other words, the brain computes a set of arm-muscle activations, or a set of arm-joint angles, that will place the hand at the target. But there is another possibility, shown in Fig. 13B: the brain never converts the target into a motor frame but instead converts feedback about the arm into the frame of spatial memory. This view has also been proposed by Shadmehr and Wise (2005). Both schemes—which we refer to as target conversion (A) and effector conversion (B)—are highly versatile (though in both, a good controller may need more information than just the error signal; and in both, there need not be a distinct subtraction step and error signal because in a neural network, conversion, comparator and controller may coalesce; the flow diagrams omit these variants for simplicity). Pure target conversion and pure effector conversion are two extremes on a continuum. The brain likely combines elements of both because they have complementary strengths. Very briefly: target conversion makes it easier to drive the arm to a specific set of joint angles, but effector conversion could explain motor equivalence in which different sets of joint angles are used to achieve the same hand position. And the effector scheme in Fig. 13B is, in one sense, more parsimonious than the target-conversion scheme (Fig. 13A): it removes the need for target conversion, while effector conversion is needed in any case to bring proprioception and efference-copy information into the frame of spatial memory, as we have tried to depict by the dashed line in Fig. 13B. However, our experiments cannot distinguish between these two possible schemes. Although the target conversion scheme tends be more widely accepted (even we adopted this viewpoint for the purposes of discussing our results), the effector conversion scheme is a viable perspective and a plausible explanation for our results.

View larger version (11K): [in this window] [in a new window]   FIG. 13. Flow diagrams illustrating 2 extreme ways in which the error signals may be computed. In the target conversion scheme (A), the target is stored in memory in an eye-fixed reference frame (and possibly converted into an eye-fixed frame in the case of auditory and tactile/proprioceptive targets) before it is converted or transformed in an arm-fixed frame (e.g., in joint coordinates). The error signal is the difference between the representations of the target in arm or joint coordinates and feedback of arm position coded in the same coordinates. In the effector conversion scheme (B), the error signal in this case is the difference between the representations of the target and of the arm in eye-fixed coordinates. In this case, feedback about arm position is converted from joint coordinates into an eye-fixed reference frame. Visual feedback of the hand would also lead to its representation in eye coordinates.

	GRANTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

	ACKNOWLEDGMENTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
The authors thank the following people for helpful suggestions on the manuscript: Drs. Patrick Byrne, J. Doug Crawford, Aarlenne Khan, Eliana Klier, and Douglas Tweed and A. Thompson. We thank J. Stemberger for technical support.

	FOOTNOTES

 
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Address for reprint requests and other correspondence: D. Y. P. Henriques, York University, School of Kinesiology and Health Science, Bethune College, 4700 Keele St., Toronto, Ontario M3J 1P3, Canada (E-mail: deniseh{at}yorku.ca)

	REFERENCES

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
Adamovich SV, Berkinblit MB, Fookson O, Poizner H. Pointing in 3D space to remembered targets. I. Kinesthetic versus visual target presentation. J Neurophysiol 79: 2833–2846, 1998.[Abstract/Free Full Text]

Batista AP, Buneo CA, Snyder LH, Andersen RA. Reach plans in eye-centered coordinates. Science 285: 257–260, 1999.[Abstract/Free Full Text]

Batista AP, Santhanam G, Yu BM, Ryu SI, Afshar A, Shenoy KV. Reference frames for reach planning in macaque dorsal premotor cortex. J Neurophysiol 98: 966–983, 2007.[Abstract/Free Full Text]

Beurze SM, Van Pelt S, Medendorp WP. Behavioral reference frames for planning human reaching movements. J Neurophysiol 96: 352–362, 2006.[Abstract/Free Full Text]

Blangero A, Ota H, Delporte L, Revol P, Vindras P, Rode G, Boisson D, Vighetto A, Rossetti Y, Pisella L. Optic ataxia is not only "optic": impaired spatial integration of proprioceptive information. NeuroImage 36, Suppl 2: T61–68, 2007.

Bock O. Contribution of retinal versus extraretinal signals towards visual localization in goal-directed movements. Exp Brain Res Exp Hirnforsch 64: 476–482, 1986.[CrossRef]

Boussaoud D, Bremmer F. Gaze effects in the cerebral cortex: reference frames for space coding and action. Exp Brain Res Exp Hirnforsch 128: 170–180, 1999.[CrossRef]

Boussaoud D, Jouffrais C, Bremmer F. Eye position effects on the neuronal activity of dorsal premotor cortex in the macaque monkey. J Neurophysiol 80: 1132–1150, 1998.[Abstract/Free Full Text]

Brouwer AM, Knill DC. The role of memory in visually guided reaching. J Vision 7: 1–12, 2007.

Buneo CA, Andersen RA. The posterior parietal cortex: sensorimotor interface for the planning and online control of visually guided movements. Neuropsychologia 44: 2594–2606, 2006.[CrossRef][Web of Science][Medline]

Buneo CA, Jarvis MR, Batista AP, Andersen RA. Direct visuomotor transformations for reaching. Nature 416: 632–636, 2002.[CrossRef][Medline]

Carrozzo M, McIntyre J, Zago M, Lacquaniti F. Viewer-centered and body-centered frames of reference in direct visuomotor transformations. Exp Brain Res Exp Hirnforsch 129: 201–210, 1999.[CrossRef]

Cisek P, Kalaska JF. Modest gaze-related discharge modulation in monkey dorsal premotor cortex during a reaching task performed with free fixation. J Neurophysiol 88: 1064–1072, 2002.[Abstract/Free Full Text]

Cohen YE, Andersen RA. A common reference frame for movement plans in the posterior parietal cortex. Nat Rev Neurosci 3: 553–562, 2002.[CrossRef][Web of Science][Medline]

Darling WG, Miller GF. Transformations between visual and kinesthetic coordinate systems in reaches to remembered object locations and orientations. Exp Brain Res 93: 534–547, 1993.[Web of Science][Medline]

Elliott D, Calvert R, Jaeger M, Jones R. A visual representation and the control of manual aiming movements. J Mot Behav 22: 327–346, 1990.[Web of Science][Medline]

Georgopoulos AP. Cortical mechanisms subserving reaching. Ciba Found Symp 132: 125–141, 1987.[Medline]

Georgopoulos AP. Spatial coding of visually guided arm movements in primate motor cortex. Can J Physiol Pharmacol 66: 518–526, 1988.

Georgopoulos AP, Stefanis CN. Local shaping of function in the motor cortex: motor contrast, directional tuning. Brain Res Rev 55: 383–389, 2007.[CrossRef][Medline]

Goodale MA, Milner AD. Separate visual pathways for perception and action. Trends Neurosci 15: 20–25, 1992.[CrossRef][Web of Science][Medline]

Henriques DY. Updating and integrating spatial information for motor control. In: ESF-EMBO Symposium: Three Dimensional Sensory and Motor Space: Perceptual Consequences of Motor Action. Sant Feliu du Guixois: 2007.

Henriques DY, Crawford JD. Role of eye, head, and shoulder geometry in the planning of accurate arm movements. J Neurophysiol 87: 1677–1685, 2002.[Abstract/Free Full Text]

Henriques DY, Klier EM, Smith MA, Lowy D, Crawford JD. Gaze-centered remapping of remembered visual space in an open-loop pointing task. J Neurosci 18: 1583–1594, 1998.[Abstract/Free Full Text]

Kalaska JF, Cohen DA, Hyde ML, Prud'homme M. A comparison of movement direction-related versus load direction-related activity in primate motor cortex, using a two-dimensional reaching task. J Neurosci 9: 2080–2102, 1989.[Abstract]

Khan AZ, Crawford JD. Coordinating one hand with two eyes: optimizing for field of view in a pointing task. Vision Res 43: 409–417, 2003.[CrossRef][Web of Science][Medline]

Khan AZ, Pisella L, Rossetti Y, Vighetto A, Crawford JD. Impairment of gaze-centered updating of reach targets in bilateral parietal-occipital damaged patients. Cereb Cortex 15: 1547–1560, 2005a.[Abstract/Free Full Text]

Khan AZ, Pisella L, Vighetto A, Cotton F, Luaute J, Boisson D, Salemme R, Crawford JD, Rossetti Y. Optic ataxia errors depend on remapped, not viewed, target location. Nat Neurosci 8: 418–420, 2005b.[Web of Science][Medline]

Lemay M, Bertram CP, Stelmach GE. Pointing to an allocentric and egocentric remembered target. Mot Control 8: 16–32, 2004.[Web of Science][Medline]

Lemay M, Proteau L. Aging affects pointing to unseen targets encoded in an allocentric frame of reference. Exp Aging Res 29: 319–333, 2003.[CrossRef][Web of Science][Medline]

McIntyre J, Stratta F, Lacquaniti F. Short-term memory for reaching to visual targets: psychophysical evidence for body-centered reference frames. J Neurosci 18: 8423–8435, 1998.[Abstract/Free Full Text]

Medendorp WP, Crawford JD. Visuospatial updating of reaching targets in near and far space. Neuroreport 13: 633–636, 2002.[CrossRef][Web of Science][Medline]

Medendorp WP, Goltz HC, Vilis T, Crawford JD. Gaze-centered updating of visual space in human parietal cortex. J Neurosci 23: 6209–6214, 2003.[Abstract/Free Full Text]

Merriam EP, Genovese CR, Colby CL. Spatial updating in human parietal cortex. Neuron 39: 361–373, 2003.[CrossRef][Web of Science][Medline]

Pesaran B, Nelson MJ, Andersen RA. Dorsal premotor neurons encode the relative position of the hand, eye, and goal during reach planning. Neuron 51: 125–134, 2006.[CrossRef][Web of Science][Medline]

Poljac E, van den Berg AV. Representation of heading direction in far and near head space. Exp Brain Res Exp Hirnforsch 151: 501–513, 2003.[CrossRef]

Pouget A, Ducom JC, Torri J, Bavelier D. Multisensory spatial representations in eye-centered coordinates for reaching. Cognition 83: B1–11, 2002.[CrossRef][Web of Science][Medline]

Ren L, Khan AZ, Blohm G, Henriques DY, Sergio LE, Crawford JD. Proprioceptive guidance of saccades in eye-hand coordination. J Neurophysiol 96: 1464–1477, 2006.[Abstract/Free Full Text]

Schlicht EJ, Schrater PR. Impact of coordinate transformation uncertainty on human sensorimotor control. J Neurophysiol 97: 4203–4214, 2007.[Abstract/Free Full Text]

Shadmehr R, Wise SP. The computational neurobiology of reaching and pointing: a foundation for motor learning. In: Computational Neuroscience. Cambridge, MA: MIT Press, 2005, p. 575.

Sober SJ, Sabes PN. Flexible strategies for sensory integration during motor planning. Nat Neurosci 8: 490–497, 2005.[Web of Science][Medline]

Sober SJ, Sabes PN. Multisensory integration during motor planning. J Neurosci 23: 6982–6992, 2003.[Abstract/Free Full Text]

Soechting JF, Flanders M. Errors in pointing are due to approximations in sensorimotor transformations. J Neurophysiol 62: 595–608, 1989.[Abstract/Free Full Text]

Soechting JF, Flanders M. Moving in three-dimensional space: frames of reference, vectors, and coordinate systems. Annu Rev Neurosci 15: 167–191, 1992.[CrossRef][Web of Science][Medline]

van Beers RJ, Baraduc P, Wolpert DM. Role of uncertainty in sensorimotor control. Philos Trans Roy Soc Lond 357: 1137–1145, 2002a.[CrossRef]

van Beers RJ, Wolpert DM, Haggard P. When feeling is more important than seeing in sensorimotor adaptation. Curr Biol 12: 834–837, 2002b.[CrossRef][Web of Science][Medline]

Van Pelt S, Medendorp WP. Gaze-centered updating of remembered visual space during active whole-body translations. J Neurophysiol 97: 1209–1220, 2007.[Abstract/Free Full Text]

Vaziri S, Diedrichsen J, Shadmehr R. Why does the brain predict sensory consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback. J Neurosci 26: 4188–4197, 2006.[Abstract/Free Full Text]

This article has been cited by other articles:

				A. A. Thompson and D. Y. P. Henriques Updating Visual Memory Across Eye Movements for Ocular and Arm Motor Control J Neurophysiol, November 1, 2008; 100(5): 2507 - 2514. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) All Versions of this Article: 99/6/2968    most recent 90225.2008v1 Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (2) Citing Articles via Google Scholar Google Scholar Articles by Sorrento, G. U. Articles by Henriques, D. Y. P. Search for Related Content PubMed PubMed Citation Articles by Sorrento, G. U. Articles by Henriques, D. Y. P.