A key feature of skilled motor behavior is the ability of the CNS to predict the consequences of its actions. Such prediction occurs when one hand pulls on an object held in the other hand; the restraining hand generates an anticipatory increase in grip force, thereby preventing the object from slipping. When manipulating a novel object, the CNS adapts its predictive response to ensure that predictions are accurately tuned to the dynamics of the object. Here we examine whether learning to predict the consequences of an action on a novel object is restricted to the actions performed during manipulation or generalizes to novel actions. A bimanual task in which subjects held an object in each hand and the relationship between actions on one object and the motion of the other could be computer controlled from trial-to-trial was used. In four conditions we varied the spatial relationship between the direction of force subjects applied to the left-hand object and the consequent direction of motion of an object held in their right hand, which subjects were required to restrain. The results show that predictive learning was local to the direction of forces experienced during learning and that the magnitude of predictive responses was greatly reduced for novel directions of action of the left hand. The pattern of generalization shows that the representation of predictive learning is spatially local and can be approximated as having a spatially narrow Gaussian basis function.
Many skills depend on our ability to anticipate the consequences of our own actions (Johansson and Cole 1994; Massion 1992). For example, when we hold an object in a precision grip between the thumb and forefinger, sufficient grip force (perpendicular to the surfaces) must be generated to prevent the object from slipping (Johansson and Westling 1984; Johansson and Cole 1992; Johansson et al. 1992). The required grip force depends on both the load force (tangential to the surfaces) that the object exerts on the fingers and the frictional properties of the object's surface. When the load on the object is increased by an external agent, grip force lags the load force by around 100 ms (Johansson et al. 1992). However, if the load force is self-generated, for example, by pushing on the object using the other hand, then grip force anticipates the load force with near zero delay, suggesting a predictive process (Johansson and Cole 1992). Anticipatory grip force modulation has been shown to be scaled to object weight (Johansson and Westling 1988), texture (Johansson and Westling 1984), shape (Jenmalm and Johansson 1997), center of mass (Wing and Lederman 1998), and previous experience (Gordon et al. 1993). Such prediction is not hard-wired but learned through development (Forssberg et al. 1991). Moreover, in the adult, predictions to novel consequences of actions can be learned (Flanagan and Wing 1997b; Witney et al. 1999, 2000).
Anticipation may rely on an internal forward model of both one's own body and the external world to capture the causal relationship between actions, as signaled by efference copy (Jeannerod et al., 1979; Sperry 1950; von Holst 1954), and their consequences (Flanagan and Wing 1997a; Kawato et al., 1987; Jordan and Rumelhart 1992; Jordan 1995; Miall and Wolpert 1996; Wolpert 1997; Wolpert et al. 1995).
Here we examine how the spatial properties of an object are represented during predictive learning. The predictive system could represent a global prediction of the dynamics of a manipulated object or, alternatively, the learned prediction could be restricted by the spatial features that have been experienced. How the spatial properties of the predictive grip force response are represented within the CNS has important implications for functional abilities. The nature of the representation determines how adaptable different features of the prediction are and, therefore, the flexibility of this aspect of skilled object manipulation. Generalization paradigms have previously been used to examine the representation of internal models. The generalization paradigm can be summarized by two main features. First, subjects are exposed to novel input–output associations over a limited region of input space. After learning of this association, the generalization of learning can be examined by testing subjects on their input–output mapping on the full region of input space. The pattern of generalization outside of the learned region reflects the structure and constraints underlying the learning system (Ghahramani et al. 1996; Ghahramani and Wolpert 1997). Previous studies have used generalization paradigms to probe the representation of both the visuomotor transformation (Imamizu et al. 1995,Ghahramani et al. 1996,Ghahramani and Wolpert 1997,Vetter et al. 1999) and control processes (Gandolfo et al. 1996; Shadmehr and Mussa-Ivaldi 1994). However, the generalization of the prediction of the consequences of action enabling skilled object manipulation has not been examined. Grip force modulation, in which anticipatory responses develop to self-generated perturbations in load force, are used in this study to examine how the predictive learning necessary for grip force control is represented.
We used a bimanual task in which subjects pushed on a fixed object held in their left hand while restraining an object held in their right hand (Fig. 1). The object in their right hand was attached to a torque motor that could generate an upward or downward load force on this object. On each trial subjects pushed on the left-hand object in one of eight possible directions. On “linked” trials this caused a load force of equal magnitude, but always in the vertical direction, to be transmitted to the object held in the right hand. On “unlinked” trials no load force was generated on the right-hand object.
Several different training paradigms were used: linked trials in all directions, in only one direction, and in two opposite directions. To examine generalization, unlinked trials were experienced in all eight directions. As no load force is generated on the fingertips of the right hand in these trials, the grip force response seen reflects a purely predictive response (Witney et al. 1999) and therefore can be used to assess generalization to directions not experienced during learning.
Six subjects (3 male, 3 female; all right-handed) gave informed consent and participated in the study. Subjects were naive to the purposes of the experiment. None of the subjects reported any sensory or motor deficits.
A local ethics committee approved the experimental protocol.
Subjects held separate cylindrical objects with each hand using a precision, thumb–index finger grip (Fig. 1 A). The cylinders had two parallel suede-covered grip surfaces with a diameter of 30 mm and with a separation of 40 mm. A 6-axes force transducer (Nano, ATI Inc.) was embedded in each cylindrical object with the mass of the transducer centered midway between the surfaces. The total mass of each object was 50 g. The force transducer allowed three translational forces to be measured with an accuracy of 0.05 N including cross talk. The object held in the left hand was made immovable by fixing it to a solid support. The right-hand object was attached by an aluminum rod (length 50 mm) to a torque motor that was under robotic control (Phantom Haptic Interfaces, Sensable Devices). Vertical forces could be generated on this object in a computer-controlled fashion with an update rate of 1,000 Hz. The mechanical bandwidth of the system was 65 Hz (where the gain dropped to 1/ ). The subjects' right forearms were anchored with velco straps and, for further stability, they grasped a vertically oriented aluminum rod with their three ulnar fingers. A horizontal wooden rod was then positioned over the gripped right-hand index finger and thumb. These measures ensured that the subject's thumb and index finger were used to maintain object stability rather than a more general postural response.
Before each trial subjects were provided with a visual display on a computer screen to ensure that the bar attached to the right-hand object was horizontal. Subjects were instructed to keep their right hand still, preventing the gripped object from slipping from their grasp. The start of each trial was signaled by a tone and occurred approximately every 3 s. On hearing this tone, subjects were required to generate a force pulse on the left-hand object in a direction indicated on a computer screen. The force pulse was required to start and end with near zero load-force and to reach a magnitude of 6 ± 1 N. To guide the subjects to both magnitude and direction, the load vector on the left-hand object, that is the horizontal and vertical components of the load, was displayed in real-time as a two-dimensional position of a cursor on a computer screen. A target zone was displayed with boundaries that were ±10° of the desired angle and ±1 N of the desired amplitude. The direction for each trial was chosen from eight possible directions equally spaced at 45° intervals from 0°, which represented vertical (Fig. 1 B). If the load generated by the subject did not fall within the desired limits, a tone sounded to indicate failure of the trial, although the trial was not repeated.
There were three possible consequences to the force pulse delivered by the subjects on the left-hand object. First, on “unlinked” trials, the torque motor generated no force and therefore the right-hand object did not move. Second, on “linked-up” (U) trials, the torque motor generated a vertical upward force of equal magnitude to the force generated on the left-hand object. Third, on “linked-down” (D) trials, the force generated on the right-hand object was in the vertical downward direction.
To prevent any prior knowledge of whether the trial was linked or unlinked, based on cues from accidental small movements of the left hand, the force on the right hand was zero until the tone in all trials. To prevent fatigue, short rest periods were given every 40 trials in all conditions.
Each subject completed four conditions comprising three generalization conditions followed by a control condition. The order of the three generalization conditions were randomized between subjects. The control condition was performed after the generalization conditions to avoid transfer effects to the generalization conditions.
For the three generalization conditions, subjects performed 180 trials. All conditions consisted of 60 training trials followed by a test period of 120 trials. For condition L0 all the training trials involved subjects generating pulses with the left hand to a target at 0° and each trial was a link-up trial. Therefore, on each training trial, the action of the left hand always generated a load force on the object held in the right hand (Fig. 1 B). The test phase used unlinked trials to test generalization of learning to the eight directions of action of the left hand. Trials were presented in batches of three, comprising two trials from the training phase followed by an unlinked test trial in one of eight directions. The directions on these trials were pseudorandomly chosen from the eight possible directions ensuring that five unlinked trials for each direction were obtained. Trials from the training phase were interleaved in this way to prevent decay of any learning. For condition L0D180 the training phase consisted of subjects generating the left force pulses at 0° and 180°, which causes a linked-up and linked-down trial, respectively. This is the situation that might be expected for a real physical object held between the hands. In the L0L180 condition subjects generated left force pulses at 0° and 180°, which causes a linked-up trial in both situations.
A control condition, Lall (linked in all directions), was used to examine whether subjects could generate predictive grip force when arbitrary directions of load in the left hand were associated with a single direction of load in the right hand. This condition was performed last by all subjects so that experience of this condition would not influence the subject's performance in the three generalization conditions. Sixty-four training trials were presented, which consisted of eight repetitions of linked-up trials in the eight different directions in a pseudorandom order. This training period was followed by a test period in which 120 trials were performed in batches of three. Within each batch two linked-up trials were followed by an unlinked test trial. The direction of the unlinked test trial was pseudorandomly varied, with each of the eight directions occurring five times on average.
For each trial the position of the right hand, the grip force and load force on both objects were recorded at 200 Hz. To quantify the magnitude and timing of anticipatory grip force, the amplitude and timing of the peak grip force was found for each trial. Maximum grip force modulation was taken as the difference between the maximum grip force (peak within a 400-ms window on either side of the maximum left-hand load force) and the baseline grip (average value of the grip force in the first 100 ms of each trial). The grip force lag was calculated as the difference between the time of the peak grip force and the time of peak left-hand load force (with negative values indicating grip force precedes peak left-hand load force).
Multivariate ANOVAs (MANOVAs) were used to compare the response on linked and unlinked trials across the four conditions. Further MANOVAs, with posthoc tests, were used to examine differences between the magnitude of grip force modulation at the training angles of 0° and 180° with the grip force response on the other unlinked generalization trials. Differences in the magnitude and timing of peak grip force modulation on the unlinked generalization trials were examined. Further MANOVAs were used to compare the magnitude and timing of the grip force modulation on the training trials of L0D180 and L0L180 conditions. To examine learning effects, the first 10 trials in the training phase in each condition were compared with the last 10 training trials of each condition.
Subjects found the task easy to perform and were able to produce consistent and accurate load forces on the left-hand object. Load force trajectories averaged for each target direction are shown in Fig.2.
In the training trials of the L0 condition a force pulse generated by the left hand in the 0° direction was associated with an upward force on the object in the right hand. On these trials the mean grip force modulation was 8.3 N, and the peak lagged the peak load force by 7.3 ms (Fig.3 A). The grip force response to unlinked test trials was greatest at 0° (the linked training angle), with an average grip force modulation of 3.3 N (Fig.4 A). Grip force modulation decreased significantly (P < 0.01) when the motion of the left hand was not in the training direction (1.4 N at 45° and 1.0 N at 315°). The peak grip force modulation on the unlinked test trial to the 0° direction was significantly smaller in amplitude with a reduced latency. This is consistent with the findings from previous studies (Witney et al. 1999, 2000).
To examine whether the specificity of grip force response was due to a suppression from repeated experience of unlinked trials we examined the first experience of each unlinked trial. Figure5a shows a similar pattern in the average response on the first unlinked trial, suggesting that this pattern is not driven by the experience of unlinked trials in the test phase. Similar patterns of generalization were also seen in individual subject data (not shown).
In the training trials of the L0D180 condition a force pulse generated by the left hand in the 0° and 180° directions was associated with an upward force and downward force, respectively, on the object in the right hand. In these trials the mean grip force modulation for the 0° direction was 7.7 N, which lagged the peak load force by 4 ms. In the 180° direction, the average grip force response was 6.7 N, with a lag of 12.6 ms. There were no significant differences between these values for the two types of training trials (Fig.3 B).
In the test phase the grip force response to unlinked trials to 0° and 180° (the linked training angles) was not significantly different, with modulation of 4.7 and 3.7 N, respectively (Fig.4 B). Grip force modulation decreased significantly (P < 0.01) when the motion of the left hand was not in the training direction.
The maximum grip force modulation of subjects on the first instance of an unlinked test trial, after the training trials, can be seen in Fig.5 B. As with the L0 condition, the grip force response on the first occurrence of an unlinked test trial is similar to the pattern of response at the end of test trials. Individual data show the same pattern of generalization as the averaged data (not shown).
In the training trials of the L0L180 condition a force pulse generated by the left hand in both the 0° and 180° directions was associated with an upward force on the object in the right hand. In the training trials, the average grip force response to the 0° and 180° directions was not significantly different in amplitude or timing. At 0°, average grip force modulation was 8.7 N and in advance of the peak load force by 15.1 ms. The grip force response to the 180° direction was an average of 6.4 N and lagged the peak load by 15.1 ms (Fig. 3 C).
No significant differences were found in the magnitude and timing of grip force modulation between condition L0L180 and condition L0D180 in early or late training training trials.
The grip force response on unlinked test trials was greatest at 0° and 180° (the linked training angles), with an average grip force modulation that was not significantly different from each other, at 5.0 and 4.0 N, respectively (Fig. 4 C). Grip force modulation decreased significantly (P < 0.01) when the motion of the left hand was not in the training direction; hype force modulation in the right hand decreased significantly (P < 0.01) when the motion of the left hand was not in the training direction. The maximum grip force response of subjects after experiencing linked trials at 0° and 180° on the first instance of an unlinked test trial can be seen in Fig. 5 C. As with the other conditions, the grip force response on the first occurrence of an unlinked trial is similar to the response after all of the unlinked test trials. Individual data show the same pattern of generalization as the averaged data (not shown).
In the training trials of the Lallcondition, a force pulse generated by the left hand in any of the eight possible directions was associated with an upward force on the object in the right hand. Over all directions the mean grip force modulation was 6.7 N with a mean lag to peak load force of 8.5 ms. In the unlinked trials of the test phase a predictive grip force response occurred at all angles (Fig. 4 D). As there is no load force on the right-hand object during these unlinked test trials, this is a predictive response. For these unlinked trials the average grip force modulation was 2.7 N, with a peak that preceded the peak load force on the left hand by 32.8 ms. The maximum grip force response of the subjects after experiencing linked trials in all directions on the first instance of an unlinked test trial can be seen in Fig.5 D.
In this study we have examined the predictive responses after experiencing different dynamic relationships between two objects held bimanually. In condition L0 just one direction was repeatedly presented during training, with the seven directions not experienced during the training phase used to probe the generalization of predictive responses in the test phase. As expected, a predictive response (the aftereffect) was seen to the training direction. Predictive grip force responses were also seen in the other directions not experienced during the training (Fig. 4 A), with predictive grip force response decaying from the direction of training. Therefore, the representation of predictive learning appears to be specific to a local region of angular space; the direction of force experienced in training. As the training does not specify the behavior of the object to the novel directions of action in the test phase, the pattern of generalization reflects the structure and constraints underlying the predictive learning (Ghahramani et al. 1996; Ghahramani and Wolpert 1997). This generalization was shown not to be due to the unlearning of a predictive response. Previously we have shown that predictive grip force is still present after a linked trial, even after six sequential unlinked trials (Witney et al. 2000). Additionally, the magnitude of the predictive component of the grip force response has been previously shown not to be scaled by subjects using sequence information to predict either presence or absence of loading but was systematically dependent on the manipulative history (Witney et al. 2000, 2001).
The predictive model was able to capture complex and non-local relationships between actions and their consequences. In both the L0D180 and L0L180 conditions subjects were trained on linked trials at two disparate angles, with forces experienced that were compatible and incompatible with a real physical object, respectively. The predictive grip force response was learned locally to the direction of forces experienced during training, with no significant differences between the conditions. A control condition, in which training was given on all eight directions of applied force, confirmed that anticipatory modulation of grip force in the right hand can develop to an arbitrary direction of load force on the left-hand object.
To examine whether the pattern of generalization could be modeled as a combination of local basis functions, we assumed that the modulation had a Gaussian response (with nonzero base) as a function of angular distance from a linked trial direction. Therefore the peak response is expected at the training direction with decay to other directions. The shape of the basis function was fixed for all conditions, and only the overall amplitude was scaled for each group of subjects to allow for inter-subject variability in the grip force levels produced. When more than one training direction was used, the basis function for each was simply summed. Figure 6 shows the observed average and fits using a Gaussian radial basis function with SD of 27.5°. This shows a good qualitative fit to the data, suggesting that predictive learning may be constructed by the combination of such predictive basis functions.
Generalization studies have been used previously to examine several transformations. For example, in a study of the visuomotor transformation (Vetter et al. 1999), subjects were trained on highly localized remappings between actual and displayed finger position during a pointing task. On testing, remapping had occurred across the workspace, with no significant decay. A global pattern of motor adaptation has also been shown in the generalization of learning novel dynamics (Shadmehr and Mussa-Ivaldi 1994). In their study, adaptation and generalization to velocity-dependent force fields during movements to targets was examined. Exposure to a force field in the left side of the workspace was found to generalize to the right side of the workspace, with the generalization being best explained in terms of joint-based coordinates. However, when dynamic learning was limited to individual movements, adaptation was found to be local in the workspace (Gandolfo et al. 1996; Sainburg and Ghez 1995). This is consistent with our study of the representation of predictive control of grip force; training was limited to individual directions of load force generation and the learning of this predictive control was found to be local in angular space. However, a fundamental difference between these studies and ours is that, in the studies of force field learning, the forces experienced are arbitrarily dependent on the state (velocity) of the hand. In our study the forces experienced in several of the conditions were consistent with a very common situation of a physical object held between the hands, yet learning was still local.
In the present study, we have used generalization of a predictive response, grip force modulation, to examine the representation of the predictive learning. Although learning of the predictive control of grip force has been examined, its generalization properties have previously not been examined. Previous studies of grip force learning have shown that grip force levels can be set without somatosensory feedback, anticipating the physical properties of the object, which include the object's weight, shape, and friction at its surface (Jenmalm and Johansson 1997; Johansson and Westling 1984, 1988; Johansson and Cole 1994). Such object properties are learned through development, indicated by increasing ability to correctly parameterize grip force to the object being manipulated (Eliasson et al. 1995;Forssberg et al. 1991, 1992, 1995). The anticipatory grip force response has been found to adapt to novel experiences. For example, the predictive grip force that occurs during rapid arm movements, anticipating the increase in load force that occurs when an object is accelerated (Flanagan and Wing 1993, 1995), adapts to altered movement dynamics. When the arm is subjected to inertial, viscous, or elastic forces during the movement, grip force prediction adapted to the new loads (Flanagan and Wing 1997b). After a few trials, grip force increased in parallel with the increase in load force, as had occurred before the additional force was added. This finding indicates that the commands for grip force adjustments are not rigidly associated to those for arm movements, but instead the anticipatory grip force response is based on an adaptable internal model of both the motor apparatus and the external load. The predictive response is also able to learn temporal delays between action and consequence. Using the virtual object paradigm, a delay of 250 ms was added to the time between the action of one hand and the consequence of a load force on the restraining hand (Witney et al. 1999). A novel predictive response was found to slowly develop to the delayed load force over several hundred trials. In contrast to this slow adaptation to a temporal delay, learning of a predictive grip force response in a virtual object with no delay builds up quickly, with substantial prediction after one experience of an association between action and consequence (Witney et al. 2000).
This study showed that adaptation to altered spatial features occurs rapidly. This, combined with the finding that the representation of spatial information is local to the direction of training, demonstrates the flexibility of the predictive grip force response to learning the spatial dynamics of objects. This flexibility underlies our ability to skillfully manipulate objects where there are many directions of experienced force at the fingertip.
In this study we were able to probe the representation underlying the predictive grip force control using a generalization paradigm. Our study shows that generalization of the anticipatory grip force response from predictive learning occurs locally to the direction of force experienced during bimanual manipulation. The pattern of generalization shows that the representation of predictive learning is spatially local and can be approximated as having a spatially narrow Gaussian basis function.
We thank James Ingram for technical and programming assistance.
This project was supported by grants from the Wellcome Trust, Medical Research Council, Biotechnology and Biological Sciences Research Council and Human Frontiers.
Address for reprint requests: D. Wolpert, Sobell Department of Motor Neuroscience, Institute of Neurology, University College London, Queen Square, London WC1N 3BG, United Kingdom (E-mail:).
- Copyright © 2003 The American Physiological Society