| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1ATR Computational Neuroscience Laboratories, Kyoto 619-0288, Japan; 2School of Kinesiology, Simon Fraser University, Burnaby, British Columbia V5A 1S6, Canada; and 3Department of Mechanical Engineering and Division of Bioengineering, National University of Singapore, 119260, Singapore
Submitted 11 December 2002; accepted in final form 16 February 2002
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
; Flanagan and Wing 1997; Flanagan et al. 2001; Krakauer et al. 1999; Lackner and Dizio 1994; Shadmehr and Mussa-Ivaldi 1994; Thoroughman and Shadmehr 1999). Studies of individuals performing goal-directed movements in novel mechanical environments have shown that the CNS acquires internal models of the external world (Kawato 1999). However, most adaptation studies have employed paradigms that involve stable interactions with the environment, whereas many tasks that humans perform, particularly those involving tool use, are inherently unstable (Rancourt and Hogan 2001).
Adaptation to perturbations that do not compromise mechanical stability appears to involve the acquisition of an inverse dynamics model through feedback error learning (Kawato et al. 1987). However, conventional feedback error learning does not address the issue of modifying mechanical impedance to counteract mechanical instability, although this has been repeatedly observed (Akazawa et al. 1983; Burdet et al. 2001a; De Serres and Milner 1991; Milner 2002; Milner and Cloutier 1993, 1998; Milner et al. 1995).
When the dynamics do not induce mechanical instability, changes in muscle activation patterns closely follow adaptive changes in joint torques although there is excess activation, particularly in the early stages of learning (Milner and Cloutier 1993; Thoroughman and Shadmehr 1999). Agonist-antagonist muscle co-contraction was found to increase on exposure to the novel dynamics and then decrease as learning progressed. The ability to co-contract specific groups of muscles would permit selective changes to the geometry of the endpoint stiffness (Hogan 1985). Recently we were able to demonstrate that the magnitude, shape, and orientation of the endpoint stiffness of the arm can be controlled in a predictive way to compensate for environmental instabilities (Burdet et al. 2001a).
Previous research has suggested the existence of two separate motor control mechanisms: inverse dynamics models and impedance control. An inverse dynamics model is a controller that computes feedforward commands of the net joint torques for movement based on the estimated effects of internal and external dynamics. An impedance controller, in contrast, modifies the impedance of the limb by co-contraction of agonist and antagonist muscles without changing net joint torque. These two controllers can operate independently. While there is no direct evidence for (or against) separate brain mechanisms being implicated in these two types of control, they perform different functions and therefore should be considered as separate controllers. Different features of our visual experience, for example, color, contour orientation, motion, and retinal disparities, are not independently encoded at the sensory level but are represented separately in the visual cortex (Ts'o and Roe 1994). Functionally, these features of the visual world provide very different information about our environment, and so the brain processes each separately. It seems possible, therefore that functionally distinct features of motor output might be represented and processed in a similarly independent fashion by the motor system. Some evidence already exists that control of reciprocal activation and co-contraction occurs in separate areas of the cortex (Humphrey and Reed 1983). Recently both Takahashi et al. (2001) and Osu et al. (2002) have provided evidence that inverse dynamics models and impedance control operate as separate mechanisms for motor control.
The present study investigated adaptation to novel force fields, in which trajectories were initially stable or unstable, to compare features of inverse dynamics model formation and impedance control. Adaptation to the force field, in which hand trajectories were stable, could be achieved by simply modifying joint torque. An inverse dynamics model alone was suffi-cient to compensate for the perturbing effects of this force field (Franklin et al. 2003). Adaptation to the other force field, in which hand trajectories were unstable, required an increase in the endpoint impedance of the arm, but no change in net joint torque, i.e., only impedance control (Burdet et al. 2001a; Franklin et al. 2003). However, we hypothesized that both of these processes are generally active during learning. To test this hypothesis, we analyzed the time course of changes in muscle activation patterns as well as joint torques, during the learning of novel force fields. In particular, we were interested in comparing the extent to which the CNS used generalized muscle co-contraction in the early stages of learning to increase endpoint stiffness and how patterns of muscle activity were later refined. Two parallel processes were identified from the evolution profile of the muscle activation patterns. One was an activation process involved in increasing the endpoint stiffness of the arm by means of muscle co-contraction during the early stages of learning. The other was a deactivation process, which led to gradual reduction in muscle activity as learning progressed.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
Apparatus
Subjects sat in a chair and moved the parallel-link direct drive air-magnet floating manipulandum (PFM) (Fig. 1) in a series of forward reaching movements performed in the horizontal plane. Their shoulders were held against the back of the chair by means of a shoulder harness. The right forearm was securely coupled to the PFM using a rigid custom-molded thermoplastic cuff. The cuff immobilized the wrist joint, permitting movement of only the shoulder and elbow joints. The subjects' right forearm rested on a support beam projecting from the handle of the PFM. Motion was, therefore limited to a single degree of freedom at the shoulder and at the elbow. The manipulandum and setup are described in detail elsewhere (Gomi and Kawato 1996, 1997).
|
Force fields
The experiment examined trajectory and electromyographic (EMG) adaptation in two force fields (Fig. 1, B and D): a velocity-dependent force field (VF), in which hand trajectories were stable from the outset, and a position-dependent (divergent) force field (DF), in which hand trajectories were initially unstable. Results were compared with those in a null field (NF). Details of the implementation and protocol have been described elsewhere (Burdet et al. 1999, 2001a). Only a general overview is given here. The force (Fx, Fy) (in N) exerted on the hand by the robotic interface in the VF was implemented as
 | (1) |
, 
) is the hand velocity (m/s) and the scaling factor, 
, was adjusted to the subject's strength (2/3 
1). The stability of the initial trajectories was tested by applying small perturbations during before effect trials and comparing the resulting trajectories (Fig. 1C). While subjects were performing movements in the NF, the force field was switched to the VF on random trials (before effects). On some of these trials, a brief triangular force pulse (25-ms duration) with an amplitude of 15 N was applied to the hand 100 ms after movement onset. Both the perturbed and unperturbed before effect trajectories are perturbed by the force field. However, all trajectories converge illustrating that they are stable. This result is consistent with the defining characteristic of Lyapunov stability that the addition of a small perturbation does not produce divergent behavior.
The DF produced a negative elastic force perpendicular to the target direction with a value of zero along the y axis, i.e., no force was exerted when trajectories followed the y axis, but the hand was pushed away whenever it deviated from the y axis. The DF was implemented as
 | (2) |
= (300 to 500; N/m) was adjusted for each subject so that it was larger than the stiffness of the arm measured in NF movements so as to produce instability. For safety reasons, the DF force field was inactivated if the subjects' trajectory deviated more than 3 cm from the y-axis. Both force fields were inactivated once the subject reached the target position. Learning
All subjects practiced in the NF on at least 1 day prior to the experiment. These training trials were used to accustom the subjects to the equipment and to the movement speed and accuracy requirements. Subjects were randomly assigned to one of two groups. Group 1 adapted to the DF on one day and adapted to the VF on another day, whereas group 2 adapted to the fields in reverse order.
Subjects first practiced in the NF until they had achieved 50 successful trials. Successful trials were those which ended inside a 2.5-cm-diameter target window within the prescribed time (0.6 ± 0.1 s). All movements were recorded whether successful or not. The movement distance was 0.25 m. Movements were self-paced so subjects were able to rest between movements if they wished. At the completion of 50 successful trials, the force field was activated. No information was given to the subjects as to when the force field trials would begin. Subjects then practiced in the force field until achieving 75 successful trials. They took a short break and then performed 100 more movements, 20 of which were random trials in the NF. The NF trials were called after effects and were recorded to confirm that subjects had adapted to the force field.
Hand-path errors
The adaptation to the force fields was quantified by calculating the error relative to a straight line joining the centers of the start and target circles. The absolute hand-path error
 | (3) |
 | (4) |
).
The hand-path error for each subject was fit with an exponential curve using a least-square error method. This fitted error was expressed as
 | (5) |
is the time constant, C is the constant error, and t refers to the trial number. Torque estimation
Time varying muscle torque at the shoulder and elbow was computed using the equations of motion for a two-link planar arm (cf. Hollerbach and Flash 1982). However, the equations used also include the contributions to joint torque from external forces applied at the hand. The joint torque was calculated as
 |
 |
is joint torque, 
is joint angle (defined as in Fig. 1), I is moment of inertia about the center of mass (cm) of the segment, l is segment length, and m is segment mass. The subscript 1 refers to the upper arm, 2 to the forearm, s to the shoulder, e to the elbow, and c to the wrist cuff. The mass and inertia of the subject's arm segments were estimated from the weight and segment lengths of each subject based on anthropometric scaling relations (Winter 1990).
Joint torque during learning was further quantified by estimating the variation in torque on a particular trial relative to the joint torque after adaptation to the force field. The learned joint torque profile was calculated using the mean of the last 20 successful trials during the learning. The absolute torque error (|error|) represents the total difference between the joint torque on a particular trial and the mean final joint torque both in terms of extent and timing. It is calculated as
 | (7) |
i is the joint torque on a given trial, m is the mean joint torque after adaptation, the superscript j refers to either the shoulder or elbow joint, t0 is the time of movement onset, and tf is the end of the time of interest. The torque error was estimated using a 1 s interval. The absolute torque error was fit with an exponential curve (Eq. 5) using least-squares similar to the hand-path errors.
To examine the development of the torque during the first few trials in the DF, we used two measures of torque change relative to the torque in the NF. The first measure, absolute torque development, was calculated as in Eq. 7 but with m referring to the mean joint torque in the NF. A second measure termed signed torque development (develop) was calculated as
 | (8) |
m again refers to the mean joint torque in the NF. Both the absolute and signed torque development were calculated over the initial 400 ms to avoid the influence of corrective movements or the safety boundary. Electromyography
Surface EMG activity of six arm muscles was recorded using pairs of silver-silver chloride surface electrodes during the learning sessions. The electrode locations were chosen to maximize the signal from a particular muscle while avoiding cross-talk from other muscles. The skin was cleansed with alcohol and prepared by rubbing in electrode paste. This was removed with a dry cloth, and pre-gelled electrodes were then attached to the skin with tape. The spacing between the electrodes of each pair was approximately 2 cm. The impedance of each electrode pair was tested to ensure that it was less than 10 k
.
The activity of two monoarticular shoulder muscles, pectoralis major and posterior deltoid, two biarticular muscles, biceps brachii and long head of the triceps, and two monoarticular elbow muscles, brachioradialis and lateral head of the triceps, was recorded. The EMG signals were analog filtered at 25 Hz (high-pass) and 1.0 kHz (low-pass) using a Nihon Kohden amplifier (MME-3132) and then sampled at 2.0 kHz. All comparisons between force field EMG and NF EMG involved data recorded on the same day without removal of the electrodes. EMG was aligned on the movement onset and averaged over 20 trials to visually compare changes that had occurred during learning.
To quantify changes during learning, the root-mean-square (rms) value of the EMG was calculated from 100 ms before movement onset until 350 ms after movement onset to include all the early movement associated activity. The muscle activity during this time will include both the feedforward command and the reflex responses. We did not extend the time interval further to avoid the confounding effects of voluntary corrective actions that redirect the limb toward the target after being perturbed by the force field during the early phase of learning. In all cases, the recorded EMG data were characterized by an initial increase in activity followed by a more gradual decrease. Both processes appeared to be exponential. To compare the time course of the changes in EMG with the time course of other kinematic and dynamic parameters, we fit this data with a model. Our model expressed EMG as a function of trial number using a double exponential with four free parameters
 | (9) |
1 and 2 represent the time constants, C is EMG at trial 0 (fixed as the mean of the EMG in the NF), and t is the trial number. One process (the 1st term) represents an exponential increase in EMG in response to exposure to the force field. It is bounded by the maximal level of activation that is possible (A + C) that has been shown to be significantly lower in co-contraction than reciprocal activation (Milner et al. 1995). The other process (the 2nd term) represents an exponential decrease in EMG. For comparisons across subjects, the EMG was normalized such that the mean NF EMG (C) was equal to a value of one. This model corresponds well with the recent work of Osu et al. (2002) that describes EMG activity during motor learning as a gradually decreasing function with occasional increases or decreases that are related to the error in previous trials. To provide further support for the choice of this model, Akaike's Information Criterion (AIC) (Akaike 1974) was calculated for this and two simpler models (APPENDIX). The time course of the EMG was compared with the time course of the hand-path error and absolute joint torque error. The hand-path error was calculated by one of two methods. In the DF, the absolute hand-path error S(|ex|), a measure of the area between the actual movement path and the straight line joining the start and end targets, was used. In the VF, the signed hand-path error S(ex), a measure of the mean directional extent by which the path deviates from the straight line, was used. Although different representations of the error were used for the DF and VF, we could have used absolute hand-path error for both DF and VF without changing any of the results or conclusions. The hand-path error and joint error for each subject were fit with an exponential curve using a least square error method. The time constants of these changes were then compared with the time constants of the change in EMG during the adaptation process using t-tests with a significance level of 0.05. These statistics were performed on the reciprocal of the time constants, i.e., the rates, to reduce the variance of processes that exhibited little change over time (long time constants). All time constants were significantly different from zero using a t-test at the 0.05 level (P < 0.00001).
Timing of reflex responses
To find an appropriate interval for EMG analysis, the onset of voluntary responses representing feedback correction in the VF and DF was examined using before effects. Movements were performed in the NF. On randomly selected trials this was changed to the force field to elicit reflex and voluntary reactions to the imposed force field. A total of 80 NF and 20 force field trials was recorded for each force field. A comparison of the before effect EMG to that of the NF trials allowed us to determine the onset of the corrective responses (Fig. 2). These generally occurred more than 200 ms after the onset of movement except in the case of the posterior deltoid muscle for which the responses were faster. In this case, a small early corrective response could be seen as early as 150 ms after the onset of movement.
|
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
|
During learning, subjects adapted to the novel forces applied by the VF by modifying the joint torques (Fig. 4, A-D). After adaptation, the shoulder joint torque became an extensor torque throughout the entire movement and increased to several times the value in the NF trials. At the elbow, only a small change in torque in the extensor direction, during the second half of the movement, was required. To adapt to the VF, subjects gradually changed both the amplitude and shape of their joint torque profile until they had compensated for the force field's effects. Early in learning, the torque varied both above and below the final adaptation profile. In particular, early in the trial the torque tended to overshoot the final torque profile, whereas later in the movement, the opposite occurred. The early effect is due to the force applied by the force field and the corresponding changes in torque produced by changes in hand trajectory. The later effect is likely the result of reflexive feedback and voluntary correction in response to the disturbance. As subjects gained more experience performing movements in this force field, the amplitude and shape of the torque profile were adjusted until trajectories were no longer disturbed by the force field. The variability in the joint torque was large among the early trials (Fig. 4, A and B). As subjects modified the torque profile, they also reduced the trial-to-trial variability in the joint torque. The absolute torque error of all subjects decreased quickly during learning with most of the adaptation occurring in the first 30-40 trials (Fig. 4, C and D). However, subjects continued to decrease joint torque errors by small amounts as their performance became more skilled and less variable. While subjects were able to reduce their hand-path trajectory errors close to the final level by approximately the 20th trial during learning, changes in the joint torque profiles indicate that further updating and refining of the feedforward commands was still taking place throughout the learning process.
|
Early trials in the DF were characterized by unstable trajectories that varied either to the left or the right of the straight line joining the initial and final targets. Adaptation to this force field did not require a change in the net joint torques. The joint torque after adaptation to the force field was similar to that in the NF (Fig. 4, E and F). The moving average of the joint torque (over 5 consecutive trials) during the learning process was also quite similar to that in the NF, although the SD remained high throughout the learning process. The computed joint torque took into account the measured hand force so the effect of any force applied to the hand by the DF due to small fluctuations in the hand-path was included. Unlike the adaptation to the VF, the absolute torque error remained fairly constant (Fig. 4, G and H).
When subjects were initially presented with the DF, their trajectories deviated to the left or the right of the target, and they rarely completed the movement successfully to the target. After learning, however, subjects were able to produce successful movements to the target. Movement along the target trajectory in the DF did not require any change in the joint torque compared with the NF. However, during the first few movements the joint torque varied greatly from trial to trial (Fig. 5). In particular, it tended to vary from an extensor torque to a flexor torque and back again. This alternating pattern, which can also be seen in the signed torque development plots (Fig. 5, C and D), did not necessarily occur on successive trials but was consistently seen on some time scale for every subject. The mean absolute torque development across all subjects increased progressively for the first six trials (Fig. 5, E and F). This occurred both for shoulder and elbow torque. An ANOVA was performed comparing the first three trials (trials 1-3) to the next three trials (trials 4-6) with subjects as a random variable. The mean absolute torque development for the second three trials was found to be significantly larger at the 0.05 level than for the first three trials for both the shoulder torque (P = 0.033) and the elbow torque (0.001). This indicates that subjects initially responded to the disturbing effects of the force field by trying to modify the net joint torques to reduce the error. However, changing the net joint torques modifies the hand-path trajectory, which generally causes larger perturbing forces to be applied by the DF.
|
Adaptation in the VF required a modification of the net joint torques at the shoulder and elbow. Specifically, adaptation produced a change in net extensor joint torque at the shoulder throughout the movement and a small extensor moment at the elbow late in the movement. Therefore we would expect to see increased activity in muscles contributing to these joint torques, particularly the posterior deltoid and long head of the triceps. After adaptation to the DF, the joint torques were not different from those in the NF. Instead, as we have shown previously, subjects modified the endpoint impedance of the limb (Burdet et al. 2001a). We therefore expected to see increased activity in one or more muscle pairs contributing to increased co-contraction. The expected changes in muscle activation patterns were confirmed from the EMG after learning (Fig. 6). In the VF, the EMG increased predominantly in the posterior deltoid and long head of the triceps muscles, the two muscles contributing the extensor torque at the shoulder needed to compensate for the force field. In contrast, in the DF the muscle activity increased in both muscles of all antagonist pairs. Because the endpoint forces and joint torques were the same in the DF and the NF, this increase in muscle activation represented balanced co-contraction.
|
After subjects experienced the VF for the first time, the muscle activity increased dramatically over the next few trials particularly for the posterior deltoid and long head of the triceps. This initial increase in EMG activity gradually diminished as learning proceeded. The change in EMG activity as learning progressed was fit with the double exponential function of Eq. 9, which represents EMG as a function of trial number. In the following sections, we present detailed analysis of the EMG from the interval of -100 to 350 ms from the start of movement. However, other intervals were also examined [(-100-200 ms), (250-350 ms), (-100-600 ms)] and all gave similar results. To illustrate the effect of the interval chosen for EMG analysis, the fitted curves for EMG activity using eight different time intervals were calculated for the long head of the triceps (Fig. 7). Although the rms EMG varies depending on the interval chosen, the overall trend for the way in which the activity changed on successive trials and the rate constants for the change in activity varied little. From this figure, we can see that the choice of interval is not critical. Our analysis using the interval -100-350 ms included both preprogrammed feedforward motor commands and feedback responses to the perturbations. However, based on before effect EMG (Fig. 2), we determined that the earliest response occurred at about 150 ms after movement start for the long head of the triceps and posterior deltoid, but for most muscles, no significant change in activity was seen prior to 350 ms. Analysis over this shorter interval (-100-200 ms) for the muscles that exhibited early responses gave similar results to the longer interval.
|
The evolution of muscle activity with practice in the VF is shown for one subject in Fig. 8. There was an increase in the EMG for all six muscles during the first few trials. A t-test was performed comparing the rms EMG of trials 2-21 to 20 trials in the NF for each muscle for each subject. Twenty-seven of the 30 comparisons showed significant increases in the rms EMG at the 0.01 significance level, indicating that the activity of all muscles increased early in learning. The peak EMG activity appeared to occur between the 3rd and 10th trials. With further practice, the EMG gradually decreased in all muscles to an asymptotic level. Similar to Fig. 6, the final rms EMG after adaptation was approximately the same as that of the NF for all muscles except those producing extensor torque at the shoulder. These muscles, the posterior deltoid and the long head of the triceps, compensated for the force applied by the VF. The actual difference after learning was tested using the same technique as for early learning. A t-test was used to compare the rms EMG of the final 20 trials in the VF to 20 trials in the NF for each muscle for each subject with the significance level set at 0.01. For the posterior deltoid and long head of the triceps, all comparisons (10/10) were significantly larger after learning compared with the NF. In the case of the lateral triceps, four of five comparisons were significantly larger after learning compared with in the NF. However, in the other three muscles, only 8 of the 15 comparisons were significantly larger in the force field.
|
The variation in EMG activity over trials was best fit by a double exponential process (Eq. 9). The results for all subjects are presented in Table 1. The activation and deactivation rates for each muscle were compared with the rate of reduction of the signed hand-path error. In the VF, subjects quickly adapted to the disturbing effects of the force field and reduced their hand-path error by two orders of magnitude within about 10 trials. Activation either led or paralleled hand-path error reduction and was characterized by a relatively short time constant, 1. The deactivation time constant, 2, was much longer. For three muscles (posterior deltoid, long head of the triceps, and brachioradialis), activation occurred more rapidly than hand-path error reduction (P = 0.012). Both the posterior deltoid and long head of the triceps contribute to the necessary increase in shoulder extensor torque. The activity of the brachioradialis acts to counteract the extensor torque at the elbow resulting from the increase in activity of the long head of triceps. The activity of the remaining muscles increased more slowly, at a rate not significantly different from the rate of hand-path error reduction (P = 0.45). The activation of both sets of muscle groups was also faster than the reduction in absolute torque error at either the shoulder (P = 0.008; P = 0.026) or elbow joint (P = 0.008; P = 0.016). Deactivation occurred at a similar rate in all muscles and proceeded much more slowly than either activation (P = 0.007), hand-path error reduction (P < 0.0001), reduction in absolute shoulder torque error (P = 0.012) or reduction in absolute elbow torque error (P < 0.0001). The fact that the reduction in hand-path error occurred faster than muscle deactivation indicates that subjects only gradually reduced their reliance on increased impedance to resist the force field even after hand-path error had been reduced to NF values. The largest increase in the activation process (A in Eq. 9) was seen for the posterior deltoid, followed by the long head of the triceps. However, all muscles showed a significant increase in muscle activation. The final level of adaptation, expressed as an increase from NF values (A-B in Eq. 9), indicates that the activation of most muscles, posterior deltoid and long head of the triceps being the exceptions, was reduced to near NF levels. The rates of the decrease in absolute shoulder and elbow torque errors were not found to be significantly different from each other (P = 0.4062). The rate of decrease in the absolute elbow torque error was found to be significantly slower than that of hand-path error (P = 0.041). However, the shoulder torque error was not found to be significantly different from the hand-path error at a level of 0.05 (P = 0.063).
|
During adaptation to the DF, there was also an initial increase in EMG activity of all muscles, which then gradually declined. The initial increase in EMG activity reached its peak after 20-40 trials. A t-test was performed comparing the rms EMG of trials 11-30 to 20 trials in the NF for each muscle for each subject. Twenty-seven of the 30 comparisons showed significant increases in the rms EMG at the 0.01 significance level indicating that all muscles increased activity early in learning. The EMG then gradually declined to an asymptotic level. Again, a t-test was performed comparing the rms EMG of the last 20 trials in the DF to 20 trials in the NF for each muscle for each subject. Twenty-six of the 30 comparisons showed a significant increase in the rms EMG at the 0.01 significance level after adaptation. The initial increase in activation, which represented co-contraction, occurred at a similar rate in all muscles. The ensuing deactivation proceeded much more slowly. This suggests that the selective control of endpoint impedance was a slow process, which likely involved tuning of the relative activation of muscle pairs. The double exponential model of Eq. 9 was again able to accurately capture the variation in EMG activity over the period of adaptation to the DF (Fig. 9).
|
The activation and deactivation time courses in the DF for each muscle were compared with the time course of the hand-path error reduction. Activation proceeded at a similar rate to hand-path error reduction (P = 0.59), whereas deactivation proceeded much more slowly (P < 0.0001). The time courses were similar for the six muscles. As in the case of the VF, activation was significantly faster than deactivation (P < 0.0001). Values for all parameters are shown in Table 1. The gain of the activation process (A) is larger for the biarticular muscles than the single joint elbow or shoulder muscles. Interestingly, the deactivation process (B) also is largest for the biarticular muscles, even though the final level after adaptation (expressed as an increase from NF values; A-B) shows the largest increase for the biarticular muscles.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
). These processes are summarized in Fig. 10.
|
Learning in the VF
The VF is similar to previously studied dynamical environments in which it has been suggested that inverse dynamics models are formed (Conditt et al.1997; Flanagan and Wing 1997; Flanagan et al. 2001; Krakauer et al. 1999; Lackner and Dizio 1994; Shadmehr and Mussa-Ivaldi 1994; Thoroughman and Shadmehr 1999). Early in learning, the activation of three muscles (posterior deltoid, long head of the triceps, and brachioradialis) increased rapidly. The increased activity may have comprised reflex activity arising from muscle stretch, voluntarily activation during the movement to correct for the initial displacement caused by the force field and predictive activation to counteract the expected force. The posterior deltoid and long head of the triceps contributed to the increase in shoulder extensor torque necessary to counteract the VF while the brachioradialis would have counteracted the extensor torque at the elbow resulting from the increase in activity of the long head of triceps. This adaptation is consistent with the development of an inverse dynamics model by means of feedback error learning (Kawato et al. 1987).
In addition, muscle activity increased due to agonist-antagonist co-contraction, which can be inferred from the increase in the activity of the remaining muscles. This co-contraction acts to increase the endpoint stiffness of the arm. Increased co-contraction during the learning of novel force fields is certainly not a new finding. Thoroughman and Shadmehr (1999) have also shown increased co-contraction during adaptation to stable dynamics, which they termed wasted contraction. However, we suggest that this increased contraction of antagonist muscles is not wasted but performs an essential role in the learning of novel dynamics by stabilizing the limb, resisting the disturbing effects of the force field, and allowing for the refinement of the feedforward inverse dynamics model (Osu et al. 2002). With improvement in the inverse dynamics model, co-contraction (limb stiffness) was reduced. This was associated with a slow deactivation process, which resulted in the activity of most muscles returning close to NF levels.
Learning in the DF
When subjects initially performed movements in the DF, their movements varied to either side of the mid-line from one trial to the next. This is evident from the joint torques, which alternated between extensor and flexor moments. The torque, relative to the NF, increased on each of the first six trials. This suggests that the subjects may have been attempting to incorporate the error information from the previous trial into the feedforward command for the next movement as would occur during feedback error learning of an inverse dynamics model. In the case of stable dynamics, this method works well and allows for quick adaptation. In contrast, this mechanism alone will not succeed when dynamics are unstable (Burdet et al. 2001b). If error information from the first trial was used to update the feedforward command for the next movement, that movement would be made to the opposite side of the mid-line of the force field because subjects would produce a force opposite to direction of the previous disturbance. The resulting error would be larger and opposite in sign to the previous error. As this process continued, joint torque would tend to increase and alternate between extensor and flexor moments, although the degree to which this would occur would depend on parameters such as the learning factor (cf. Scheidt et al. 2001) and the magnitude of motor noise.
However, early in the learning period there was an increase in agonist-antagonist co-contraction, which can be interpreted as an increase in endpoint stiffness. The hand-path error was reduced at about the same rate as the increase in stiffness. This provides evidence that the increased stiffness directly contributed to the reduction in hand-path error. This is not unexpected because increased endpoint stiffness counteracts the instability of the DF.
As the subject became more successful in counteracting the instability of the DF, the EMG was gradually reduced. This reduction in superfluous co-contraction would reduce metabolic energy requirements and possibly also reduce variability in motor output that tends to increase with muscle activity (Clancy and Hogan 1995; Harris and Wolpert 1998; van Galen and van Huygevoort 2000). We suggest that an interplay between two competitive processes, increased muscle activation in response to trajectory errors, and a drive to minimize muscle activation necessary to perform the task, results in a selective control of endpoint impedance as seen after learning in the DF (Burdet et al. 2001a; Franklin et al. 2003). The final level of muscle activity, relative to the NF, is expressed as A-B (Table 1). The largest values are found for the biarticular muscles (long head of the triceps and biceps brachii), indicating that these muscles may play the most prominent role in the selective control of the endpoint impedance in the DF.
Impedance controller
It is clear that the CNS is able to control the impedance of the limb. Increased muscle activity and stiffness has been seen during adaptation to many types of environments (De Serres and Milner 1991; Milner 2002; Milner and Cloutier 1993; Takahashi et al. 2001; Thoroughman and Shadmehr 1999). More recently, we have shown that this impedance can be directionally tuned to the environment (Burdet et al. 2001a). However, it is not yet clear how the CNS produces optimally oriented limb impedance. We propose the existence of an impedance controller that employs a fast activation process in response to error signals and a slower deactivation process. This controller will initially increase the impedance of the limb during adaptation to any novel dynamics. It will also attempt to minimize the activation of all muscles. By employing these two opposing processes, the initial high-impedance will eventually be reduced to a minimal level necessary for stability.
It appears that the CNS engages a similar process of impedance control during the initial period of adaptation whether trajectories are inherently stable or unstable. When an additional force must be applied to the environment, as in the case of the VF, there is a natural increase in limb impedance due to muscle activation (Franklin et al. 2003). Even though stability may be guaranteed by this naturally occurring impedance, the CNS chooses to augment it by co-contraction during the earliest phase of learning. The superfluous impedance is later eliminated once the inverse dynamics model had been acquired. However, when the naturally occurring impedance is insufficient to provide stability, as in the case of the DF, the impedance controller generates a global increase in stiffness, which is selectively reduced as the optimal stiffness geometry is determined (Franklin et al. 2003). Takahashi et al. (2001) have also suggested that impedance control can coexist with the formation of inverse dynamics models for control. Robotic implementations for learning novel tasks often use high impedance as a method of achieving faster learning (Katayama et al. 1998; Sanger 1994). The increased stiffness can reduce the disturbing effects of the novel dynamics to provide better tracking of a desired trajectory during the early stages of learning. The dynamics during learning are then closer to the desired final dynamics, which will increase the speed of learning.
Learning rates for impedance control and inverse dynamics model formation
The inverse dynamics model learning and impedance learning occur simultaneously, although the former appears to proceed at a faster rate than the latter. Several lines of evidence support this conclusion. Learning in the VF, ultimately realized by inverse dynamics model formation, is faster than learning in the DF, achieved by impedance control. Deactivation of all muscles (late phase of impedance learning) is slower than acquisition of accurate torque profiles (late phase of inverse dynamics model learning) in the VF. Activation of all muscles in the DF (early phase of impedance learning) is slower than activation of the posterior deltoid and the triceps long head in the VF (early phase of inverse dynamics model learning). In summary, the early phase of inverse dynamics model learning is almost simultaneous or slightly faster than the early phase of impedance learning, which is much faster than the late phase of inverse dynamics model learning, which, in turn, is slightly faster than the late phase of impedance learning. The interrelation of these two learning processes suggests that they could be unified in a single model.
Unified model for motor learning
The following natural extension of feedback error learning could coherently unify the two learning processes and at least qualitatively reproduce our results as well as other recent data on motor learning (Burdet et al. 2001a; Osu et al. 2002). First, centrally generated feedforward motor commands would comprise both a reciprocal component for agonist and antagonist muscle pairs (difference in muscle activation similar to net joint torque) and a co-activation component (summation of agonist and antagonist muscle activation similar to joint stiffness). This concept originated with Feldman and has been elaborated by his collaborators (Feldman 1980 a,b; Gribble et al. 1998; Latash 1992; Levin et al. 1992). Second, the feedforward co-activation signal to antagonists should increase on trials following perturbation of the hand path during early learning even when only agonist muscles are stretched. This hypothesis is required to account for activation of pectoralis major up to the fifth trial in the VF (Fig. 8) despite the fact that it was not stretched, as all the trajectories deviated to the left (Fig. 3A, trials 1-6). This mechanism also contributes to the co-activation of all muscles in the DF because trajectories deviate to left on some trials and to the right on others. Third, the feedforward co-activation signal decays with a large time constant as manifested by the deactivation time constants of all muscles. We are currently developing methods to quantitatively evaluate this conceptual model both experimentally and by means of computer simulations.
|
APPENDIX |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
). The EMG at trial 0 was set equal to the mean EMG in the NF prior to presentation of the force field. In the first model, EMG was expressed as a linear function of trial number. In the second model, EMG was expressed as an exponential function of trial number as used previously to characterize hand-path error (Burdet et al. 2001a; Flanagan et al. 1999) and suggested by the results of Thoroughman and Shadmehr (1999). The third model expressed EMG as a double exponential function of trial number with four free parameters (Eq. 9).
The three models were fit to the rms EMG of consecutive trials during learning in both the VF and DF. Because increasing the number of parameters of a model can over fit the data without adding any information, the residuals were used in Akaike's information criterion (AIC) (Akaike 1974) to determine if the added parameters of the double-exponential function were justified by explaining more of the variation in the data. To calculate a single representative value of AIC for each model, the data from six muscles and five subjects were combined. Because the EMG magnitude varied among muscles and subjects, the data were first normalized by dividing the residual by the sum of the fitted EMG data for each muscle and subject. The residuals, now expressed as a function of how well they fit the data, were then combined to calculate a single AIC value
 | (A1) |
is the integrated squared error in EMG modeling for the jth muscle of the ith subject. This was performed separately for VF and DF data to determine which model fit the data for learning in each force field most appropriately.
The AIC values were calculated for each model in both the VF and the DF. In both fields, the double-exponential model fit the data better. In the VF, the lowest AIC was obtained for the double-exponential model (42,696). This was 50 less than AIC for the single exponential model and 2,023 less than AIC for the linear model. A difference of more than 2 is usually considered statistically significant (Sakamoto et al. 1986). The results for the DF were similar, with the lowest AIC (45,374) for the double exponential model, which was 209 less than AIC of the single exponential model and 1,521 less than AIC for the linear model. This supports our choice of the double-exponential model, comprising an activation and a deactivation process, to describe adaptation to both the VF and DF dynamics.
|
ACKNOWLEDGMENTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
This research was supported by the Telecommunications Advancement Organization of Japan, the Natural Sciences and Engineering Research Council of Canada, and the Human Frontier Science Program.
|
FOOTNOTES |
|---|
Address for reprint requests: D. W. Franklin, ATR Computational Neuroscience Laboratories, 2-2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan (E-mail: dfrank{at}atr.co.jp).
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
Akazawa K, Milner TE, and Stein RB. Modulation of reflex EMG and stiffness in response to stretch of human finger muscle. J Neurophysiol 49: 16-27, 1983.
Burdet E, Osu R, Franklin DW, Milner TE, and Kawato M. Measuring stiffness during arm movements in various dynamic environments. In: Proceedings of the 1999 ASME Annual Symposium on Haptic Interfaces and Virtual Environments for Teleoperator Systems, Nashville, TN: ASME, 1999, p. 421-428.
Burdet E, Osu R, Franklin DW, Milner TE, and Kawato M. The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature 414: 446-449, 2001a.[Medline]
Burdet E, Tee KP, Chew CM, Franklin DW, Osu R, Kawato M, and Milner TE. Stability and learning in human arm movements. Proceedings of the 2001 International Conference on Computational Intelligence, Robotics and Autonomous Systems, Singapore, p. 335-361, 2001b.
Clancy EA and Hogan N. Multiple site electromyograph amplitude estimation. IEEE Trans Biomed Eng 42: 203-211, 1995.[ISI][Medline]
Conditt MA, Gandolfo F, and Mussa-Ivaldi FA. The motor system does not learn the dynamics of the arm by rote memorization of past experience. J Neurophysiol 78: 554-560, 1997.
) and without (
) a force perturbation (15 N for 25 ms) were recorded. The onset of the force perturbation (100 ms after start) is shown by the black arrow. Both series of trajectories converge to the same path demonstrating the stability of the interaction between the arm and the VF. D: (top) force in the divergent force field (DF) plotted as a function of the hand position. The force vector increases in magnitude with the distance from the y axis. Bottom: force vectors when hand trajectories (· · ·) are slightly deviated from a straight line along the y axis to the left and right. The DF amplifies trajectory variations.
, the EMG of the last 20 trials of the NF. A baseline was determined from the mean NF root-mean-square (rms) EMG (· · ·) and was set to a value of 1 for each muscle. 
