| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Department of Biomedical Engineering, Washington University, Saint Louis, Missouri
Submitted 28 February 2006; accepted in final form 9 May 2006
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
; Gandolfo et al. 1996; Shadmehr and Brashers-Krug 1997; Shadmehr and Mussa-Ivaldi 1994).
To study motor learning, participants are often asked to make reaching movements while holding the handle of a robotic manipulandum that generates perturbing forces. Each unexpected perturbation challenges the nervous system to generate a within-movement feedback response; feedback may also generate a training signal to update the predictive control of arm-movement dynamics between movements (Kawato et al. 1987). Over a series of movements, training can restore desired performance, despite environmental perturbations.
Foundational psychophysical studies demonstrated that humans could quickly adapt their motor behavior to move skillfully in novel dynamic environments (Lackner and Dizio 1994; Shadmehr and Mussa-Ivaldi 1994). Recent experiments have shown how learning transfers across training sets after changing the movement direction (Gandolfo et al. 1996; Sainburg et al. 1999), hand position (Shadmehr and Moussavi 2000), movement speed (Goodbody and Wolpert 1998), or task (Conditt et al. 1997). These studies exposed subjects to perturbing forces for tens or hundreds of movements and then retested them in a baseline condition or novel environment. Such experiments can identify the time course of motor adaptation, but by integrating over several movements, cannot identify how specific errors inform that adaptation.
Left largely unexamined is how humans abstract information from one movement to update their prediction of environmental dynamics. Preliminary studies (Scheidt et al. 2001; Thoroughman and Shadmehr 2000) have used system identification to examine adaptation across single movements. This identification, coupled with neural network models (Thoroughman and Shadmehr 2000), has suggested that incremental motor adaptation is proportional to motor error and gated by the positions and velocities generated within specific movements. This hypothesis has explained motor learning in several viscous (Donchin et al. 2003) and position-dependent (Hwang et al. 2003) environments.
Previous studies trained subjects in a perturbing field that was either on or off during an entire movement. Within a single movement, however, subjects moved through a variety of positions and velocities. These studies integrated over the entire movement to identify how people transform sensed error into the next update of predictive control. In the present study, we consider how the precise features of error induce incremental updates in subsequent control.
We introduce transient perturbations to study the dependence of adaptation, across single trials, on the fine details of sensory feedback. To study this trial-by-trial response, we initially sought to minimize the formation of a motor memory over many movements by presenting pulses infrequently and by balancing leftward and rightward pulses. We found that movements perturbed with 70-ms force pulses induced aftereffects in the next, unperturbed movement that countered the direction of the pulse. These aftereffects were insensitive to the within-movement positional placement of the pulse. We next investigated the aftereffects following pulses of varying magnitude and, surprisingly, found that aftereffects did not scale with the pulse magnitude but instead coarsely countered the pulse direction. Last, we show that subjects can build motor memories after repeated exposure to force pulses.
We conclude that, in certain tasks and with certain perturbations, humans adapt to the direction, but not the magnitude, of movement error. This result was surprising, in light of the proportional responses measured previously in studies of trial-by-trial adaptation (Scheidt et al. 2001; Thoroughman and Shadmehr 2000), and suggests that the transformation of error into adaptation can change, depending on the dynamics of the environment.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
Movements were 10 cm in length, 0.5 s in duration, and in a single direction, away from a subject's body in the horizontal plane. All subjects trained for 2 days. On the first day, subjects trained without experiencing perturbing forces (the null-field condition); forces were introduced on the second day of training. On each day, subjects made 4 sets of 150 movements (experiments 1 and 3) or 180 movements (experiment 2). Subjects rested for approximately 3 min between sets.
The motor system is inherently noisy, and subjects' movements naturally varied from trial to trial. To minimize this variance, subjects were asked to "mimic" a training dot (experiments 2 and 3 only). The dot reproduced a straight-line, minimum jerk trajectory, 0.5 s in duration. On the first day, the dot appeared in 100% of movements in the first set, 75% in the second set, 50% in the third set, and 25% in the fourth set; on the second day, the dot appeared in 20% of movements but never on pulsed or postpulse trials. The training dot began after subjects exceeded the forward threshold for movement, 2.5 cm/s.
Full visual feedback was given throughout the experiment, either on a vertically mounted computer monitor (experiments 1 and 3) or projected on a horizontal screen mounted above the robotic arm (experiment 2). With the monitor, subjects moved their arms through a horizontal plane aligned with their shoulder; with the projection screen, subjects moved their hands through a horizontal plane under the screen, slightly below their shoulders. Timing feedback was given by changing the target color (red, <450 ms; green, 450–550 ms; blue, >550 ms). After a movement, the manipulandum carried subjects back to the starting location. Subjects' arms were supported against gravity by a sling attached to the ceiling by a 10-ft cable. Subjects were instructed to move quickly and smoothly to the target. When the training dot was available, they were instructed to use the dot to time their movements.
Experiment 1
We perturbed the reaching movements of 20 subjects with pulses of force to determine how the within-movement position of a transient force pulse would affect the pulsed movement and the predictive control of subsequent movements. Subjects (9 women and 11 men) ranged in age from 18 to 27 yr (mean, 21.9 ± 2.5 yr). Pulses were 70 ms in duration with amplitude 18 N and had a bell-shaped force profile that was generated by truncating a Gaussian in time (Fig. 1B, inset). Subjects experienced 12 varieties of pulses: subjects were pulsed 2, 3, 4, 5, 6, or 7 cm into movement, either to the left or the right (±90° to the direction of motion). Force pulses were experienced pseudo-randomly in 20% of trials; only a single force pulse was experienced on each pulsed trial, and subjects experienced all pulse types within a single training set. Pulsed movements never occurred back to back. Thus, movements before and after pulsed trials were always made in a null field. Fifteen subjects were perturbed by force pulses introduced 3, 5, or 7 cm into movement. Five subjects experienced pulses at 3 additional positions: 2, 4, and 6 cm.
|
In a second experiment, we perturbed the reaching movements of 12 individuals with pulses of variable magnitude (6, 12, and 18 N). Pulses were 70 ms in duration, introduced 3, 5, or 7 cm into movement, and were directed either to the left or the right. Subjects (4 women and 8 men) ranged in age from 18 to 27 yr (mean, 23.0 ± 2.4 yr). Pulses were experienced pseudo-randomly in 20% of trials, but never back to back; all pulse types were experienced in a single set.
Experiment 3
The first and second experiments were designed to interrogate subjects' incremental adaptive response. They were specifically designed to suppress the formation of a motor memory, by introducing pulses infrequently and equally balancing leftward and rightward forces. In a third experiment, we did just the opposite; we presented pulses frequently with a strong directional bias to determine whether subjects could form a motor memory over hundreds of pulsed movements.
Six subjects (2 women and 4 men), ranging in age from 19 to 24 yr (mean, 21.3 ± 2.1) participated in the third experiment. In the first two training sets, pulses were presented in every movement. Catch trials, trials in which the pulse was unexpectedly removed, were introduced pseudo-randomly in 20% of trials in the third and fourth sets. All pulses were 70 ms in duration, rightward-going, and experienced 5 cm into movement. The magnitude (6, 12, and 18 N), but not the direction, of the pulse changed pseudo-randomly from trial to trial.
Performance metrics
To quantify subjects' trajectories, three performance metrics were used. Movement area quantified the signed area bounded by the trajectory and a straight line connecting the movement start and target. Perpendicular displacement (p.d.) quantified the horizontal displacement from movement start. Take-off angle, averaged over a 5 mm window 1 cm into movement, quantified the initial direction of movement. Take-off angles were measured with respect to a straight line connecting the movement start and target. Adaptation was quantified by subtracting prepulse metrics from postpulse metrics, where pre- and postpulse are defined as the movements immediately before and after a pulsed movement, respectively. By use of these performance metrics, we considered how the magnitude of the pulse and the position and velocity at which the pulse was experienced affected subsequent predictive control.
Statistical significance
We used t-tests and analyses of variance (ANOVAs) to determine the significance of the above metrics and changes in metrics across pulse types. In ascertaining the significance of linear fits, we used an F-statistic, derived from mean square errors of residuals and fits.
Simulation
We simulated a feedback controller to calculate a stiff and viscous feedback torque (equation [1]); the feedback signal was driven by changes in shoulder and elbow positions (q, [2 x 1]) and velocities (
, [2 x 1]) from their desired states (qd, d). The desired trajectory was assumed to be minimum jerk and 0.5 s in duration. Stiff and viscous matrices were taken from the literature and assumed a joint-based coordinate system (Shadmehr and Mussa-Ivaldi 1994):
 | (1) |
). The feedback torque was converted to feedback force via a Jacobian transformation. |
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
We investigated the specific, real-time components of motor learning by perturbing human reaching movements with 70-ms pulses of force at varying positions within a 10-cm movement. To focus on the trial-by-trial response, we designed each set to suppress the generation of a motor memory by setting the left/right perturbation bias to zero and introducing forces in only 20% of trials. We ask how error, abstracted from a single trial, changes the prediction of movement dynamics in the very next trial. Do the specific spatial details of feedback differentially affect incremental updates of control?
We began by observing the entire time-series of positions in prepulse, pulsed, and postpulse movements (Fig. 1) to characterize the normal human response. Prepulse trajectories (Fig. 1A) were relatively straight with a single peaked, bell-shaped velocity profile, curved only slightly off a minimum jerk trajectory (Flash and Hogan 1985). Force pulses noticeably perturbed movement trajectories (Fig. 1B). Pulsed movements were kinematically identical to prepulse movements until the force pulse was administered; thus, any changes in postpulse trajectories were due to the 70-ms force pulse only. Postpulse trajectories deviated from their prepulse counterparts (Fig. 1C). Leftward pulses reliably altered predictive control such that subjects moved rightward in the postpulse movement. Conversely, subjects compensated leftward after a rightward pulse.
Changes in predictive control were similar regardless of the position at which subjects were pulsed. Subjects adapted their predictive control starting very early in the next movement; postpulse movements deviated from prepulse trajectories almost immediately, regardless of when the pulse was experienced (Fig. 1D). The remainder of the postpulse movement opposed the pulse direction but was insensitive to the pulse placement. This analysis of full trajectories indicates robust adaptation to pulses that is noticeably insensitive to the position at which the perturbing pulse is experienced.
To quantify adaptation, movements were characterized by movement area. The average prepulse trajectory had an area of –0.47 cm2. To compare prepulse movements across pulse type, a correlation analysis was performed that compared subjects' mean performance within pulse type to the average prepulse movement, across pulse type and subjects. We found that prepulse movements were statistically indistinguishable across pulse type (ANOVA, P = 0.68); thus, subjects' expectation of force was constant in prepulse movements over the course of the experiment.
Pulsed movements were kinematically different across pulse position (Fig. 1B); the magnitude of the pulsed movement area decreased as pulse position increased (ANOVA, P < 0.001). Even when pulsed early in movement perpendicular displacement (p.d.) continued to increase until very late in movement. Subjects approached the target along a line perpendicular to the direction of movement, regardless of pulse type.
Although pulsed movements were different, postpulse trajectories were very similar. The adapted response, calculated by subtracting the average prepulse area from postpulse area, was similar regardless of pulse position within a pulse direction (ANOVA, leftward pulses, P = 0.87; rightward pulses, P = 0.63). On average, subjects adapted 1.17 cm2 and –0.80 cm2 following leftward and rightward pulses, respectively. The directional sensitivity of the adapted response was washed out in the next trial, two trials after the pulsed movement (t-test, P = 0.34).
Quantifying adaptation via an area metric sacrifices temporal resolution within the adapted movement. Therefore, the degree to which pulses perturbed prepulse trajectories was also quantified by subtracting subjects' average prepulse p.d. from postpulse p.d. Adaptation following leftward pulses was defined to be positive (postpulse p.d.-prepulse p.d. > 0, t-test, P < 0.001); adaptation to rightward pulses was defined to be negative (t-test, P < 0.001). Within a pulse direction, adaptation was statistically indistinguishable regardless of pulse position, whether p.d. was measured early (3 cm, Fig. 2A), middle (5 cm, Fig. 2B), or late (7 cm, Fig. 2C) in movement (ANOVA, rightward pulses: P = 0.44, 0.69, and 0.92 for p.d. measured at 3, 5, and 7 cm into movement, respectively; leftward pulses: P = 0.30, 0.44, and 0.46, respectively). Mid-movement adaptation was noticeably larger following leftward pulses (two-dimensional ANOVA with factors pulse position and pulse direction, P < 0.001).
|
Although we only controlled for the position at which the pulse was experienced, pulses naturally occurred over a wide range of velocities (Fig. 3A). We could therefore investigate the relationship between the velocity at the time of pulse and subsequent adaptation (Fig. 3, B–D). Adaptation (quantified by p.d.) to leftward pulses was marginally dependent on the velocity at which the pulse was experienced, but only when measured in the middle of (5 cm) or late (7 cm) in movement (linear fit, 5 cm, P = 0.047; 7 cm, P = 0.041); across-trial adaptation was lowest when the pulse was experienced at the highest velocities (35–40 cm/s). This suggests that, unlike position, adaptation is not completely untuned to velocity. The effect, however, is small. Even in the most significant case (leftward pulses, p.d. at 7 cm), a linear fit (Fig. 3D, dashed line) predicted adaptation following 5-cm pulses (which are experienced near the peak of the speed profile, approximately 33 cm/s, on average) to be 75% of that following 2-cm pulses (which are experienced at a slower speed, approximately 23 cm/s). Furthermore, we found no significant relationship between the velocity at which a pulse was experienced and the subsequent change in p.d., when p.d. was measured after rightward pulses, early, in the middle of, or late in movement (P = 0.13, 0.13, and 0.18, respectively), and after leftward pulses, early in movement (P = 0.20).
|
In the first experiment, we found that subjects were insensitive to the position of the force pulse. In a second experiment, we specifically tested for sensitivity to pulse magnitude. Twelve right-handed subjects made 10-cm reaching movements in the horizontal plane. Force pulses were introduced either to the left or the right, 3, 5, or 7 cm into a movement, and were 6, 12, or 18 N in amplitude. Prepulse movements were again invariant across pulse types (P = 0.79). The kinematic error induced by the forces scaled with the magnitude of the perturbation (Fig. 4, A–C). If the forces pushed to the left, people reliably adapted by moving to the right, beginning at movement initiation. If the forces pushed right, people adapted by moving to the left. Adaptation was quantified with respect to the average prepulse movement using p.d. and take-off angle. Adaptation was again insensitive to pulse position, whether measured early, in the middle of, or late in movement (rightward pulses: P = 0.20, 0.25, and 0.29 respectively; leftward pulses: P = 0.33, 0.42, and 0.24). Adaptation was surprisingly also insensitive to the magnitude of the pulse (Fig. 4, D–F). Adaptation to leftward and rightward pulses was statistically indistinguishable regardless of pulse strength, whether p.d. was measured early (3 cm, Fig. 5A), in the middle of (5 cm, Fig. 5B), or late in (7 cm, Fig. 5C) movement (rightward pulses: P = 0.16, 0.26, and 0.41 respectively; leftward pulses: P = 0.79, 0.47, and 0.17). Take-off angles were also indistinguishable regardless of the pulse location (P = 0.96 and 0.36) or pulse magnitude (P = 0.48 and 0.73, for rightward and leftward pulses, respectively). Statistics were calculated using a balanced, two-dimensional ANOVA with factors pulse position and magnitude.
|
|
|
This result was surprising, in light of the proportional responses measured in novel dynamic environments. This component of adaptation therefore cannot scale with any previously hypothesized error signal, nor can it depend on a real-valued critical metric. This result, combined with previous studies, suggests that people can adopt different modes of adaptation that can or cannot scale with error.
Force pulses and motor memory formation
The first two experiments were designed to look explicitly at incremental steps of adaptation across single movements. To that end, we suppressed the formation of a motor memory by introducing forces in only 20% of trials, and by setting the left/right bias of those forces to zero. In light of the surprising results of the previous experiment, we now ask, do subjects form a motor memory after repeated exposure to 70 ms pulses of variable magnitude, or are the forces so nonecological that they are unlearnable? Six subjects made four sets of 150 reaching movements in a single direction. Subjects experienced rightward pulses with magnitude 6, 12, or 18 N in every trial in the first two training sets; all pulses were applied 5 cm into movement. Catch trials, trials in which the force pulse was unexpectedly removed, were introduced pseudo-randomly in 20% of trials in the third and fourth sets.
Initially, pulsed-movement trajectories scaled with magnitude and were consistent with those trajectories seen previously (Fig. 7, A–C, solid lines). With practice, subjects reduced pulse-induced curvature (Fig. 7, A–C, dashed lines). Adaptation was quantified by movement area, which subjects significantly reduced with training (P = 0.048, 0.026, and 0.017 for 6, 12, and 18 N pulses, respectively). Overall, subjects adapted to the mean pulse strength. Subjects under-adapted to the strongest, 18 N pulses, as evidenced by a rightward, pulse-induced hook, and over-adapted to the weakest, 6 N pulses, as evidenced by a hook in the opposite direction of the pulse.
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
; Thoroughman and Shadmehr 2000). We adopted a different perturbation paradigm to interrogate the within-movement specificity of motor adaptation. We chose to use 70-ms pulses of force (Fig. 8A), which elsewhere have been used to interrogate feedback control in normal and neurologically impaired subjects (Smith et al. 2000). The specificity of the force pulse enabled a novel identification of the temporal and spatial resolution of trial-by-trial motor adaptation. Pulses generated kinematic errors perpendicular to the movement direction. These errors were similar in shape and amplitude but shifted in time (Fig. 8B). Errors coincided with different parallel positions and velocities. These errors drove a feedback response (simulated by multiplying actual positions and velocities by stiffness and viscosity matrices; see METHODS), the shape and magnitude of which was again preserved; only the within-movement timing varied (Fig. 8C).
|
) and velocities (Donchin et al. 2003; Thoroughman and Shadmehr 2000). We saw no significant sensitivity to the positional context of movement error; aftereffects start at the beginning of movement regardless of whether error is experienced early, in the middle of, or late in the previous trial. This was reasonable, because the within-movement pulse-placement domain (5 cm) was smaller than the approximately 12-cm positional tuning evidenced by previous across-movement generalization studies (Hwang et al. 2003). Adaptation was weakly dependent on the velocity at which the pulse was experienced; velocity-sensitivity was only significant in the middle of and at the end of postpulse movements when the pulse was directed leftward. Even the significant sensitivity was small and maintained a substantial transfer of adaptation across movement velocities. We conclude that people broadcast information broadly across all components of individual movements. We then perturbed subjects with force pulses of magnitude 6, 12, and 18 N (Fig. 8D). Pulses generated position (Fig. 8E) and velocity errors that scaled with pulse magnitude; these errors drove a feedback response that also scaled with magnitude (Fig. 8F). Dynamic error, kinematic error, and the feedback response, all candidate error signals in current theories of motor adaptation, therefore all scaled with the magnitude of the force pulse. Subjects displayed clear sensitivity to the direction of the pulse but no sensitivity to the pulse magnitude. Adaptation following 6 and 18 N pulses, averaged across pulse position, showed no significant difference (Fig. 6); a proportional response would require adaptation to be three times larger following 18 N pulses than 6 N pulses.
Aftereffects were approximately 10% of the maximum kinematic error. Previous experiments (Thoroughman and Shadmehr 2000; Thoroughman and Taylor 2005) have identified a 25%–30% transfer of error; the small size of the 18 N after effect, therefore, is unlikely to be caused by saturation. Recent loss-function estimates (Kording and Wolpert 2004) suggest that large errors could result in a less than proportional response, but the approximately 0.5-cm errors experienced here are smaller than those used previously to identify proportionality in adaptation (Scheidt et al. 2001). This insensitivity is in opposition to the natural proportionality of signals available in the pulsed movement; seemingly, subjects transformed the proportionality of their feedback response through a function that flattened the response to the magnitude of the error while maintaining or amplifying dependence on error direction.
This result contradicts the fundamental assumptions of a wide variety of biological and theoretical learning hypotheses. Theories of motor learning include adaptation driven by motor error and feedback control induced by sensory error. (Wolpert and Ghahramani 2004). Each of these theories scales adaptation with the magnitude of previous error. This core hypothesis dates back to the Marr-Albus model of the cerebellum, which suggests that the firing rate of cerebellar climbing fibers explicitly encodes the size of the error (Kawato 2002). Even reinforcement learning, which lacks a specific target behavior to which quantified error can be calculated, features learning steps that depend on expected reward and scale with error magnitude (Barto 2003; Gullapalli 1990).
Proportionality to error also extends beyond motor theories to the learning rules of back propagation (Rumelhart and McClelland 1986) and adapting basis function networks (Pouget and Snyder 2000). Recently, basis functions have been used to quantify the generalization of motor error across the movement space (Donchin et al. 2003; Hwang et al. 2003; Thoroughman and Shadmehr 2000). These models relied on two assumptions: 1) the nervous system registered the experienced error with the coincident movement state, such that weight updates were specific to the neurons that caused the error, and 2) each increment of error generated a proportional change in the subsequent response. Our second experiment invalidated this second assumption; we witnessed surprising insensitivity to error magnitude. Therefore this architecture cannot predict the disproportionality seen across different magnitudes. Rather, it maintains a strictly proportional transformation of error into adaptation.
These hypotheses predict the observed sensitivity to pulse direction but fail to predict the observed insensitivity to pulse magnitude. We are not suggesting that the insensitivity we observed completely disproves these hypotheses; several studies of human motor learning are consistent with prior theories. Recently, for example, system identification techniques, each presuming proportionality, have been used to explain adaptation to position-dependent (Hwang et al. 2003) and viscous forces (Donchin et al. 2003; Thoroughman and Shadmehr 2000), as well as to visuo-motor perturbations (Baddeley et al. 2003). The presumption of proportionality in these analyses was not explicitly tested, but the good fits of the state-space models are consistent with theoretical predictions.
Experiments 1 and 2 and their subsequent analyses interrogate the trial-by-trial response of the adaptive system. This focus on short-term learning left the question: because people have difficulty learning time-dependent forces (Conditt and Mussa-Ivaldi 1999; Karniel and Mussa-Ivaldi 2003), could the unusual temporal properties of the perturbation prevent subjects from building a motor memory? In our third experiment, we found that subjects could build and maintain improved performance when adapting to 70-ms pulses that varied in magnitude, but not direction. Furthermore, in catch trials, subjects moved opposite the pulse direction, revealing motor memory formation. We do not know whether the motor adaptation learned in these experiments is dependent on time or on state, such as hand position or velocity. Therefore, we do not refute that adapted representations are state-dependent (Conditt and Mussa-Ivaldi 1999). The learnability of the pulses, however, indicates that, in experiments 1 and 2, the measured sensitivity (to direction) and insensitivity (to position and magnitude) were not artifacts of the unusual timing and shape of the perturbing force.
Proportionality has been explicitly demonstrated in stochastic environments, when viscous forces are applied throughout movement and when the gain is drawn from a biased distribution with nonzero mean (Scheidt et al. 2001). We have identified three factors that differ between this study and our first two experiments: the duration of force within each movement, the proportion of movements within which forces are experienced, and the directional bias of the distribution of forces. We designed experiment 3 to determine whether subjects could adapt over many movements when consistently exposed to biased forces; subjects did build a motor memory over many trials. This consistent exposure to pulses did not enable the single trial analysis we performed in experiments 1 and 2. The adaptation occurring in experiment 3 may, in fact, differ from trial-by-trial adaptation quantified in experiments 1 and 2. We hypothesize that protocols that systematically interpolate between ours and that of Scheidt et al. (2001) will determine the necessary experience to induce trial-by-trial adaptation that partially or completely transitions between disproportional and proportional adaptations.
These permutations of experimental paradigms will determine the conditions under which disproportional adaptation occurs but will not necessarily elucidate why people switch between adaptive strategies (Fig. 9). We hypothesize that subjects adopt one of three strategies to transform real valued error into incremental adaptation. A first strategy is zero adaptation (Fig. 9A); sensory experiences can fail to drive adaptation because of a lack of magnitude, relevance, or consistency (Sanger 2004). If a response is elicited, previous theories of motor adaptation have posited that learning strives to minimize a real-valued movement error; machine learning theory predicts that these strategies regress across error (Alpaydin 2004). Instead, we see trial-by-trial adaptation that categorizes error into leftward or rightward bins, effectively processing sensed error through a left/right categorization (Fig. 9B) before generating an adaptive response. This categorization generates a binary signal to drive incremental adaptation; this process therefore rejects real-valued error minimization in favor of minimizing a cost that values only the direction of error.
|
). We do not hypothesize that our subjects learn to classify experiences during training, because after effects are insensitive to pulse magnitude from the beginning of training. We instead hypothesize that subjects have previously learned a leftward/rightward classification and then applied this classification to generate aftereffects in lieu of a regressive, proportional learning strategy (Fig. 9C).
A similar partitioning of sensory space is well studied in categorical perception (Kuhl 2004). Exposing infants to natural speech patterns causes the categorization of a continuum of sounds into distinct consonants (Kuhl 2000). Partitioning stimulus space into distinct categories has also been hypothesized to underlie visual, auditory, and tactile object recognition (Kohonen and Hari 1999; Kuhl 2000). Repeated presentation of continuous stimuli with relevant categories also leads to a physical partitioning of the cortical representation of sensation (Kohonen and Hari 1999). We hypothesize that the disproportional aftereffects amount to categorical adaptation, as the leftward/rightward partition of sensed error drives adaptation instead of continuous metrics. This may indicate, for this class of perturbations, a neural representation of movement error that is categorical rather than continuous. This is different from categorical perception. Subjects do not adopt this categorization scheme immediately; the mid-movement, error-induced feedback response scales proportionally with pulse magnitude. Instead, categorization is applied across movements, to the transformation of sensed error to adaptation.
Our focus is consistent with experiments and theories that primarily ascribe motor adaptation to changes in the predictive control of dynamics; even the most advanced trial-by-trial analyses (Smith et al. 2006) account for complex adaptive behaviors by fixing the modeled feedback response while changing predictive, feed-forward control. To investigate trial-by-trial behavior we applied pulses infrequently, and therefore we could not test for the possibility that pulses induced changes in the feedback response (Burdet et al. 2001; Todorov 2004). It is possible that categorical adaptation of predictive control leaves intact changes in the feedback response that remain proportional to sensed error. We nevertheless identify adaptive changes early in movement that can be ascribed only to adaptive predictive control; these adaptations relate to error categorically, not proportionally.
The richness of human motor behavior arises from our ability to learn new movements. Trial-by-trial motor adaptation has been hypothesized to depend on two terms: an abstraction of a quantitative error metric from a movement and the application of that error to update the predictive control of the next movement. We designed our first experiment to investigate the within-movement sensitivity of the second term: would the position or velocity at which error was experienced differentially influence the adaptive response? Although we found the adaptive process to depend subtly on the speed at which error was sensed, it was largely insensitive to the fine context within which the error was experienced. We then altered our protocol to expose subjects to errors of varying magnitude. To our surprise, aftereffects did not increase in magnitude with increases in induced error. Our experiments therefore suggest, for this set of perturbations, insensitive encoding in both terms of adaptation; insensitivity to error magnitude in the first term, and insensitivity to movement context in the second. Coupled with a recent discovery of flexible error generalization across movement direction (Thoroughman and Taylor 2005), we now consider both components of learning to adapt as a function of environmental experience.
|
GRANTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
|
ACKNOWLEDGMENTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
|
FOOTNOTES |
|---|
Address for reprint requests and other correspondence: Kurt A. Thoroughman, Department of Biomedical Engineering, Washington University, 1 Brookings Dr., Campus Box 1097, Saint Louis, Missouri, 63130, USA (E-mail: thoroughman{at}biomed.wustl.edu)
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSACKNOWLEDGMENTSREFERENCES |
|---|
Baddeley RJ, Ingram HA, and Miall RC. System identification applied to a visuomotor task: near-optimal human performance in a noisy changing task. J Neurosci 23: 3066–3075, 2003.
Barto AG. Reinforcement learning in motor control. In: The Handbook of Brain Theory and Neural Networks, 2nd ed., edited by Arbib MA. Cambridge, MA: MIT Press, 2003, p. 968–972.
Burdet E, Osu R, Franklin DW, Milner TE, and Kawato M. The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature 414: 446–449, 2001.[CrossRef][Medline]
Conditt MA, Gandolfo F, and Mussa-Ivaldi FA. The motor system does not learn the dynamics of the arm by rote memorization of past experience. J Neurophysiol 78: 554–560, 1997.
Conditt MA and Mussa-Ivaldi FA. Central representation of time during motor learning. Proc Natl Acad Sci USA 96: 11625–11630, 1999.
Donchin O, Francis JT, and Shadmehr R. Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. J Neurosci 23: 9032–9045, 2003.
Flash T and Hogan N. The coordination of arm movements: an experimentally confirmed mathematical model. J Neurosci 5: 1688–1703, 1985.[Abstract]
Gandolfo F, Mussa-Ivaldi FA, and Bizzi E. Motor learning by field approximation. Proc Natl Acad Sci USA 93: 3843–3846, 1996.
Goodbody SJ and Wolpert DM. Temporal and amplitude generalization in motor learning. J Neurophysiol 79: 1825–1838, 1998.
Gullapalli V. A stochastic reinforcement learning algorithm for learning real-valued functions. Neural Networks 3: 671–692, 1990.
Hwang EJ, Donchin O, Smith MA, and Shadmehr R. A gain-field encoding of limb position and velocity in the internal model of arm dynamics. PLoS Biol 1:E25, 2003.[Medline]
Karniel A and Mussa-Ivaldi FA. Sequence, time, or state representation: how does the motor control system adapt to variable environments? Biol Cybern 89: 10–21, 2003.[ISI][Medline]
Kawato M. Cerebellum and motor control. In: Handbook of Brain Theory and Neural Networks, 2nd ed., edited by Arbib MA. Cambridge, MA: MIT Press, 2002.
Kawato M, Furukawa K, and Suzuki R. A hierarchical neural-network model for control and learning of voluntary movement. Biol Cybern 57: 169–185, 1987.[CrossRef][ISI][Medline]
Kohonen T and Hari R. Where the abstract feature maps of the brain might come from. Trends Neurosci 22: 135–139, 1999.[CrossRef][ISI][Medline]
Kording KP and Wolpert DM. The loss function of sensorimotor learning. Proc Natl Acad Sci USA 101: 9839–9842, 2004.
Kuhl PK. Early language acquisition: cracking the speech code. Nat Rev Neurosci 5: 831–843, 2004.[CrossRef][ISI][Medline]
Kuhl PK. Language, mind, and brain: Experience alters perception. In: The New Cognitive Neurosciences, 2nd ed., edited by Gazzaniga MS. Cambridge, MA: MIT Press, 2000, p. 99–115.
Lackner JR and Dizio P. Rapid adaptation to Coriolis force perturbations of arm trajectory. J Neurophysiol 72: 299–313, 1994.
Pouget A and Snyder LH. Computational approaches to sensorimotor transformations. Nat Neurosci 3 Suppl: 1192–1198, 2000.
Rumelhart DE and McClelland JL. Parallel distributed processing: Explorations in the microstructure of cognition. Volume 1: Foundations. Cambridge, MA: MIT Press, 1986.
Sainburg RL, Ghez C, and Kalakanis D. Intersegmental dynamics are controlled by sequential anticipatory, error correction, and postural mechanisms. J Neurophysiol 81: 1045–1056, 1999.
Sanger TD. Failure of motor learning for large initial errors. Neural Comput 16: 1873–1886, 2004.
Scheidt RA, Dingwell JB, and Mussa-Ivaldi FA. Learning to move amid uncertainty. J Neurophysiol 86: 971–985, 2001.
Shadmehr R and Brashers-Krug T. Functional stages in the formation of human long-term motor memory. J Neurosci 17: 409–419, 1997.
Shadmehr R and Moussavi ZM. Spatial generalization from learning dynamics of reaching movements. J Neurosci 20: 7807–7815, 2000.
Shadmehr R and Mussa-Ivaldi FA. Adaptive representation of dynamics during learning of a motor task. J Neurosci 14: 3208–3224, 1994.[Abstract]
Slotine JE and Li W. Applied nonlinear control. Upper Saddle River, NJ: Prentice-Hall, 1991.
Smith MA, Brandt J, and Shadmehr R. Motor disorder in Huntington's disease begins as a dysfunction in error feedback control. Nature 403: 544–549, 2000.[CrossRef][Medline]
Smith MA, Ghazizadeh A, and Shadmehr R. Interacting adaptive processes with different time-scales underlie short-term motor learning. PLoS Biol 4: e179, 2006.[CrossRef][Medline]
Thoroughman KA and Shadmehr R. Learning of action through adaptive combination of motor primitives. Nature 407: 742–747, 2000.[CrossRef][Medline]
Thoroughman KA and Taylor JA. Rapid reshaping of human motor generalization. J Neurosci 25: 8948–8953, 2005.
Todorov E. Optimality principles in sensorimotor control. Nat Neurosci 7: 907–915, 2004.[CrossRef][ISI][Medline]
Wolpert DM and Ghahramani Z. Computational motor control. In: The Cognitive Neurosciences, 3rd ed., edited by Gazzaniga MS. Cambridge, MA: MIT Press, 2004, p. 485–494.
This article has been cited by other articles:
|
|
 |
|
  M. S. Fine and K. A. Thoroughman Trial-by-Trial Transformation of Error Into Sensorimotor Adaptation Changes With Environmental Dynamics J Neurophysiol, September 1, 2007; 98(3): 1392 - 1404. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. A. Taylor and K. A. Thoroughman Divided Attention Impairs Human Motor Adaptation But Not Feedback Control J Neurophysiol, July 1, 2007; 98(1): 317 - 326. [Abstract] [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
