Although previous experiments have identified that errors in movement induce adaptation, the precise manner in which errors determine subsequent control is poorly understood. Here we used transient pulses of force, distributed pseudo-randomly throughout a movement set, to study how the timing of feedback within a movement influenced subsequent predictive control. Human subjects generated a robust adaptive response in postpulse movements that opposed the pulse direction. Regardless of the location or magnitude of the pulse, all pulses yielded similar changes in predictive control. All current supervised and unsupervised theories of motor learning presume that adaptation is proportional to error. Current neural models that broadly encode movement velocity and adapt proportionally to motor error can mimic human insensitivity to pulse location, but cannot mimic human insensitivity to pulse magnitude. We conclude that single trial adaptation to force pulses reveals a categorical strategy that humans adopt to counter the direction, rather than the magnitude, of movement error.
Humans can quickly adapt their motor output to interact with objects in their environment. This flexible control is necessary because of changing environmental dynamics—on short timescales due to external perturbations and on longer timescales due to growth and changing body proportions—or after the onset of neurological disease. The study of motor learning provides a unique window into the neural and computational processes underlying motor behavior by identifying specific transformations of sensation into action, rather than correlating neural activity to a particular component of movement. One such transformation transforms a desired movement trajectory into a prediction of the muscle forces or joint torques necessary to achieve the trajectory. Human psychophysical studies have revealed that people perform this inverse dynamic transformation when learning novel dynamics (Conditt et al. 1997; Gandolfo et al. 1996; Shadmehr and Brashers-Krug 1997; Shadmehr and Mussa-Ivaldi 1994).
To study motor learning, participants are often asked to make reaching movements while holding the handle of a robotic manipulandum that generates perturbing forces. Each unexpected perturbation challenges the nervous system to generate a within-movement feedback response; feedback may also generate a training signal to update the predictive control of arm-movement dynamics between movements (Kawato et al. 1987). Over a series of movements, training can restore desired performance, despite environmental perturbations.
Foundational psychophysical studies demonstrated that humans could quickly adapt their motor behavior to move skillfully in novel dynamic environments (Lackner and Dizio 1994; Shadmehr and Mussa-Ivaldi 1994). Recent experiments have shown how learning transfers across training sets after changing the movement direction (Gandolfo et al. 1996; Sainburg et al. 1999), hand position (Shadmehr and Moussavi 2000), movement speed (Goodbody and Wolpert 1998), or task (Conditt et al. 1997). These studies exposed subjects to perturbing forces for tens or hundreds of movements and then retested them in a baseline condition or novel environment. Such experiments can identify the time course of motor adaptation, but by integrating over several movements, cannot identify how specific errors inform that adaptation.
Left largely unexamined is how humans abstract information from one movement to update their prediction of environmental dynamics. Preliminary studies (Scheidt et al. 2001; Thoroughman and Shadmehr 2000) have used system identification to examine adaptation across single movements. This identification, coupled with neural network models (Thoroughman and Shadmehr 2000), has suggested that incremental motor adaptation is proportional to motor error and gated by the positions and velocities generated within specific movements. This hypothesis has explained motor learning in several viscous (Donchin et al. 2003) and position-dependent (Hwang et al. 2003) environments.
Previous studies trained subjects in a perturbing field that was either on or off during an entire movement. Within a single movement, however, subjects moved through a variety of positions and velocities. These studies integrated over the entire movement to identify how people transform sensed error into the next update of predictive control. In the present study, we consider how the precise features of error induce incremental updates in subsequent control.
We introduce transient perturbations to study the dependence of adaptation, across single trials, on the fine details of sensory feedback. To study this trial-by-trial response, we initially sought to minimize the formation of a motor memory over many movements by presenting pulses infrequently and by balancing leftward and rightward pulses. We found that movements perturbed with 70-ms force pulses induced aftereffects in the next, unperturbed movement that countered the direction of the pulse. These aftereffects were insensitive to the within-movement positional placement of the pulse. We next investigated the aftereffects following pulses of varying magnitude and, surprisingly, found that aftereffects did not scale with the pulse magnitude but instead coarsely countered the pulse direction. Last, we show that subjects can build motor memories after repeated exposure to force pulses.
We conclude that, in certain tasks and with certain perturbations, humans adapt to the direction, but not the magnitude, of movement error. This result was surprising, in light of the proportional responses measured previously in studies of trial-by-trial adaptation (Scheidt et al. 2001; Thoroughman and Shadmehr 2000), and suggests that the transformation of error into adaptation can change, depending on the dynamics of the environment.
Thirty-eight healthy, right-handed subjects performed reaching movements while holding the handle of a planar five-bar, two-link robotic arm (Interactive Motion Technologies, Cambridge, MA). All protocols were approved by Washington University's Hilltop Human Studies Committee, and all subjects gave informed consent. The manipulandum generates forces using two brushless torque motors; each torque motor controls a single link. Angular position was recorded with two rotational encoders (Gurley Precision Instruments, Troy, NY) and velocity via a built-in tachometer (Kollmorgen Motion Technologies, Commack, NY) at 200 Hz. All data were measured with respect to a rectangular coordinate frame centered over the workspace and coplanar with the center of rotation of a subject's shoulder. Subjects were seated such that their elbows were flexed 90° at movement start. This posture was the starting point of all reaching movements.
Movements were 10 cm in length, 0.5 s in duration, and in a single direction, away from a subject's body in the horizontal plane. All subjects trained for 2 days. On the first day, subjects trained without experiencing perturbing forces (the null-field condition); forces were introduced on the second day of training. On each day, subjects made 4 sets of 150 movements (experiments 1 and 3) or 180 movements (experiment 2). Subjects rested for approximately 3 min between sets.
The motor system is inherently noisy, and subjects' movements naturally varied from trial to trial. To minimize this variance, subjects were asked to “mimic” a training dot (experiments 2 and 3 only). The dot reproduced a straight-line, minimum jerk trajectory, 0.5 s in duration. On the first day, the dot appeared in 100% of movements in the first set, 75% in the second set, 50% in the third set, and 25% in the fourth set; on the second day, the dot appeared in 20% of movements but never on pulsed or postpulse trials. The training dot began after subjects exceeded the forward threshold for movement, 2.5 cm/s.
Full visual feedback was given throughout the experiment, either on a vertically mounted computer monitor (experiments 1 and 3) or projected on a horizontal screen mounted above the robotic arm (experiment 2). With the monitor, subjects moved their arms through a horizontal plane aligned with their shoulder; with the projection screen, subjects moved their hands through a horizontal plane under the screen, slightly below their shoulders. Timing feedback was given by changing the target color (red, <450 ms; green, 450–550 ms; blue, >550 ms). After a movement, the manipulandum carried subjects back to the starting location. Subjects' arms were supported against gravity by a sling attached to the ceiling by a 10-ft cable. Subjects were instructed to move quickly and smoothly to the target. When the training dot was available, they were instructed to use the dot to time their movements.
We perturbed the reaching movements of 20 subjects with pulses of force to determine how the within-movement position of a transient force pulse would affect the pulsed movement and the predictive control of subsequent movements. Subjects (9 women and 11 men) ranged in age from 18 to 27 yr (mean, 21.9 ± 2.5 yr). Pulses were 70 ms in duration with amplitude 18 N and had a bell-shaped force profile that was generated by truncating a Gaussian in time (Fig. 1B, inset). Subjects experienced 12 varieties of pulses: subjects were pulsed 2, 3, 4, 5, 6, or 7 cm into movement, either to the left or the right (±90° to the direction of motion). Force pulses were experienced pseudo-randomly in 20% of trials; only a single force pulse was experienced on each pulsed trial, and subjects experienced all pulse types within a single training set. Pulsed movements never occurred back to back. Thus, movements before and after pulsed trials were always made in a null field. Fifteen subjects were perturbed by force pulses introduced 3, 5, or 7 cm into movement. Five subjects experienced pulses at 3 additional positions: 2, 4, and 6 cm.
In a second experiment, we perturbed the reaching movements of 12 individuals with pulses of variable magnitude (6, 12, and 18 N). Pulses were 70 ms in duration, introduced 3, 5, or 7 cm into movement, and were directed either to the left or the right. Subjects (4 women and 8 men) ranged in age from 18 to 27 yr (mean, 23.0 ± 2.4 yr). Pulses were experienced pseudo-randomly in 20% of trials, but never back to back; all pulse types were experienced in a single set.
The first and second experiments were designed to interrogate subjects' incremental adaptive response. They were specifically designed to suppress the formation of a motor memory, by introducing pulses infrequently and equally balancing leftward and rightward forces. In a third experiment, we did just the opposite; we presented pulses frequently with a strong directional bias to determine whether subjects could form a motor memory over hundreds of pulsed movements.
Six subjects (2 women and 4 men), ranging in age from 19 to 24 yr (mean, 21.3 ± 2.1) participated in the third experiment. In the first two training sets, pulses were presented in every movement. Catch trials, trials in which the pulse was unexpectedly removed, were introduced pseudo-randomly in 20% of trials in the third and fourth sets. All pulses were 70 ms in duration, rightward-going, and experienced 5 cm into movement. The magnitude (6, 12, and 18 N), but not the direction, of the pulse changed pseudo-randomly from trial to trial.
To quantify subjects' trajectories, three performance metrics were used. Movement area quantified the signed area bounded by the trajectory and a straight line connecting the movement start and target. Perpendicular displacement (p.d.) quantified the horizontal displacement from movement start. Take-off angle, averaged over a 5 mm window 1 cm into movement, quantified the initial direction of movement. Take-off angles were measured with respect to a straight line connecting the movement start and target. Adaptation was quantified by subtracting prepulse metrics from postpulse metrics, where pre- and postpulse are defined as the movements immediately before and after a pulsed movement, respectively. By use of these performance metrics, we considered how the magnitude of the pulse and the position and velocity at which the pulse was experienced affected subsequent predictive control.
We used t-tests and analyses of variance (ANOVAs) to determine the significance of the above metrics and changes in metrics across pulse types. In ascertaining the significance of linear fits, we used an F-statistic, derived from mean square errors of residuals and fits.
We simulated a feedback controller to calculate a stiff and viscous feedback torque (equation ); the feedback signal was driven by changes in shoulder and elbow positions (q, [2 × 1]) and velocities (q̇, [2 × 1]) from their desired states (qd, q̇d). The desired trajectory was assumed to be minimum jerk and 0.5 s in duration. Stiff and viscous matrices were taken from the literature and assumed a joint-based coordinate system (Shadmehr and Mussa-Ivaldi 1994): (1) We transformed the position and velocity of the end effector into the angular positions and velocities of the shoulder and elbow joint. Kinematics were modeled assuming two-link, planar motion (Slotine and Li 1991). The feedback torque was converted to feedback force via a Jacobian transformation.
Adaptation to transient perturbations of constant magnitude
We investigated the specific, real-time components of motor learning by perturbing human reaching movements with 70-ms pulses of force at varying positions within a 10-cm movement. To focus on the trial-by-trial response, we designed each set to suppress the generation of a motor memory by setting the left/right perturbation bias to zero and introducing forces in only 20% of trials. We ask how error, abstracted from a single trial, changes the prediction of movement dynamics in the very next trial. Do the specific spatial details of feedback differentially affect incremental updates of control?
We began by observing the entire time-series of positions in prepulse, pulsed, and postpulse movements (Fig. 1) to characterize the normal human response. Prepulse trajectories (Fig. 1A) were relatively straight with a single peaked, bell-shaped velocity profile, curved only slightly off a minimum jerk trajectory (Flash and Hogan 1985). Force pulses noticeably perturbed movement trajectories (Fig. 1B). Pulsed movements were kinematically identical to prepulse movements until the force pulse was administered; thus, any changes in postpulse trajectories were due to the 70-ms force pulse only. Postpulse trajectories deviated from their prepulse counterparts (Fig. 1C). Leftward pulses reliably altered predictive control such that subjects moved rightward in the postpulse movement. Conversely, subjects compensated leftward after a rightward pulse.
Changes in predictive control were similar regardless of the position at which subjects were pulsed. Subjects adapted their predictive control starting very early in the next movement; postpulse movements deviated from prepulse trajectories almost immediately, regardless of when the pulse was experienced (Fig. 1D). The remainder of the postpulse movement opposed the pulse direction but was insensitive to the pulse placement. This analysis of full trajectories indicates robust adaptation to pulses that is noticeably insensitive to the position at which the perturbing pulse is experienced.
To quantify adaptation, movements were characterized by movement area. The average prepulse trajectory had an area of −0.47 cm2. To compare prepulse movements across pulse type, a correlation analysis was performed that compared subjects' mean performance within pulse type to the average prepulse movement, across pulse type and subjects. We found that prepulse movements were statistically indistinguishable across pulse type (ANOVA, P = 0.68); thus, subjects' expectation of force was constant in prepulse movements over the course of the experiment.
Pulsed movements were kinematically different across pulse position (Fig. 1B); the magnitude of the pulsed movement area decreased as pulse position increased (ANOVA, P < 0.001). Even when pulsed early in movement perpendicular displacement (p.d.) continued to increase until very late in movement. Subjects approached the target along a line perpendicular to the direction of movement, regardless of pulse type.
Although pulsed movements were different, postpulse trajectories were very similar. The adapted response, calculated by subtracting the average prepulse area from postpulse area, was similar regardless of pulse position within a pulse direction (ANOVA, leftward pulses, P = 0.87; rightward pulses, P = 0.63). On average, subjects adapted 1.17 cm2 and −0.80 cm2 following leftward and rightward pulses, respectively. The directional sensitivity of the adapted response was washed out in the next trial, two trials after the pulsed movement (t-test, P = 0.34).
Quantifying adaptation via an area metric sacrifices temporal resolution within the adapted movement. Therefore, the degree to which pulses perturbed prepulse trajectories was also quantified by subtracting subjects' average prepulse p.d. from postpulse p.d. Adaptation following leftward pulses was defined to be positive (postpulse p.d.-prepulse p.d. > 0, t-test, P < 0.001); adaptation to rightward pulses was defined to be negative (t-test, P < 0.001). Within a pulse direction, adaptation was statistically indistinguishable regardless of pulse position, whether p.d. was measured early (3 cm, Fig. 2A), middle (5 cm, Fig. 2B), or late (7 cm, Fig. 2C) in movement (ANOVA, rightward pulses: P = 0.44, 0.69, and 0.92 for p.d. measured at 3, 5, and 7 cm into movement, respectively; leftward pulses: P = 0.30, 0.44, and 0.46, respectively). Mid-movement adaptation was noticeably larger following leftward pulses (two-dimensional ANOVA with factors pulse position and pulse direction, P < 0.001).
Adaptation was also quantified by take-off angle (see methods). Clockwise rotations were defined as positive (following leftward pulses), and counterclockwise rotations, negative (following rightward pulses). Take-off angles were indistinguishable across pulse position for both leftward and rightward pulses (ANOVA, leftward pulses, P = 0.40; rightward pulses, P = 0.90), and significantly different from 0° (t-test, leftward pulses, P < 0.001; rightward pulses, P = 0.020). The average take-off angles, averaged across pulse position, were 1.2° and −0.6° for leftward and rightward pulses, respectively.
Although we only controlled for the position at which the pulse was experienced, pulses naturally occurred over a wide range of velocities (Fig. 3A). We could therefore investigate the relationship between the velocity at the time of pulse and subsequent adaptation (Fig. 3, B–D). Adaptation (quantified by p.d.) to leftward pulses was marginally dependent on the velocity at which the pulse was experienced, but only when measured in the middle of (5 cm) or late (7 cm) in movement (linear fit, 5 cm, P = 0.047; 7 cm, P = 0.041); across-trial adaptation was lowest when the pulse was experienced at the highest velocities (35–40 cm/s). This suggests that, unlike position, adaptation is not completely untuned to velocity. The effect, however, is small. Even in the most significant case (leftward pulses, p.d. at 7 cm), a linear fit (Fig. 3D, dashed line) predicted adaptation following 5-cm pulses (which are experienced near the peak of the speed profile, approximately 33 cm/s, on average) to be 75% of that following 2-cm pulses (which are experienced at a slower speed, approximately 23 cm/s). Furthermore, we found no significant relationship between the velocity at which a pulse was experienced and the subsequent change in p.d., when p.d. was measured after rightward pulses, early, in the middle of, or late in movement (P = 0.13, 0.13, and 0.18, respectively), and after leftward pulses, early in movement (P = 0.20).
Adaptation to transient perturbations of varying magnitude
In the first experiment, we found that subjects were insensitive to the position of the force pulse. In a second experiment, we specifically tested for sensitivity to pulse magnitude. Twelve right-handed subjects made 10-cm reaching movements in the horizontal plane. Force pulses were introduced either to the left or the right, 3, 5, or 7 cm into a movement, and were 6, 12, or 18 N in amplitude. Prepulse movements were again invariant across pulse types (P = 0.79). The kinematic error induced by the forces scaled with the magnitude of the perturbation (Fig. 4, A–C). If the forces pushed to the left, people reliably adapted by moving to the right, beginning at movement initiation. If the forces pushed right, people adapted by moving to the left. Adaptation was quantified with respect to the average prepulse movement using p.d. and take-off angle. Adaptation was again insensitive to pulse position, whether measured early, in the middle of, or late in movement (rightward pulses: P = 0.20, 0.25, and 0.29 respectively; leftward pulses: P = 0.33, 0.42, and 0.24). Adaptation was surprisingly also insensitive to the magnitude of the pulse (Fig. 4, D–F). Adaptation to leftward and rightward pulses was statistically indistinguishable regardless of pulse strength, whether p.d. was measured early (3 cm, Fig. 5A), in the middle of (5 cm, Fig. 5B), or late in (7 cm, Fig. 5C) movement (rightward pulses: P = 0.16, 0.26, and 0.41 respectively; leftward pulses: P = 0.79, 0.47, and 0.17). Take-off angles were also indistinguishable regardless of the pulse location (P = 0.96 and 0.36) or pulse magnitude (P = 0.48 and 0.73, for rightward and leftward pulses, respectively). Statistics were calculated using a balanced, two-dimensional ANOVA with factors pulse position and magnitude.
As an additional test of magnitude sensitivity, we averaged across pulse position and plotted the response (postpulse p.d. − prepulse p.d.) as a function of pulse magnitude (±6, ±12, and ±18 N), and asked whether the response was significantly different from a proportional response (Fig. 6). If the postpulse response was proportional to pulse magnitude, after effects following 18 N pulses should be triple the magnitude of after effects following 6 N pulses. After effects after 18 N pulses were instead comparable in size to those following 6 N pulses; the difference between this flat response and a proportional response was significant for both leftward (ANOVA, P < 0.001) and rightward (P < 0.002) pulses. Therefore, we conclude that the average human response was not proportional to pulse magnitude.
To test whether subjects were adapting to the average pulse strength (within a pulse direction), we looked for linear trends in the data (postpulse p.d. − prepulse p.d.), across movements. For five of six pulse types, we found no differences in the adaptive response across sets (the slope of a linear fit was not significantly different from 0, P = 0.90, 0.47, and 0.99 for 6, 12, and 18 N leftward pulses; P = 0.85 and 0.44 for 12 and 18 N rightward pulses). Only the response to 6 N, rightward pulses varied significantly; this response showed a very small, yet significant, decrease in magnitude across movement sets (slope fit, P = 0.048). An averaging mechanism, however, would predict a slope with increasing magnitude. We therefore see no evidence that subjects began training a with magnitude sensitivity that waned over the course of training; subjects were just as insensitive to pulse magnitude throughout the training sets.
This result was surprising, in light of the proportional responses measured in novel dynamic environments. This component of adaptation therefore cannot scale with any previously hypothesized error signal, nor can it depend on a real-valued critical metric. This result, combined with previous studies, suggests that people can adopt different modes of adaptation that can or cannot scale with error.
Force pulses and motor memory formation
The first two experiments were designed to look explicitly at incremental steps of adaptation across single movements. To that end, we suppressed the formation of a motor memory by introducing forces in only 20% of trials, and by setting the left/right bias of those forces to zero. In light of the surprising results of the previous experiment, we now ask, do subjects form a motor memory after repeated exposure to 70 ms pulses of variable magnitude, or are the forces so nonecological that they are unlearnable? Six subjects made four sets of 150 reaching movements in a single direction. Subjects experienced rightward pulses with magnitude 6, 12, or 18 N in every trial in the first two training sets; all pulses were applied 5 cm into movement. Catch trials, trials in which the force pulse was unexpectedly removed, were introduced pseudo-randomly in 20% of trials in the third and fourth sets.
Initially, pulsed-movement trajectories scaled with magnitude and were consistent with those trajectories seen previously (Fig. 7, A–C, solid lines). With practice, subjects reduced pulse-induced curvature (Fig. 7, A–C, dashed lines). Adaptation was quantified by movement area, which subjects significantly reduced with training (P = 0.048, 0.026, and 0.017 for 6, 12, and 18 N pulses, respectively). Overall, subjects adapted to the mean pulse strength. Subjects under-adapted to the strongest, 18 N pulses, as evidenced by a rightward, pulse-induced hook, and over-adapted to the weakest, 6 N pulses, as evidenced by a hook in the opposite direction of the pulse.
Catch trials late in training resulted in large after effects that opposed the direction of the force pulse, which is indicative of motor memory formation. Aftereffect area (Fig. 7D, dashed line), −1.89 cm2, averaged across the first 10 catch trials of the third set, was significantly different from the average null-field movement area (Fig. 7D, solid line), −0.15 cm2 (P < 0.001). Aftereffects deviated from their null-field counterparts early in movement; p.d. continued to increase until very late in movement.
Previous studies have shown trial-by-trial adaptation to perturbations that persisted throughout the execution of single movements (Scheidt et al. 2001; Thoroughman and Shadmehr 2000). We adopted a different perturbation paradigm to interrogate the within-movement specificity of motor adaptation. We chose to use 70-ms pulses of force (Fig. 8A), which elsewhere have been used to interrogate feedback control in normal and neurologically impaired subjects (Smith et al. 2000). The specificity of the force pulse enabled a novel identification of the temporal and spatial resolution of trial-by-trial motor adaptation. Pulses generated kinematic errors perpendicular to the movement direction. These errors were similar in shape and amplitude but shifted in time (Fig. 8B). Errors coincided with different parallel positions and velocities. These errors drove a feedback response (simulated by multiplying actual positions and velocities by stiffness and viscosity matrices; see methods), the shape and magnitude of which was again preserved; only the within-movement timing varied (Fig. 8C).
Previous work has demonstrated a broad generalization of movement-by-movement adaptation across positions (Hwang et al. 2003) and velocities (Donchin et al. 2003; Thoroughman and Shadmehr 2000). We saw no significant sensitivity to the positional context of movement error; aftereffects start at the beginning of movement regardless of whether error is experienced early, in the middle of, or late in the previous trial. This was reasonable, because the within-movement pulse-placement domain (5 cm) was smaller than the approximately 12-cm positional tuning evidenced by previous across-movement generalization studies (Hwang et al. 2003). Adaptation was weakly dependent on the velocity at which the pulse was experienced; velocity-sensitivity was only significant in the middle of and at the end of postpulse movements when the pulse was directed leftward. Even the significant sensitivity was small and maintained a substantial transfer of adaptation across movement velocities. We conclude that people broadcast information broadly across all components of individual movements.
We then perturbed subjects with force pulses of magnitude 6, 12, and 18 N (Fig. 8D). Pulses generated position (Fig. 8E) and velocity errors that scaled with pulse magnitude; these errors drove a feedback response that also scaled with magnitude (Fig. 8F). Dynamic error, kinematic error, and the feedback response, all candidate error signals in current theories of motor adaptation, therefore all scaled with the magnitude of the force pulse. Subjects displayed clear sensitivity to the direction of the pulse but no sensitivity to the pulse magnitude. Adaptation following 6 and 18 N pulses, averaged across pulse position, showed no significant difference (Fig. 6); a proportional response would require adaptation to be three times larger following 18 N pulses than 6 N pulses.
Aftereffects were approximately 10% of the maximum kinematic error. Previous experiments (Thoroughman and Shadmehr 2000; Thoroughman and Taylor 2005) have identified a 25%–30% transfer of error; the small size of the 18 N after effect, therefore, is unlikely to be caused by saturation. Recent loss-function estimates (Kording and Wolpert 2004) suggest that large errors could result in a less than proportional response, but the approximately 0.5-cm errors experienced here are smaller than those used previously to identify proportionality in adaptation (Scheidt et al. 2001). This insensitivity is in opposition to the natural proportionality of signals available in the pulsed movement; seemingly, subjects transformed the proportionality of their feedback response through a function that flattened the response to the magnitude of the error while maintaining or amplifying dependence on error direction.
This result contradicts the fundamental assumptions of a wide variety of biological and theoretical learning hypotheses. Theories of motor learning include adaptation driven by motor error and feedback control induced by sensory error. (Wolpert and Ghahramani 2004). Each of these theories scales adaptation with the magnitude of previous error. This core hypothesis dates back to the Marr-Albus model of the cerebellum, which suggests that the firing rate of cerebellar climbing fibers explicitly encodes the size of the error (Kawato 2002). Even reinforcement learning, which lacks a specific target behavior to which quantified error can be calculated, features learning steps that depend on expected reward and scale with error magnitude (Barto 2003; Gullapalli 1990).
Proportionality to error also extends beyond motor theories to the learning rules of back propagation (Rumelhart and McClelland 1986) and adapting basis function networks (Pouget and Snyder 2000). Recently, basis functions have been used to quantify the generalization of motor error across the movement space (Donchin et al. 2003; Hwang et al. 2003; Thoroughman and Shadmehr 2000). These models relied on two assumptions: 1) the nervous system registered the experienced error with the coincident movement state, such that weight updates were specific to the neurons that caused the error, and 2) each increment of error generated a proportional change in the subsequent response. Our second experiment invalidated this second assumption; we witnessed surprising insensitivity to error magnitude. Therefore this architecture cannot predict the disproportionality seen across different magnitudes. Rather, it maintains a strictly proportional transformation of error into adaptation.
These hypotheses predict the observed sensitivity to pulse direction but fail to predict the observed insensitivity to pulse magnitude. We are not suggesting that the insensitivity we observed completely disproves these hypotheses; several studies of human motor learning are consistent with prior theories. Recently, for example, system identification techniques, each presuming proportionality, have been used to explain adaptation to position-dependent (Hwang et al. 2003) and viscous forces (Donchin et al. 2003; Thoroughman and Shadmehr 2000), as well as to visuo-motor perturbations (Baddeley et al. 2003). The presumption of proportionality in these analyses was not explicitly tested, but the good fits of the state-space models are consistent with theoretical predictions.
Experiments 1 and 2 and their subsequent analyses interrogate the trial-by-trial response of the adaptive system. This focus on short-term learning left the question: because people have difficulty learning time-dependent forces (Conditt and Mussa-Ivaldi 1999; Karniel and Mussa-Ivaldi 2003), could the unusual temporal properties of the perturbation prevent subjects from building a motor memory? In our third experiment, we found that subjects could build and maintain improved performance when adapting to 70-ms pulses that varied in magnitude, but not direction. Furthermore, in catch trials, subjects moved opposite the pulse direction, revealing motor memory formation. We do not know whether the motor adaptation learned in these experiments is dependent on time or on state, such as hand position or velocity. Therefore, we do not refute that adapted representations are state-dependent (Conditt and Mussa-Ivaldi 1999). The learnability of the pulses, however, indicates that, in experiments 1 and 2, the measured sensitivity (to direction) and insensitivity (to position and magnitude) were not artifacts of the unusual timing and shape of the perturbing force.
Proportionality has been explicitly demonstrated in stochastic environments, when viscous forces are applied throughout movement and when the gain is drawn from a biased distribution with nonzero mean (Scheidt et al. 2001). We have identified three factors that differ between this study and our first two experiments: the duration of force within each movement, the proportion of movements within which forces are experienced, and the directional bias of the distribution of forces. We designed experiment 3 to determine whether subjects could adapt over many movements when consistently exposed to biased forces; subjects did build a motor memory over many trials. This consistent exposure to pulses did not enable the single trial analysis we performed in experiments 1 and 2. The adaptation occurring in experiment 3 may, in fact, differ from trial-by-trial adaptation quantified in experiments 1 and 2. We hypothesize that protocols that systematically interpolate between ours and that of Scheidt et al. (2001) will determine the necessary experience to induce trial-by-trial adaptation that partially or completely transitions between disproportional and proportional adaptations.
These permutations of experimental paradigms will determine the conditions under which disproportional adaptation occurs but will not necessarily elucidate why people switch between adaptive strategies (Fig. 9). We hypothesize that subjects adopt one of three strategies to transform real valued error into incremental adaptation. A first strategy is zero adaptation (Fig. 9A); sensory experiences can fail to drive adaptation because of a lack of magnitude, relevance, or consistency (Sanger 2004). If a response is elicited, previous theories of motor adaptation have posited that learning strives to minimize a real-valued movement error; machine learning theory predicts that these strategies regress across error (Alpaydin 2004). Instead, we see trial-by-trial adaptation that categorizes error into leftward or rightward bins, effectively processing sensed error through a left/right categorization (Fig. 9B) before generating an adaptive response. This categorization generates a binary signal to drive incremental adaptation; this process therefore rejects real-valued error minimization in favor of minimizing a cost that values only the direction of error.
Machine learning theory describes a classification learning scheme by which a system trains to classify inputs into a small number of groups (Alpaydin 2004). We do not hypothesize that our subjects learn to classify experiences during training, because after effects are insensitive to pulse magnitude from the beginning of training. We instead hypothesize that subjects have previously learned a leftward/rightward classification and then applied this classification to generate aftereffects in lieu of a regressive, proportional learning strategy (Fig. 9C).
A similar partitioning of sensory space is well studied in categorical perception (Kuhl 2004). Exposing infants to natural speech patterns causes the categorization of a continuum of sounds into distinct consonants (Kuhl 2000). Partitioning stimulus space into distinct categories has also been hypothesized to underlie visual, auditory, and tactile object recognition (Kohonen and Hari 1999; Kuhl 2000). Repeated presentation of continuous stimuli with relevant categories also leads to a physical partitioning of the cortical representation of sensation (Kohonen and Hari 1999). We hypothesize that the disproportional aftereffects amount to categorical adaptation, as the leftward/rightward partition of sensed error drives adaptation instead of continuous metrics. This may indicate, for this class of perturbations, a neural representation of movement error that is categorical rather than continuous. This is different from categorical perception. Subjects do not adopt this categorization scheme immediately; the mid-movement, error-induced feedback response scales proportionally with pulse magnitude. Instead, categorization is applied across movements, to the transformation of sensed error to adaptation.
Our focus is consistent with experiments and theories that primarily ascribe motor adaptation to changes in the predictive control of dynamics; even the most advanced trial-by-trial analyses (Smith et al. 2006) account for complex adaptive behaviors by fixing the modeled feedback response while changing predictive, feed-forward control. To investigate trial-by-trial behavior we applied pulses infrequently, and therefore we could not test for the possibility that pulses induced changes in the feedback response (Burdet et al. 2001; Todorov 2004). It is possible that categorical adaptation of predictive control leaves intact changes in the feedback response that remain proportional to sensed error. We nevertheless identify adaptive changes early in movement that can be ascribed only to adaptive predictive control; these adaptations relate to error categorically, not proportionally.
The richness of human motor behavior arises from our ability to learn new movements. Trial-by-trial motor adaptation has been hypothesized to depend on two terms: an abstraction of a quantitative error metric from a movement and the application of that error to update the predictive control of the next movement. We designed our first experiment to investigate the within-movement sensitivity of the second term: would the position or velocity at which error was experienced differentially influence the adaptive response? Although we found the adaptive process to depend subtly on the speed at which error was sensed, it was largely insensitive to the fine context within which the error was experienced. We then altered our protocol to expose subjects to errors of varying magnitude. To our surprise, aftereffects did not increase in magnitude with increases in induced error. Our experiments therefore suggest, for this set of perturbations, insensitive encoding in both terms of adaptation; insensitivity to error magnitude in the first term, and insensitivity to movement context in the second. Coupled with a recent discovery of flexible error generalization across movement direction (Thoroughman and Taylor 2005), we now consider both components of learning to adapt as a function of environmental experience.
This work was supported by the McDonnell Center for Higher Brain Function at Washington University.
We thank D. N. Tomov for technical management, and M. A. Smith, D. L. Barbour, D. W. Moran, K. J. Feller, J. A. Taylor, and S. M. Springer for helpful comments.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2006 by the American Physiological Society