Journal of Neurophysiology

Trial-by-Trial Transformation of Error Into Sensorimotor Adaptation Changes With Environmental Dynamics

Michael S. Fine, Kurt A. Thoroughman

Abstract

Humans can rapidly change their motor output to make goal-directed reaching movements in a new environment. Theories that describe this adaptive process have long presumed that adaptive steps scale proportionally with error. Here we show that while performing a novel reaching task, participants did not adopt a fixed learning rule, but instead modified their adaptive response based on the statistical properties of the movement environment. We found that as the directional bias of the force distribution shifted from strongly biased to unbiased, participants transitioned from an adaptive process that scaled proportionally with error to one that adapted to the direction, but not magnitude, of error. Participants also modified their response as the likelihood of the perturbation changed; as the likelihood decreased from 80 to 20% of trials, participants adopted an increasingly disproportional strategy. We propose that people can rapidly switch between learning processes within minutes of experiencing a novel environment.

INTRODUCTION

People readily adapt their behavior to learn new movements, manipulate new tools, and respond to sensory stimuli. Sensorimotor adaptation also underlies the maintenance of function as our bodies change because of fitness and aging. The importance of adaptation becomes especially prominent after the onset of neurological disease. Many treatments for neurological deficits caused by degenerative disorders, stroke, and trauma (Krakauer 2006) engage unaffected brain tissue to overtake function from the damaged brain; such plasticity is possible only through learning.

Classic theories of learning (Jordan and Rumelhart 1992; Kawato et al. 1987; Pouget and Snyder 2000; Wolpert and Flanagan 2002) postulate that adaptation scales proportionally with the size of an error. Mathematically, this strategy drives error to a minimum (Alpaydin 2004). Behavioral studies have provided some correlative evidence for this strategy (Scheidt et al. 2001), but explicit tests remain sparse. A recent study (Kording and Wolpert 2004) has suggested that large errors are somewhat undervalued, but this and other studies presume learning strategies that are not influenced by experience.

We recently asked participants to make reaching movements while holding the handle of a robotic arm that generated perturbing forces. We showed that, in contrast to classic theories, the human response to infrequent, pulsatile perturbations contained no proportionality to any error signal (Fine and Thoroughman 2006). Participants instead implemented a categorical strategy; a participant adapted to the left after a rightward pulse and to the right after a leftward pulse, but the magnitude of their response was insensitive to the magnitude of the error. This result established that certain stimuli can induce alternate strategies that are not necessarily proportional to error but did not address what properties of the environment induced that strategy.

Here we use another arm-reaching paradigm and ask how two properties of the environment, the likelihood a force will occur, and the directional bias of the force distribution, affect adaptive sensorimotor processes. Three groups of participants made reaching movements away from their bodies while experiencing viscous perturbations. Two groups experienced viscous forces whose directional bias transitioned from unbiased—forces were presented equally to the left and right—to strongly biased in a single direction. The third group experienced forces whose direction was fixed, but changed likelihood; forces were experienced in 20, 50, or 80% of trials.

We found that participants modified their adaptive strategy to move in each novel environment. As the directional bias of the force distribution transitioned from unbiased to strongly biased, participants transitioned from a categorical to a proportional strategy. In addition, participants adopted a markedly disproportional strategy as the likelihood of the perturbation decreased from 80 to 20%. We suggest that the same stimulus can elicit a different response depending on the statistical properties of the environment and that the careful design of experimental protocols can switch the functional processing of error from proportional to categorical.

METHODS

Experiment

Thirty right-handed participants made 10-cm reaching movements while holding the handle of a planar five-bar, two-link robotic manipulandum (Interactive Motion Technologies, Cambridge, MA) that generated viscous perturbations on pseudorandomly chosen trials. All protocols were approved by the Washington University Hilltop Human Studies Committee. Participants moved to a single target, located directly away from their bodies in the horizontal plane. All movements started at the origin of a rectangular coordinate system centered over the workspace and aligned with participants' shoulders. Positive x- and y- were defined as to the right and forward, respectively. Participants' arms were supported against gravity by a sling attached to the ceiling by a 10-ft cable. Full visual feedback was given throughout the experiment. Targets were projected onto a screen mounted above the manipulandum's handle. The desired movement duration was 500 ± 50 ms, and timing feedback was given by changing the target color (red, <450 ms; green, 450–550 ms; blue, >550 ms). The robot carried participants back to the starting location after a movement. Data were collected at 200 Hz.

Two groups (groups 1 and 3; Table 1) consisted of 12 participants each. A third group (group 2) consisted of six participants. Participants ranged in age from 18 to 25 yr [group 1, 21.9 ± 2.15 (SD) yr; group 2, 19.5 ± 2.81 yr; group 3, 21.5 ± 1.38 yr]. Participants trained for 4 days; each day, they made four sets of 180 movements. Participants rested for 3 min between sets.

View this table:
TABLE 1.

Participant groups

On the first day of training, participants trained without the robot generating any force. On days 2–4, participants experienced viscous forces (as in Scheidt et al. 2001) Math(1) where and are the Cartesian components of velocity. The force direction was determined by Bn, the gain of the nth movement.

The first group (group 1) of participants experienced forces in 80% of trials. The viscous strength (B, Eq. 1), was drawn from a distribution either strongly biased [B = (0, −6, −12, −18, −24, −30, −36) Ns/m to the left, weakly biased [B = (18, 9, 0, −9, −18, −27, −36) Ns/m] to the left, or unbiased [B = (36, 24, 12, 0, −12, −24, −36) Ns/m]. Only one bias condition was experienced each day. Another group of six participants (group 2) experienced forces biased to the right. The final group (group 3) experienced forces in 20, 50, or 80% of trials. The strength of the force was drawn from a distribution strongly biased to the left; the magnitude, but not the direction of the force varied from trial to trial. The day-to-day presentation of factors (i.e., 20, 50, 80%, or zero, weak, strong) was shuffled across participants to counterbalance for across-day effects. Within a single factor, forces were pseudorandomly presented; however, each participant experienced the same forces in the same order.

Participants used a training dot to time their movements. The dot swept out a straight-line, minimum jerk trajectory, 0.5 s in duration. While learning the baseline task (day 1), the dot was visible in 100, 75, 50, and 25% of trials in sets 1–4, respectively. On subsequent days, the dot was visible on 20% of pseudorandom trials; the dot was never visible on catch trials, nonforce trials immediately after trials in which the force was present. Participants were instructed to move quickly and accurately to the target and to use the training dot to time their movements.

Analysis

We analyzed adaptation using a state-space model (Eq. 2) Math(2) The output of the model, movement error (yn), equaled the difference between the mid-movement (5 cm) perpendicular displacement (p.d.) and the average null-field movement, calculated by averaging the last 50 movements, across participants, on day 1. Movement error depended on a scalar parameter D and the prediction error (Bnn), the difference between the actual gain (Bn) and the modeled estimate of gain (n). The modeled estimate was a function of the previous estimate weighted by a scalar A and a sensitivity vector S⃗ (1 × 7), which parameterizes how adaptation depends on each viscous gain. Each element of the sensitivity vector corresponded to one of seven viscous gains [e.g., for a distribution strongly biased to the left, B = (0, −6, −12, −18, −24, −30, −36), where B = 0 represents a nonforce trial]. For example, if a participant was completely insensitive to the weakest perturbation, B = −6, i.e., if that perturbation did nothing to change participants' estimate of gain, in the example above, the second element of the sensitivity vector would equal zero. If participants used a proportional strategy, sensitivity elements would increase proportionally with their corresponding gains. If instead, all forces led to equal updates in prediction, each element of the sensitivity vector would be equal.

The vector p⃗n (1 × 7) was designed so only a single element of the sensitivity vector contributed to the update of the estimate. An element of p⃗n equaled 1 when the corresponding perturbation was applied; elements equaled 0 otherwise. We used the Gauss-Jordan method to optimize for the D, A, and S⃗ that minimized the squared-difference between predicted and actual participant performance.

We also fit the data with a model that assumed proportionality Math(3) We used the Gauss-Jordan method to optimize for the D, A, and α (a constant) that minimized the squared-difference between predicted and actual participant performance.

The shape of the sensitivity vector was quantified using two metrics: a slope metric, derived from a linear fit of sensitivity elements plotted against viscous gain or p.d., and an asymmetry index (AI), which equaled the sum of sensitivity elements.

Model performance was assessed using the root-mean-square of the residual error, and the variance accounted for (VAF). The VAF was given by the variance of the residual error divided by the variance of the data, subtracted from 1 Math(4)

Feedback analysis

We quantified the within-movement feedback response using two metrics: the maximum p.d. and the integrated p.d. The integrated error was calculated from the time of peak perpendicular force to the end of movement.

We also quantified the feedback response using a linear model of displacement in the perpendicular (x) direction Math(5) where x, ẋ, and Math equal the human time-series' of x (perpendicular) position, velocity, and acceleration, equals the forward movement velocity, and Fx equals the robot-generated force in the x-direction. We related these known signals with four fitted parameters: an effective mass m, a stiffness k, a parameter η that multiplied stiffness to determine viscosity (Shadmehr and Mussa-Ivaldi 1994), and a parameter γ that estimated the human prediction of the perpendicular force, x, by multiplying the parallel velocity (). τ delayed the x-position and velocity by 70 ms, consistent with short-loop feedback control (Allum 1975). For each environment, we optimized for a single m, γ, and η and determined how the stiffness, k, changed with the gain of the perturbation.

We initially tested a model that used two delays, τ = 70 and τ = 140 ms, indicative of short- and long-loop feedback control, respectively. When both delays were present, we found that the response was dominated by the short-loop reflex; the long-loop response did not significantly contribute to the overall estimate of acceleration. We therefore modeled the response with only a single, short-loop delay (τ = 70 ms).

We also tested a model that contained a single, long-loop delay (τ = 140 ms); increasing the delay to 140 ms had no effect on the relationship between the stiffness, k, and the gain of the perturbation, B; however, the optimization returned values for m and η that were not physiologically plausible (for more, see discussion).

Statistical significance

We used t-tests and standard bootstrapping techniques to determine statistical significances (Efron and Tibshirani 1998). When using a t-test to ascertain changes in slope, we subtracted slopes within participants and compared the difference across participants.

When bootstrapping, we randomly drew a participant from the participant pool (groups 1 and 3, 12 participants; group 2, 6 participants), replaced that participant, and drew again. We repeated this draw 12 times (for groups 1 and 3; 6 times for group 2) and averaged the positional data across the resampled population of participants. We used the Gauss-Jordan method to optimize for the D, A, and S⃗ that minimized the squared-difference between the predicted and resampled data; using S⃗, we calculated the slope and/or AI. We repeated this process 1,000 times, and the distribution of metrics was sorted across all 1,000 samples to determine P values and CIs. P < 0.05 was considered significant. In Figs. 8 and 9, positional data were averaged within each sample before calculating the slope; therefore the x- (positional error) and y-data (sensitivities) changed across each sample.

FIG. 1.

Force profiles. Participants made reaching movement in the horizontal plane while experiencing a viscous perturbation. The gain of perturbation was drawn from 1 of 3 distributions, each with a different directional bias: (A) unbiased [zero bias, B = (−36, −24, −12, 0, 12, 24, 36) Ns/m], (B) weakly biased [B = (−36, −27, −18, −9, 0, 9, 18)], or (C) strongly biased to the left [B = (−36, −30, −24, −18, −12, −6, 0) Ns/m]. When B < 0, the perturbation pushed the hand leftward; when B > 0, it pushed rightward.

FIG. 2.

Full movement trajectories, averaged across all movements with like viscous gain, across all participants, for the zero (A), weak (B), and strong bias conditions (C). We focused our analysis on mid-movement (5 cm) perpendicular displacement (p.d.), taken from intersection of colored traces with dotted black line.

FIG. 3.

Feedback response, as quantified by the stiffness, k, was constant, regardless of gain of perturbation, B, or bias conditions.

FIG. 4.

Participants responded to each perturbation by adapting in the very next movement. The magnitude of the response depended on the directional bias of the force distribution. The slope of a linear fit (dotted lines) increased significantly from the zero to strong (bootstrap, P = 0.046) bias conditions (see bar plot). The error bars represent the 95% CI of the mean.

FIG. 5.

Elements of the sensitivity vector, plotted against their corresponding perturbation gains, for each leftward-bias condition. We quantified participants' adaptive strategy using a state-space model that did not explicitly presume proportionality. To learn, the model updated its prediction of viscous gain. The model's prediction of gain depended on its previous prediction and sensitivity parameter vector, S⃗, which identified how participants' strategy changed with directional bias of force distribution. We found that the same participants changed their sensitivity, to the same forces, when those forces were presented with varying directional bias. The slope of a linear fit (dotted lines) changed significantly as the environment transitioned from the zero to strong (bootstrap, P = 0.011) and weak to strong (bootstrap, P = 0.019) bias conditions (see bar plot). The error bars represent the 95% CI of the mean.

FIG. 6.

Model fits. The state-space model predicted human behavior (p.d., 5 cm into movement). Averaged across participants, it accounted for >98% of the variance for the zero (A), weak (B), and strong (C) bias conditions.

FIG. 7.

Elements of the sensitivity vector, plotted against their corresponding perturbation gains, for each rightward-bias condition. A 2nd group of participants experienced forces that were unbiased, weakly biased to the right [B = (−18, −9, 0, 9, 18, 27, 36) Ns/m], or strongly biased to the right [B = (0, 6, 12, 18, 24, 30, 36) Ns/m]. We quantified adaptation using a state-space model; the model's sensitivity vector, S⃗, identified how participants changed their adaptive strategy in each environment. Participants transitioned from categorical to proportional adaptive strategies as the bias transitioned from zero to strong. The slope of a linear fit (dotted lines) increased as participants transitioned from the zero to weak (bootstrap, P = 0.024), zero to strong (bootstrap, P < 0.001), and weak to strong (bootstrap, P = 0.006) bias conditions. The error bars represent the 95% CI of mean, (see bar plot).

FIG. 8.

Elements of the sensitivity vector, plotted against their corresponding perturbation gains, for each likelihood condition [all likelihood conditions were strongly biased to the left, B = (0, −6, −12, −18, −24, −30, −36) Ns/m]. The same participants increased their sensitivity, to the same forces, when the perturbation's likelihood increased from 20 to 80%. The slope of a linear fit (dotted lines) increased as the perturbation likelihood increased from 20 to 80% (bootstrap, P = 0.005) and from 50 to 80% (bootstrap, P = 0.016). The error bars represent the 95% CI of the mean (see bar plot).

FIG. 9.

Elements of sensitivity vector plotted against average mid-movement p.d., for each bias (leftward) condition. The slope of a linear fit (dotted lines) changed significantly as the environment transitioned from the zero to strong (bootstrap, P = 0.022) and weak to strong (bootstrap, P = 0.010) bias conditions. The error bars represent the 95% CI of the mean (see bar plot).

To test for across-day effects, we resampled the data 1,000 times, with replacement; for each random draw of participants, we calculated a timing index—the day, on average, the zero bias condition was experienced. We divided the timing index into the mean-subtracted AI and slope metrics and asked if the result was different from zero.

RESULTS

Changing the directional distribution of forces

To determine if the statistical properties of the movement environment could change the trial-by-trial adaptive response, we asked a group of 12 human participants to make reaching movements while holding a robotic arm. The arm generated a viscous force that pushed participants to the left or right. The forces could be unbiased (equally balanced to the left and right), weakly biased to the left, or strongly biased to the left, and were experienced in 80% of trials in each bias condition (Fig. 1).

We averaged individual movement trajectories across all replicates of a particular viscous gain and collapsed across participants (Fig. 2). We began our analysis by quantifying participants' within-movement feedback response using two metrics: the maximum p.d. and the integrated error. We plotted both metrics against viscous gain and found that the slope of a linear fit was similar for all bias conditions (ANOVA, maximum p.d., P = 0.90; integrated error, P = 0.97).

In addition, we calculated the strength of the feedback response through a linear parameterization (for details, see methods, Eq. 5) that quantified how participants used errors in position and velocity to generate a corrective response. The feedback response (as quantified by the strength of the stiffness, k) was constant, regardless of the gain of the perturbation, B, for any bias condition (ANOVA, P = 0.53, 0.67, and 0.44 for the zero, weak, and strong bias conditions, respectively; Fig. 3). In addition, the slope (k vs. B) was not significantly different from 0 (t-test, P = 0.85, 0.95, and 0.29 for the zero, weak, and strong bias conditions, respectively) and was constant across conditions (ANOVA, P = 0.60). Averaged across the gain of the perturbation, k equaled 198 ± 35, 208 ± 27, and 204 ± 34 kg/s2 for the zero, weak, and strong bias conditions, respectively, where the ± interval equals the 95% CI of the mean (for additional coefficient estimates, see appendix). We conclude that the feedback generated by participants was constant across all conditions, regardless of the directional bias of the force distribution.

Participants responded to each unexpected perturbation (n) by adapting in the very next movement. Adaptation was defined by subtracting the movement before a perturbation (n − 1) from the movement after (n + 1) and calculating the change in mid-movement (5 cm) p.d. Both movements (before and after) were mean-corrected for the viscosities experienced in those movements (Fig. 2, intersection of colored traces with dotted black line) before subtraction.

We found that the magnitude of the response depended on the directional bias of the force distribution; the same participants responded differently in different bias conditions (Fig. 4). Participants transitioned from a strategy that was more proportional to one that was more categorical as the bias transitioned from strong to zero.

We calculated the slope of the adaptive response as a function of leftward viscosities (see Fig. 4, B < 0, dotted lines). The slope increased as the bias conditions transitioned from zero to strong (slope = −0.0027 ± 0.0024, −0.0046 ± 0.0026, and −0.0066 ± 0.0025 for the zero, weak, and strong bias conditions, respectively, where the ± interval equals the 95% CI of the mean). It increased significantly from the zero to strong bias conditions (t-test, P = 0.047), and increased, although not significantly, from the zero to weak (t-test, P = 0.12) and weak to strong bias conditions (t-test, P = 0.32). Because the zero-bias condition was equally balanced to the left and right, we also quantified a second slope metric, calculated from a linear fit of the data points corresponding to positive gains (s+, B > 0; Fig. 4). The average of both leftward and rightward slope metrics (s and s+) was significantly less than the slope of a line connecting B = −12 and B = 12 (t-test, P = 0.027), indicative of a nonlinear strategy.

Did participants adapt differently as they performed more trials? To rule out learning effects, we looked for linear trends in the data. For each bias condition, we asked if the time series of adaptation (after − before, at fixed gain), had a slope significantly different from zero. We found no difference in the response for any viscosity or bias condition (for all, P > 0.05). For the zero-bias condition, participants were just as insensitive to changes in viscosity at the beginning of training as they were at the end.

State-space analysis

Our previous analysis estimated adaptation across single trials by comparing movements before (n − 1) and after (n + 1) all replicates of a particular viscous gain. Although we corrected for the mean displacement induced by the robot in these movements, each before movement (n − 1) was preceded by a movement (n − 2) that was also perturbed. Our previous analysis cannot account for this added complexity. To quantify the trial-by-trial evolution of adaptation in each environment, we turned to a novel state-space analysis (Eq. 2). Previous models (Donchin et al. 2003; Scheidt et al. 2001; Smith et al. 2006; Thoroughman and Shadmehr 2000; Thoroughman and Taylor 2005) presumed proportionality and therefore could not account for alternative strategies. Here, we modeled adaptation using a set of state-space equations that did not explicitly presume proportionality. The model updated its prediction of the viscous gain, B, depending on the previous prediction and a sensitivity parameter vector, S⃗. Through S⃗, the model identified how the adaptive strategy depended on the directional bias of each environment (for details, see methods).

We plotted each element of the sensitivity vector against its corresponding viscosity for each bias condition (Fig. 5; for additional coefficient estimates, see appendix). We found that participants transitioned from a largely categorical strategy to one that was more proportional, as the force bias transitioned from zero to strong. We quantified the shape of the sensitivity vector using a slope metric; again, slope was calculated from a linear fit of the sensitivity elements with negative gain (s, B < 0; Fig. 5). The slope increased significantly as the bias condition changed from weak to strong (bootstrap, P = 0.019; slope = 0.036 ± 0.018 and 0.075 ± 0.031 for the weak and strong bias conditions, respectively) and from zero to strong (bootstrap, P = 0.011; slope = 0.027 ± 0.024 for the zero bias condition). The increase from the zero to weak condition was not significant (bootstrap, P = 0.24). All were different from 0 (bootstrap, zero, P = 0.016; weak and strong, P < 0.001) and from a proportional response (bootstrap, zero and weak, P < 0.001; strong, P = 0.001), in which the sensitivity to a force with gain −36 Ns/m would be 6 times higher than the sensitivity to a force with gain −6 Ns/m. Although true proportionality was not achieved by changes in bias alone, changes in the slope of the sensitivity vector make clear that the adaptive response, to the same forces and within the same group of participants, can change depending on the properties of the movement environment.

We also quantified a second slope metric for the zero-bias condition, calculated from a linear fit of the sensitivity elements with positive gain (s+ = 0.042 ± 0.017, B > 0; Fig. 5). Like the first slope metric (s, B < 0), this second metric was significantly less than the slope of the strong bias condition (bootstrap, P = 0.045) and was not significantly different from the slope of the weak bias condition (bootstrap, P = 0.31). Both slope metrics (s and s+) were significantly less than the slope of a line connecting B = −12 and B = 12 (bootstrap, P = 0.016 and 0.007 for s+ and s, respectively).

The zero-bias condition was further quantified using an asymmetry index (AI). Although the viscous force was equally balanced to the left and right (Fig. 1A), the AI was greater than 0 (bootstrap, AI = 3.409 Ns/m, P < 0.001), indicating a rightward sensitivity bias, the direction opposite the bias of the force distribution; adaptation after rightward forces was greater than adaptation after leftward forces.

The model accounted for 99.1, 98.7, and 98.3% of the variance for the zero, weak, and strong bias conditions, respectively, with low residual error (RE = 1.58, 1.39, and 1.42 mm for the zero, weak, and strong bias conditions, respectively). Model fits are shown in Fig. 6, AC, for the first 100 movements.

We also fit the data with a model that explicitly forced proportionality (for details, see methods, Eq. 3). The residual error increased 161% for the zero bias condition (RE = 4.13 mm) and 22% for the weak bias condition (RE = 1.70 mm). A model that assumed proportionality fit the strong bias condition almost as well as a model that did not assume proportionality (RE = 1.48 mm). These results are consistent with a strategy that became increasing proportional as the bias increased from zero to strong.

Finally, we asked: what percentage of the gain estimate, n+1 (see methods, Eq. 2), was driven by the sensitivity vector versus the constant scaling factor A. We broke the estimate down into its components, AB̂n and S⃗, and calculated the percentage of n+1 driven by the sensitivity vector S⃗. On average, the vector contributed 21, 31, and 20% of the estimate for the zero, weak, and strong bias conditions, respectively. We conclude that the sensitivity vector significantly added to the overall estimate. Zeroing the sensitivity vector (setting S⃗ = 0) increased the RE 2–7 times, to 4.77, 2.51, and 7.59 mm, for the zero, weak, and strong bias conditions, respectively.

Rightward bias

To explore the asymmetry of the sensitivity vector, we tested a second group of six participants with forces biased in the opposite direction (Table 1, group 2); forces were either unbiased or were weakly or strong biased to the right. Forces were experienced in 80% of trials.

The slope, calculated from a linear fit of the sensitivity elements with positive gain, again changed with bias condition (Fig. 7; for additional coefficient estimates, see appendix). The slope increased significantly as the bias increased from zero to strong (bootstrap, P < 0.001; slope = 0.0065 ± 0.024 and 0.075 ± 0.025 for zero and strong bias conditions, respectively) and as the bias increased from weak to strong (bootstrap, P = 0.006; slope = 0.030 ± 0.017 for the weak bias condition). The slope also increased as the bias increased from zero to weak (bootstrap, P = 0.024). For the zero-bias condition, we again quantified a second slope metric, calculated from a linear fit of the sensitivity elements with negative gain (s = 0.025 ± 0.030, B < 0). Like the first slope metric (s+), this second metric was significantly less than the slope of the strong bias condition (bootstrap, P = 0.003). The slope was also less than the slope of the weak bias condition, but the change was not significant (bootstrap, P = 0.40). Both metrics (s and s+) were significantly less than the slope of a line connecting B = −12 and B = 12 (bootstrap, P = 0.006 and P < 0.001 for s+ and s, respectively), and neither metric was statistical different from zero (bootstrap, P = 0.32 and 0.072 for s+ and s, respectively).

The AI was significantly negative (AI = −2.76 Ns/m, P < 0.001), indicating a strong leftward bias. We conclude that participants increased their sensitivity to rightward (AI > 0) and leftward forces (AI < 0) when those forces were biased to the left (B < 0) and right (B > 0), respectively, i.e., participants were more sensitive to forces in the direction opposite the directional bias of the force distribution.

Overall, when the force distribution was unbiased, we concluded that participants were sensitive to the direction of the force, but within a direction were largely insensitive to changes in force strength. The slope increased as the force distribution became more directionally biased. Considering the leftward- and rightward-biased force distributions together, we conclude that participants transitioned from a categorical to a proportional strategy as the bias transitioned from zero to strong.

Across-day effects

For each zero-bias condition (groups 1 and 2), participants were more sensitive to forces in the direction opposite the directional bias of the force distribution; the sensitivity vector was biased rightward when the force distribution was biased to the left and leftward when the force distribution was biased to the right. Here we ask if the asymmetry was a function of the order in which environments were presented. We hypothesized that the asymmetry would decrease if the zero-bias condition was presented on the first day of training, when participants had no expectation of directional bias, and would increase if the zero-bias condition was experienced later in training, after an expectation of force direction had been established.

For the leftward, zero-bias condition, the AI increased (rightward asymmetry increased), as the zero bias condition was experienced later in training; however, the trend was not significant (bootstrap, P = 0.36). The results were similar for the rightward, zero-bias condition. The AI decreased (leftward asymmetry increased) as the zero-bias condition was experienced later in training; again, the change was not significant (bootstrap, P = 0.31). We conclude that the order in which the zero-bias condition was presented did not significantly affect the asymmetry of the sensitivity vector.

Changing the likelihood of forces

Last, we studied whether changing the likelihood of forces would alter the adaptive response. We asked a third group of 12 participants to make reaching movements (10 cm, 0.5 s) while holding a robotic arm that generated viscous forces. We changed the likelihood of forces across training days (Table 1, group 3); forces were presented in 20, 50, or 80% of trials. The strength of the force was drawn from a strongly biased distribution, such that the magnitude, but not direction, of force varied from trial to trial.

We found that participants moved marginally faster when forces were presented at a higher likelihood; because the strength of the perturbation was velocity-dependent, the overall strength of the force perturbation changed across days of training. We quantified the strength of a force using the peak of its force profile; on average, the strength increased (2-dimensional ANOVA with factors gain and likelihood, P < 0.002) 2.8% from the 20–50% likelihood condition and 3.9% from the 50–80% likelihood condition.

We again quantified the feedback response using a linear parameterization. The response was constant, regardless of the gain of the perturbation (ANOVA, P = 0.99, 0.99, and 0.94 for the 20, 50, and 80% likelihood conditions, respectively). The average response, as quantified by k, equaled 179 ± 35, 190 ± 36, and 200 ± 27 kg/s2 for the 20, 50, and 80% likelihood conditions, respectively, where the ± interval equals the 95% CI of the mean. In addition, the slope of a linear fit (k vs. B) was not significantly different from 0 for any likelihood condition (t-test, P = 0. 91, 0. 35, and 0.94 for the 20, 50, and 80% likelihood conditions, respectively) and was constant across conditions (ANOVA, P = 0.607).

We also quantified the response by calculating the maximum p.d. and integrated error. The slope of a linear fit (maximum p.d. vs. gain, integrated error vs. gain) was consistent across all likelihood conditions (ANOVA, maximum p.d., P = 0.53; integrated error, P = 0.63). We conclude that the feedback available to each participant was similar across all likelihood conditions.

We quantified adaptation, across single movements, using our novel state-space analysis (Eq. 2, see methods), and plotted each element of the sensitivity vector against its corresponding viscous gain for each likelihood condition (Fig. 8; for additional coefficient estimates, see appendix). We found that participants increased their sensitivity, to the same forces, when those forces were presented with a higher likelihood. We quantified the relationship between sensitivity elements by performing a linear fit of the six elements corresponding to nonzero gain. The slope of the fit increased significantly from the 20 to 80% condition (bootstrap, P = 0.005; slope = 0.032 ± 0.022 and 0.087 ± 0.041 for the 20 and 80% likelihood conditions, respectively), and from the 50 to 80% condition (bootstrap, P = 0.016; slope = 0.048 ± 0.020 for the 50% likelihood condition), and increased, although not significantly, from the 20 to 50% condition (bootstrap, P = 0.12). All were different from zero (bootstrap, P < 0.001) and from a proportional response (bootstrap, 20 and 50%, P < 0.001; 80%, P = 0.002).

Our state-space model assumed the strength of the perturbation was fixed. Viscosities remained constant throughout training, but because the speed of movement increased ∼7% from low to high likelihood, the magnitude of forces increased by the same amount. However, changes in the slope of the sensitivity function were over an order of magnitude larger. Slopes more than doubled from the 20 to 80% likelihood condition. Therefore we do not believe that changes in force strength can account for the changes in adaptive strategy.

Model performance was evaluated using VAF (Eq. 3) and the root-mean-square of the RE. The model accounted for greater than 98% of the variance with low residual error (VAF = 98.4, 98.9, and 98.3% and RE = 1.01, 1.01, and 1.24 mm for the 20, 50, and 80% likelihood conditions, respectively).

We again fit the data with a model that forced proportionality. The residual error increased 51% for the 20% likelihood condition (RE = 1.53 mm), 39% for the 50% likelihood condition (RE = 1.40 mm), and 19% for the 80% likelihood condition (RE = 1.48 mm). The sensitivity vector contributed 23, 25, and 31% of the overall estimate of gain, n+1, for the 20, 50, and 80% likelihood conditions, respectively. Finally, setting each element of the sensitivity vector to 0 increased the RE 4–8 times, to 4.05, 6.05, and 8.24 mm, for the 20, 50, and 80% likelihood conditions, respectively.

Sensitivity to positional error

Participants modified their adaptive strategy to move in each novel environment (Figs. 5, 7, and 8). Other studies have shown that adaptation depends on both the strength of previous forces and previous kinematic errors (Scheidt et al. 2001). Therefore a second approach would be to quantify the sensitivity to p.d., because the same viscosity can generate different trajectories in different environments (Fig. 2; compare B = −36 Ns/m in each environment). We therefore performed a parallel analysis in which the slope metric depended on the positional error instead of the viscous gain. We found that these slopes also changed as a function of environmental bias or likelihood.

For each leftward-bias condition, slope was recalculated from a linear fit of the sensitivity elements indexing viscous gains less than 0, plotted against mid-movement p.d. Unlike our previous analyses, each sensitivity element mapped to a different x-datum. In each environment, participants' shifted their expectation of force toward the mean of the force distribution, which led to a shift in the left-right position of each fit (Fig. 9; for comparison, see Fig. 5).

Despite this movement along the x-axis, the slope increased significantly as the bias condition changed from zero to strong (bootstrap, P = 0.022; slope = 0.37 ± 0.35 and 1.02 ± 0.41 for the zero and strong bias conditions, respectively, where the ± interval equals the 95% CI of the mean) and from weak to strong (bootstrap, P = 0.010; slope = 0.50 ± 0.25 for the weak bias condition). The increase from the zero to weak condition was not significant (bootstrap, P = 0.25).

We repeated the above analysis for each likelihood condition, with similar results (Fig. 10). The slope increased significantly as the likelihood condition changed from 20 to 80% (bootstrap, P = 0.005; slope = 0.39 ± 0.28 and 1.17 ± 0.57 for the 20 and 80% likelihood conditions, respectively, where the ± interval equals the 95% CI of the mean) and from 50 to 80% (bootstrap, P = 0.014; slope = 0.63 ± 0.28 for the 50% likelihood condition). The increase from the 20 to 50% likelihood condition was not significant (bootstrap, P = 0.070).

FIG. 10.

Elements of the sensitivity vector plotted against average mid-movement p.d., for each likelihood condition. The slope of a linear fit (dotted lines) changed significantly as the environment transitioned from the 20 to 80% (bootstrap, P = 0.005) and 50 to 80% (bootstrap, P = 0.014) likelihood conditions. The error bars represent the 95% CI of the mean (see bar plot).

We conclude that trial-by-trial processing of both viscous force and positional error into incremental adaptation changed with the statistics of the environment.

DISCUSSION

Motor theorists, psychophysicists, and neurophysiologists have several notions of how error induces learning. Theories have hypothesized that kinematic error (the position of the arm) (Jordan and Rumelhart 1992), stiff and viscous feedback (Wolpert and Ghahramani 2004), and/or motor error (Kawato et al. 1987) could all serve as signals to drive adaptation across single movements. Each of these theories assumes that adaptation scales proportionally with the size of the error, similar to the Marr-Albus model of the cerebellum (Kawato 2002; Marr 1969). The Marr-Albus model posits that the synaptic efficacy of Purkinje cells changes proportionally with an error signal carried by climbing fibers and encoded as a difference in complex firing rates from baseline. Competing hypotheses, such as reinforcement learning, do not posit a training signal but instead formulate a critical signal that will drive performance to an optimum without explicit knowledge of error. Even this unsupervised hypothesis, however, presumes changes in the critical signal scale proportionally with the size of the error (Barto 2003; Gullapalli 1990). All of these hypotheses, therefore, need the adaptive response to scale proportionally with error.

Current state-space models presume proportionality and have successfully mimicked human behavior (Donchin et al. 2003; Smith et al. 2006; Thoroughman and Shadmehr 2000; Thoroughman and Taylor 2005). One study explicitly showed proportionality when forces were strongly biased and presented in every trial (Scheidt et al. 2001). These studies used forces that, within a direction, were strongly biased and presented with high likelihood. Catch trials, trials in which the force was unexpectedly removed, never exceeded 17% of movements. Therefore participants likely adopted a proportional, or near proportional, strategy; we found that participants approached proportionality if the forces were strongly biased and presented in 80% of movements. The goodness of their state-space fits supports such a strategy.

Here we report a very different result. We used a novel state-space analysis to show that participants modified their adaptive strategy to move in a variety of viscous environments. Participants became less sensitive, to the same forces, as the overall likelihood of forces decreased; the slope of a linear fit of sensitivity elements decreased. In addition, changing the directional bias of the force distribution from unbiased to strongly biased induced participants to transition between categorical and proportional strategies, even though the likelihood of force presentation was high (80%). For the zero-bias condition, participants were largely insensitive to the magnitude of the perturbation within a direction.

We recently reported that the response to 70-ms pulses of force, each gated by position and presented infrequently with zero directional bias, was also categorical in nature (Fine and Thoroughman 2006). In this study, participants applied the same categorical strategy to whole movement, viscous forces, implying that the directional bias of the forces, and not the force duration or state dependency (time vs. velocity), drove the categorization of learning.

Consideration of feedback control

We suggest that changes in the slope of the sensitivity function reflect changes in the processing of sensory signals into subsequent predictive control. One could instead consider the possibility that the statistics of the environment are affecting the way in which participants are sensing the error itself. Two observations argue against that possibility. The first is the qualitative success of participants to reach the target regardless of the strength of the perturbation or the environment (Fig. 2). This feedback control requires the coordination of both short and long loop feedback responses across spinal and supraspinal processing (Allum 1975; Marsden et al. 1972) and suggests that the sensory representation of movement through vision (Ghez et al. 1995) and proprioception (Gordon et al. 1995) remains intact. The second observation was a quantitative identification of the feedback response driven by perpendicular positions and velocities (Fig. 3, for leftward bias). We found that the within-movement feedback response was equally (and proportionally) driven by the kinematics of movement across all perturbation strengths. Although our fits did not require the addition of a long-loop reflex, our perturbation was not designed nor well suited to differentiate between the short- and long-loop contributions of feedback.

Our integrated view of mid-movement processing is that all components of mid-movement feedback control seem intact and unaffected by the directional bias or likelihood of the perturbation. We believe therefore that the observed changes in behavior occur after the reception of an intact feedback signal.

Neural correlates of adaptation

Recently, studies of motor learning have used system identification, coupled with adapting basis function networks, to quantify adaptation across single movements and the generalization of that adaptation to neighboring movement directions (Donchin et al. 2003; Hwang et al. 2003; Thoroughman and Shadmehr 2000; Thoroughman and Taylor 2005). Thoroughman and Shadmehr (2000) suggested that modifications to interneuronal connections are proportional to motor error and gated by the velocity-tuned activity of individual presynaptic neurons. Other studies have indicated that this model explains learning in a variety of viscous environments (Donchin et al. 2003; Thoroughman and Taylor 2005), as well as position-dependent forces (Hwang et al. 2003). Prominent velocity tuning in motor cortices (Moran and Schwartz 1999) and the cerebellum (Coltz et al. 1999) lend neurophysiological credence to the model.

Networks of neurons have also been used in decision theory to categorize sensory stimuli (Gold and Shadlen 2001; Kristan and Shaw 1997). Gold and Shadlen (2001) proposed that overlapping populations of broadly tuned neurons can categorize by comparing the spike rates of appropriately chosen neurons; the difference in spike rates is proportional to the logarithm of the likelihood ratio, which quantifies the probability a sensory stimuli falls within a predefined category. We theorize that overlapping networks could account for the categorization of forces into leftward and rightward bins. A “critic” could gate the output of several networks, modulating the adaptive strategy depending on the dynamics of the environment. This strategic modulation would allow both categorical and proportional strategies, as well as “in-between” strategies, to be controlled by the same process. The cerebellum is a likely candidate for such a critic; its unique structure is well suited for rapidly switching between networks by selectively gating the neurons appropriate for each strategy (Schweighofer et al. 1998a,b).

Recently, a study of saccadic adaptation in nonhuman primates has shown that Purkinje cells in the vermis of the oculomotor cerebellum responded categorically to error (Soetedjo and Fuchs 2006). Complex spike activity responded to the direction but not magnitude of eye position errors; this activity may drive adaptation. We are not suggesting that the oculomotor cerebellum encodes error during reaching, but that the architecture may preexist for categorization in the cerebellum.

Theoretical and experimental implications

Brain theories have to date assumed that neuronal tuning provides a computational framework for estimation during sensorimotor adaptation (Pouget and Snyder 2000) and that the potentiation or depression of synaptic efficacy underlying adaptation is proportional to error (Poggio and Bizzi 2004; Pouget and Snyder 2000); these assumptions both simplify and expedite training. Recent experiments, however, suggest that both these features of learning are surprisingly and rapidly flexible. While adapting to forces with high spatial complexity, Thoroughman and Taylor (2005) showed that participants change the way that they generalize sensed error to neighboring directions. They suggest a change in neural tuning might underlie this process. Another study (Krakauer et al. 2006) showed that error generalization can depend on previous contextual cues, which challenges the idea of an invariant learning rule. Here we show that the adaptive strategy is also a function of the statistics of the movement environment. Participants changed their trial-by-trial response depending on the likelihood or direction of the perturbation. Some of these changes in behavior could be driven by conscious and/or cognitive processes (Taylor and Thoroughman 2007). Psychophysical measures of sensory perception in each environment could identify possible cognitive contributions.

Our approach identified the flexibility of the sensorimotor transformation; neuronal recordings could reveal if this flexibility arises from changes in sensory, associative, prefrontal, or motor activity. This flexibility could also provide a powerful metric to determine whether candidate neuronal activity encodes adaptation by switching proportionality in concert with induced changes in behavior. If neurons encode feedback, their activity would remain constant, even as the statistics of the environment change; if however, neurons encode error-induced adaptation, we predict that their responses would change with the statistics of the environment. Discovery of these neuronal correlates of learning would enable a search for areas that observe, critique, and change the internal processes that underlie adaptation.

APPENDIX

When quantifying feedback, we optimized for a stiffness, k, which transformed errors in position and velocity into a within-movement feedback response. The coefficient k was statistically identical across all bias and likelihood conditions. In addition, neither the mass of the arm, m, nor the coefficient, η, which linked stiff and viscous terms, changed with the directional bias of the force distribution, or the likelihood of the perturbation. Coefficient estimates are given in Table 2.

View this table:
TABLE 2.

Coefficient estimates for the feedback simulation

The coefficient γ transformed the forward movement velocity into an estimation of perpendicular force x; because was roughly constant across both bias and likelihood conditions, γ scaled with the human expectation of perturbation strength. For the zero bias condition, γ was not significantly different from 0 (t-test, P = 0.17). For the weak and strong bias conditions, the magnitude of γ was significantly larger, indicating that participants built an internal expectation of force over the course of training.

The magnitude of γ also increased with the likelihood of the perturbation. When the likelihood of the perturbation was 20%, participants' expectation of force was not significantly different from 0 (t-test, P = 0.31). Participants built significant expectations when the likelihood was 50 and 80%.

We quantified adaptation using a novel state-space analysis (Eq. 2) and found that the slope of the sensitivity vector, S⃗, changed with the statistics of the environment. Additional coefficient estimates for each group are given in Table 3. The coefficient D estimates arm compliance (the inverse of stiffness), and transforms within-movement errors in gain estimation (B − ) into kinematic displacements (y). D was relatively constant within a group; when D changed significantly (within a group), the changes were always <10%.

View this table:
TABLE 3.

Coefficient estimates for each movement environment

The coefficient A determined the percentage of the gain estimate retained from trial to trial. When A changed within a group, it changed no more than 11%. In contrast, changes in the slope of the sensitivity function were an order of magnitude larger.

GRANTS

This work was supported by the Whitaker Foundation and National Institute of Child Health and Human Development Grant HD-055851.

Acknowledgments

We thank E. D. Herzog, M. A. Smith, K. J. Feller, J. A. Semrau, J. A. Taylor, D. N. Tomov, P. A. Wanda, and K. J. Wisneski for helpful comments and D. N. Tomov for technical management.

Footnotes

  • The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

REFERENCES

View Abstract