Extensive theoretical, psychophysical, and neurobiological work has focused on the mechanisms by which short-term learning develops into long-term memory. Better understanding of these mechanisms may lead to the ability to improve the efficiency of training procedures. A key phenomenon in the formation of long-term memory is the effect of over learning on retention—discovered by Ebbinghaus in 1885: when the initial training period in a task is prolonged even beyond what is necessary for good immediate recall, long-term retention improves. Although this over learning effect has received considerable attention as a phenomenon in psychology research, the mechanisms governing this process are not well understood, and the ability to predict the benefit conveyed by varying degrees of over learning does not yet exist. Here we studied the relationship between the duration of an initial training period and the amount of retention 24 h later for the adaptation of human reaching arm movements to a novel force environment. We show that in this motor adaptation task, the amount of long-term retention is predicted not by the overall performance level achieved during the training period but rather by the level of a specific component process in a multi-rate model of short-term memory formation. These findings indicate that while multiple learning processes determine the ability to learn a motor adaptation, only one provides a gateway to long-term memory formation. Understanding the dynamics of this key learning process may allow for the rational design of training and rehabilitation paradigms that maximize the long-term benefit of each session.
The relationship between the properties of a training session and the strength and duration of the resultant memory trace has been a focus of psychological and neurobiological research since the late 1800s (Ebbinghaus 1913). Exposure to a few training stimuli can lead to long-lasting memories in organisms with even the simplest of nervous systems such as nematodes (Rankin et al. 1990), Drosophila (DeZazzo and Tully 1995), and mollusks (Carew et al. 1972). Nevertheless a clear and comprehensive understanding of the processes underlying long-term memory formation has not yet been achieved. Better understanding of these processes may lead to a better understanding of disorders of learning and memory and to the rational design of optimally efficient training paradigms and improved rehabilitation procedures.
A key finding from Ebbinghaus's pioneering human work (Ebbinghaus 1913) that has received considerable subsequent attention in the psychology literature has been termed the over learning effect (Krueger 1929; Luh 1922). This effect characterizes a fundamental property in the relationship between the duration of a training session and the level of subsequent retention: after reaching a high level of performance during an initial training period, additional training that has little effect on performance can lead to substantial improvements in long-term retention. For example, if reaching a high level of performance (typically set as 1 error-free trial, although this does not imply that the subject achieved perfect performance in subsequent trials) required 10 trials, 50% over learning requires performing an additional 5 trials. Initial studies focused on verbal recall (memorizing lists of syllables or nouns) (Krueger 1929; Luh 1922) on which most of the work to date has been done (for a review, Driskell et al. 1992). However, the over learning effect has been demonstrated on a wide variety of learning paradigms including motor skill learning (e.g., disassembling and assembling a device) (Melnick 1971; Melnick et al. 1972; Schendel and Hagman 1982, 1991). Tests for retention given after a set time interval posttraining (for example, 24 h) typically revealed that the greater the degree of over learning, the greater the retention of the task (Craig et al. 1972; Driskell et al. 1992; Postman 1962; Rohrer et al. 2005). Despite the abundance of research, previous models of over learning (Craig et al. 1972; Schendel and Hagman 1991) predict neither the rate nor the maximal amount of improvement in retention conferred by increased training.
The study of various motor adaptations has been a useful model for understanding the mechanisms underlying motor memory formation. A motor adaptation is a change in motor output on a task made after repeated exposure to an environmental perturbation. Examples include saccade adaptation (McLaughlin 1967), visual rotation (Krakauer and Shadmehr 2006; Krakauer et al. 2000), and dynamic force-field adaptation (Shadmehr and Mussa-Ivaldi 1994; Smith et al. 2006; Thoroughman and Shadmehr 2000). In the last several years, computational modeling methods have been applied to help produce an increasingly quantitative understanding of the progression (Scheidt et al. 2001; Smith et al. 2006; Thoroughman and Shadmehr 2000) and internal representation (Bhushan and Shadmehr 1999; Donchin et al. 2003; Krakauer et al. 2000; Thoroughman and Shadmehr 2000) of the motor adaptation process. Although it has been shown that motor adaptation training can lead to the formation of enduring motor memories (Alahyane and Pelisson 2005; Brashers-Krug et al. 1996; Krakauer et al. 1999) and that the passage of time is important in consolidating these memories (Krakauer and Shadmehr 2006; Overduin et al. 2006) surprisingly little work has explored how the characteristics of an initial training period influence the ability to retain newly learned motor adaptations. One such study (Yin and Kitazawa 2001) demonstrated a clear over learning effect for prism adaptation in monkeys: training for 50 or 250 trials produced substantial initial learning but no clear 24-h retention, whereas 500 initial learning trials produced clear retention despite similar initial learning performance.
We recently showed that interactions between two distinct processes underlie the short-term motor adaptation that subjects experience during a single training session (Smith et al. 2006) and that interactions between these two processes explain several phenomena in motor adaptation that were previously considered to be unrelated such as savings, interference, and spontaneous recovery. In this model, one learning process rapidly responds to error but has poor retention, whereas another process responds slowly to error but retains information well from one trial to the next. Can the ability to parse learning into these two components allow us to better understand long-term memory formation? Here we studied how the passage of time interacts with each of these learning processes and whether these two processes differentially contribute to the formation of long-term motor memories. We hypothesize that the slower learning process contributes to long-term retention more strongly than the faster process, resulting in not only greater long-term retention for subjects trained for longer periods, but disproportionately so with respect to the amount of adaptation achieved. That is, as adaptation progresses improvements in the amount of retention outpace improvements in the amount of learning.
Forty-eight healthy right-handed participants (mean age: 27 yr) without known neurological impairment were recruited. All participants gave informed consent and the experimental protocols were approved by the Harvard University Committee on Human Subjects Research.
The experimental paradigm used a dynamic force-field adaptation task (Scheidt et al. 2000; Shadmehr and Mussa-Ivaldi 1994; Smith et al. 2006) in which each subject was trained to move his or her dominant hand to targets in the horizontal plane while grasping a robot manipulandum (Fig. 1A). The manipulandum measured hand position, velocity, and force, and its motors were used to dynamically apply force patterns to the hand, all of which were updated at a sampling rate of 200 Hz. The position of the hand was displayed as a small round cursor (3 mm) on a vertically oriented computer monitor in front of the participant (refresh rate of 75 Hz). Participants reached to circular targets, 1 cm diam, that were spaced 10 cm from each starting position. We instructed participants to “make quick movements to the targets.” In addition, subjects were told that the reaction time was not important—they could wait as long as they wished after target appearance before starting each movement—but when ready, they were to move in a rapid motion toward each target. The endpoint of each movement was used as the starting point for the subsequent movement, and midline movements were made in two target directions—toward and away from the chest of each participant. Data from both movement directions were analyzed in this study.
Three trial types were employed in this study: null, force-field, and error-clamp trials. Null trials were used for initial practice. In these trials the robot motors were disabled so that they did not produce force. During force-field trials, the motors were used to produce a force pattern on the hand that could perturb hand motion. The forces were proportional in magnitude and perpendicular in direction to the velocity of hand motion (Fig. 1A). The relationship between the force (f) and velocity (ẋ) vectors was determined by the 2 × 2 matrix B via the relationship f = Bẋ as shown explicitly below
During error-clamp trials, the robot motors were used to constrain movements in a straight line toward the target by effectively counteracting any motion perpendicular to the target direction (Scheidt et al. 2000; Smith et al. 2006). This was achieved by applying a stiff one-dimensional spring (6 kN/m) and damper [150 N/(m/s)] in the axis perpendicular to the target direction. In these trials, lateral errors (i.e., errors aligned with the direction of the force-field perturbation) were kept very small so that the lateral force patterns produced by subjects due to training-induced predictive compensation of the force-field could be measured independently of feedback responses driven by lateral errors. Although we could not achieve a perfect error clamp, perpendicular displacement from a straight line to the target was held to <0.6 mm and averaged ∼0.2 mm in magnitude.
Except for differences in the duration of the initial training period and the time interval for retention testing, all participants followed the same basic experimental protocol (Fig. 1B). There was a baseline block of trials, followed immediately by a variable-length training block which was in turn followed by a retention block either 5 min or 24 h later. All groups performed the same baseline block consisting of 150 trials in each of the two target directions. The first 50 of these trials in each direction were null trials (robot motors turned off), and in the last 100 of these baseline trials, error-clamp measurement trials were randomly applied ∼20% of time (18 error-clamp and 82 null trials in the forward direction; 19 error-clamp and 81 null trials in the backward direction) to provide baseline estimates of the lateral force forces. The force patterns associated with the error-clamp trials in this block were used as a baseline for estimating learning-related changes in force-patterns during the subsequent task epochs. The first group of participants (n = 10), designated 160L24 (160 trial learning session − 24 h retest), performed 160 training trials in each target direction in a clockwise curl force field. Forty error-clamp trials were randomly interspersed among these training trials (20% of the 200 total trials) to provide estimates of the learned feedforward changes in lateral force profiles. The second subject group (n = 10), 160L0, performed the same experiment as the 160L24 group without the 24 h retest delay. This group was tested for retention after a 5-min delay. The last three subject groups, 11L24 (n = 8), 30L24 (n = 10), and 103L24 (n = 10) were identical to the 160L24 group (24-h retest delay) except that the learning phase was stopped after 11, 30, and 103 force-field trails rather than 160. Four, 7, and 25 error-clamp trials were interspersed among the force-field trials during the training period for these groups, respectively, corresponding to a train period durations of 15, 37, and 128 total trials. Retention of learning was determined using a series of consecutive error-clamp trials. The first two of these trials were used for the analysis presented here, but very similar results were obtained using the first one to five trials.
Analysis of force profiles
Because the force-field perturbations applied during this experiment consisted of force patterns perpendicular to the direction of motion, we focused our analysis on the lateral (perpendicular) force profiles that participants generated during movement. In general, lateral force could reflect an adaptive compensation of expected lateral force or an on-line corrective response to errors detected during the course of movement. Specifically, we looked at the progression of lateral force profiles during error-clamp trials in the null, initial learning, and error-clamp blocks of the experiment. During these trials, lateral errors were kept small (<0.6 mm), so lateral force profiles essentially reflected adaptive compensation of the force-field perturbations. Because full compensation of the force-field perturbation on a particular trial (i.e., the ideal force profile) required a lateral force profile proportional to the velocity profile on that same trial (and this velocity profile varied from 1 trial to another), we assessed the amount of adaptation on each error-clamp trial by computing a force-field compensation factor found by linear regression of the ideal lateral force profile on each error-clamp trial (the force pattern required for full force-field compensation on that trial) onto the baseline-subtracted lateral force profile that was actually produced. This entails finding the regression coefficients (K0 and K1) that would minimize (in a least-squares sense) the error [ε(t)] in the fit
The K1 regression coefficient represents the sensitivity of the actual human-produced force pattern to the ideal compensatory force pattern. This quantity characterizes the overall amount of force-field compensation in a given trial (Smith et al. 2006) and serves as our learning metric which we refer to as the adaptation coefficient. If the applied force and the desired force perfectly coincide, this adaptation coefficient is 1, if they are directly opposed, it is −1, and if they are unrelated it will be zero. This regression procedure is illustrated in the first two panels of Fig. 2. Figure 2A shows FACTUAL(t) and FIDEAL(t) plotted against time for several selected error-clamp trials, whereas B shows these quantities plotted against one another along with the best-fit straight line having a slope of K1 and an offset of K0.
We used the learning rules for the multi-rate learning model presented in Smith et al. (2006) along with the error equations shown in the following text to iteratively compute the time course of adaptation for the simulated experiment.
Perturbation (force-field) trials
Error clamp trials
e(n)—error on trial n
f(n)—strength of force-field disturbance on trial n
x(n)—state of learned motor output on trial n
Multi-rate learning model (Smith et al. 2006)
xf(n), xs(n)—Internal states that represent the contributions of the fast and slow learning processes to net motor output [x(n)].
Bf, Bs—learning rates for the fast and slow learning processes.
Af, As—retention factors for the fast and slow learning processes.
Note that since only a single force-field magnitude was used for all simulations, we normalized by the perturbation magnitude, i.e., the presence of a force-field on trial n was represented by f(n) = 1. Therefore the values of xf(n) and xs(n) that we represent are intrinsically normalized so that they represent the fraction of the force-field environment that has been learned. For the simulations of multi-rate model presented in this paper, the model parameters were derived from a least-squares fit to the initial learning data shown in Fig. 2C. These parameters were: Af = 0.85, As = 0.998, Bf = 0.11, and Bs = 0.021. These parameters differed from those used in Smith et al. (2006); however, we found that these differences were not meaningful for the current study. A detailed comparison of the different model parameter estimates and their consequences is given in the supplementary materials.1
The mulit-rate model was used to predict how the 24-h retention level should vary with the duration of the initial training period if the slow learning process, the fast learning process, or both directly contributed to 24-h retention. Furthermore, the model predictions shown in Figs. 3 and 4C are not fits to the 24-h retention data. Instead a single scaling factor representing the maximum retention level was used to scale each of the model-predicted slow process, fast process and net adaptation levels to match the experimental data for only the 160L24 group. This enabled the comparison between the retention patterns in the experimental data and the model predictions for the other subject groups, i.e., to what extent was the 24-h retention data proportional to the constituents of the multi-rate learning model. Note that the solid curves in Fig. 3 represent the unscaled model outputs, and the dotted curves represent the scaled predictions.
We assessed the relationship between initial learning and long-term retention on a force-field adaptation task (Scheidt et al. 2000; Shadmehr and Mussa-Ivaldi 1994; Smith et al. 2006) (see methods and Fig. 1 for a description) in which five groups of subjects were tested for retention after various amounts of training. Four of the groups (11L24, 30L24, 103L24, and 160L24, where 11L24 signifies 11 initial learning trials with a retention test at 24 h, etc) were tested for retention 24 h after initial training, and one group (160L0 signifying 160 initial learning trials with a retention test at 0 h 5 min) was tested 5 min after initial training as a control. Figure 2C shows the time course of the initial adaptation during the null-field baseline trials and initial learning period for all five subject groups. The adaptation coefficient (the slope of the linear regression of the actual human-produced lateral force pattern onto the pattern that would be ideal for counteracting the force-field environment, see methods and A and B of Fig. 2) is plotted along the ordinate and trial number on the abscissa. As shown in the figure, all of the groups effectively learned the force-field, and with increased training (more trials), adaptation was further increased. The data show a rapid phase of adaptation during the first 10–15 trials and then a slower, more prolonged improvement. The three groups with the longest training periods (103L24, 160L24, and 160L0-blue, cyan, and green traces) reached the highest adaptation coefficients-∼80% of the ideal force-field compensation as shown in Fig. 2C.
This result is further demonstrated in Fig. 2D, which shows the average lateral force patterns measured during error-clamp trials at the end (last 10%) of the learning period. All five groups produced lateral force profiles that approached the desired force profile (which is the opposite of the robot-applied force profile) by the end of the learning block; however, groups 160L0, 160L24, and 103L24 displayed nearly identical force compensation profiles that are all slightly greater than those displayed by the 30L24 and 11L24 groups.
The average force profiles during the retention testing (5-min or 24-h break) are displayed in Fig. 2E. The 160L0 group (green trace) displays only a small decrease from the corresponding late-learning force profile when tested for retention 5 min after the learning period. However, the retention force profiles obtained 24 h after testing (160L24 group, cyan trace) differ markedly from those produced at the end of the learning period (160L0 group, green trace). All groups demonstrate a decrease in the force profile with the largest difference between learning and retention occurring for the group with the least amount of training, the 11L24 group. The force profiles displayed in Fig. 2, D and E, show the overlearning effect: with increased training in the motor adaptation task there is increased retention.
The relationship between learning and retention for each group is shown more explicitly in Fig. 3 in which we quantified the level of 24-h retention for each group. Our hypothesis was that if 24-h retention depended chiefly on the level of the slow learning state in our multi-rate learning, one would expect that the 160L24 group, which through the longer learning period allowed the slow learning state to reach a higher level, would show greater retention than the 103L24, 30L24, and 11L24 groups. This prediction appears to hold in our data (Fig. 3A). This panel displays the adaptation coefficients for the 24-h retention data alongside simulations of our multi-rate learning model of short-term motor adaptation. In this model, a fast (orange trace) and slow (blue trace) learning process combine to produce the overall motor adaptation (black trace). We scaled the raw model output (values of slow state, fast state, and net learning at 160 trials of training) to the retention level obtained after 24 h for 160 trials of training (160L24 group), resulting in three different predictions for the pattern of retention following varying degrees of overlearning shown in A–C. The blue dashed line in Fig. 3A shows the retention pattern predicted by our multi-rate rate if the slow learning process serves as the gateway to long-term retention. The average levels of retention exhibited by the 103L24, 30L24, and 11L24 groups are well accounted for by this prediction (R2 = 0.99, P < 10−4). This was not the case for predictions made by a model where the fast process (orange dashed line in Fig. 3B) or the total initial adaptation level (black dashed line in Fig. 3C) lead directly to long-retention. This suggests that long-term retention is not strongly related to these parameters.
To examine this hypothesis in greater detail, we plotted the predicted level of the slow learning process at the end of the initial training period against the amount of retention obtained by each group in our study (Fig. 3D). This plot shows that the groups tested for retention following a 24-h break display remarkably similar amounts of retention with respect to the amount of slow system learning achieved in the initial training period, i.e., all groups retain ∼65% of the initial slow process learning 24 h later. In contrast, neither the fast process alone (Fig. 3E) nor the overall learning level (F) show a constant percentage of retention. The retention proportion ranges widely from 20 to 50% of the overall learning level and from 35 to 400% of the fast process level but narrowly from 60 to 70% of the slow process level.
To quantify the relationship between 24-hour retention and levels of the individual learning processes we fit the following simple model to our retention data LTR = α·xf(n) + β·xs(n). Here xf(n) and xs(n) represent the levels of the fast and slow learning processes for a given number of initial learning trials, n (see methods). Note that the parameters of this model, α and β, represent the specific contribution that each learning process makes toward long-term retention after 24 h, LTR. We fit this model with a simple linear regression to the group average retention data [R2 >0.99, F(2,2) = 10,020, P < 10−4] and to all of the individual subject 24-h retention data [R2 = 0.78, F(2,36) = 65.2, P < 10−6], and found that while the contribution of the slow learning process, β, was significantly different from 0 (95% confidence intervals, 0.49 ± 0.055), the contribution of the fast learning process, α, was not (95% confidence intervals, 0.052 ± 0.13). This suggests that the slow learning process contributes to long-term retention but the fast learning process does not, in agreement with the results displayed in Fig. 3.
Figure 4 summarizes the results of this study. The adaptation levels achieved near the end (last 10%) of the learning period for the four long-term retention groups are displayed in A. There are significant differences (P < 0.002) between the groups that experienced extensive training (160L24 and 103L24) and those experienced more limited training (30L24 and 11L24). The amount of retention of the motor adaptation for each group is shown in Fig. 4B. Similar to the results presented in panel a, the duration of training had a significant effect on the amount of adaptation retained; the 160L24 and 103L24 groups retained significantly more of the adaptation than the 30L24 and 11L24 groups (P < 0.009 in both cases). The percent retention (amount of retention after 24 h relative to the amount of adaptation at the end of training) is shown in Fig. 4C. Note that if recall were proportional to the total amount of initial learning (as in the predictions presented in Fig. 3, C and F) rather than to the level of the slow process, the percent recall would be held constant across groups. Remarkably, the two groups with the longest initial training periods, 160L24 and 103L24, show not only greater overall retention than the 11L24 group but also retain a greater percentage of their initial learning (P < 0.006 and P < 0.025, respectively). Note that the percent retention displayed by each group closely matches that predicted from our two-state model shown as the gray line using the alpha and beta found in the previous paragraph in Fig. 4C (R2 = 0.98), suggesting that this model has the ability to predict not only the overall retention level but the amount of forgetting as well.
This work investigated the processes involved in the evolution of short-term learning to long-term memory. Our results show that although the amount of 24-h retention is not well determined by the overall amount of initial learning, it is strongly related to that part of the initial learning accounted for by a slowly evolving subcomponent of the learning process. In our data, the level of this slow learning process on termination of initial training session is by itself able to predict the pattern of 24-h retention with an R2 of 99%. Furthermore, an extension of this model that adds a factor for a possible contribution from the fast learning process fails to significantly improve on this prediction. This suggests that the two distinct learning processes that underlie short-term motor adaptation contribute very differently to the long-term retention of motor memories—the slow process contributes strongly, while the fast learning process contributes weakly or not at all.
An extensive amount of theoretical and neurobiological work has focused on the mechanisms by which short-term motor memories are consolidated into a stable, enduring memory (Brashers-Krug et al. 1996; Nezafat et al. 2001; Shadmehr and Brashers-Krug 1997; Shadmehr and Holcomb 1997). For example, when subjects perform the same task used in this study 24 h after initial training, their performance is significantly improved over naïve performance (Brashers-Krug et al. 1996). This 24-h recall is disrupted when transcranial magnetic stimulation (TMS) is applied to the primary motor cortex prior to the initial training, whereas the initial learning of the task is comparable to control subjects (Richardson et al. 2006). Retention of initial learning has also been demonstrated for eye movement adaptation; 24 h after performing a gain reduction task, saccadic eye movement gain remains significantly lower than preadaptation gain (Alahyane and Pelisson 2005).
Here we have shown that the slow learning process provides a gateway to long-term memory formation. The dynamics of this slow process appear to determine the capacity for long-term retention of motor learning and consequently the properties of the overlearning effect. While this slow process accounts for as little as 10% of initial learning, it asymptotes at a level that accounts for ∼80% of overall learning. Therefore our model predicts that as training proceeds, both the amount of retention and the fraction of learning will increase. On the other hand, this model dictates when the slow learning process will asymptote, and thus predicts when the overlearning effect will saturate. Both of these predictions appear to be borne out in the experimental results.
Other studies have also shown that there are multiple time scales of learning during different types of motor adaptation (Fusi et al. 2007; Kording et al. 2007; Robinson et al. 2006). For example, the eye velocity of optokinetic nystagmus (involuntary eye movements in response to continuous movement of the visual field) is also characterized by two components: a rapid rise followed by a slower increase to steady state (Cohen et al. 1977). Interestingly, the rapid rise in eye velocity has been shown to be affected by particular neural lesions (Zee et al. 1987). The ability to disrupt specific time scales in the learning of motor adaptation tasks similar to that used in this study has been demonstrated (Della-Maggiore et al. 2004). During learning, when TMS was applied over the posterior parietal cortex, the initial rapid adaptation was unaffected, whereas the later gradual increase in learning was significantly reduced. Recently, imaging techniques have been used to identify the neural networks associated with the different time courses of learning during motor adaptation (Krakauer et al. 2004; Tunik et al. 2007). Together these results suggest that subcomponents of the learning process with different learning rates may be attributed to particular neural areas and opens the possibility that these areas may be manipulated to impair or enhance the formation of long-term motor memories.
On a more general note, this study shows that a simple computational model of motor adaptation has the ability to parse the overall performance in a learning task into one component that determines the amount of long-term retention and another component that does not. This ability may be a powerful tool for the rational design of motor training and rehabilitation paradigms that aim to maximize the long-term benefit of individual training sessions. Our results suggest that maximizing the long-term benefit of a training session does not necessarily come from maximizing the overall level of learning but rather from maximizing the amount of learning achieved by a single constituent learning process–the slow process. While the evolution of this slow component cannot be directly measured, it can be determined from computational modeling of learning processes that is informed by intimate knowledge of the interactions between them. This ability underscores the potential power of a computational modeling approach to create improved training procedures—such modeling can be used to design training paradigms that aim to maximize the efficiency for producing gains in the slow learning process as an effective surrogate for long-term retention.
This work was supported by Sloan Research Fellowship from the Alfred P. Sloan Foundation and a Scholar Award from the McKnight Endowment for Neuroscience to M. A. Smith.
We thank M. Ifediba for help with the experiments and G. Sing, G. Stanley, E. Robertson, and K. Blum for helpful discussions.
↵1 The online version of this article contains supplemental data.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2008 by the American Physiological Society