|
|
||||||||
1Department of Biomedical Engineering and 2Department of Neurology and Neuroscience, The Johns Hopkins School of Medicine, Baltimore, Maryland
Submitted 6 January 2008; accepted in final form 16 March 2008
|
|
ABSTRACT |
|---|
|
|
|
INTRODUCTION |
|---|
|
Recently, we proposed that a simple model of memory can explain this general phenomenon (Smith et al. 2006
). In this model, learning is affected by two factors: prediction error, which causes the brain to adapt and change its behavior to minimize errors in subsequent trials, and passage of time, which causes forgetting. The model explained that in a typical short-term training paradigm (many minutes), behavior was supported by two processes: a fast adaptive process that was highly sensitive to error but had poor retention and a slow adaptive process that had poor sensitivity to error but had robust retention. Behavior was the sum of these two processes. The model explained that spontaneous recovery occurred because a typical adaptation/reverse-adaptation protocol triggered a specific chain of events: during the initial long period of adaptation, most of the behavior became dependent on the slow adaptive process. Reverse-adaptation training forced behavior to return to baseline not through washing out of the learning, but by introducing a fast process that competed with the slow process. With passage of time after reverse-adaptation, the fast process faded, allowing an apparent recovery of the initial response.
To test the model, we needed a protocol to continuously assay the state of the learner's memory during the spontaneous recovery period. This is difficult, however, because the state of the learner can change because of errors that might occur on various trials and because of the passage of time from trial to trial. Theoretically, spontaneous recovery is best observed if passage of time is the only factor that influences behavior. Therefore we needed a protocol in which behavior was recorded continuously while errors were eliminated in every trial. These "error-clamp" trials were recently introduced in the context of reaching (Scheidt et al. 2000
; Smith et al. 2006
). Here, we used a similar approach in a saccade adaptation experiment to test whether behavior exhibited spontaneous recovery after adaptation/reverse-adaptation training.
|
|
METHODS |
|---|
|
Experimental setup
The position of either the right or the left eye was measured with a magnetic-field search-coil system (Robinson 1963
) using directional scleral annuli (Skalar Medical, Delft, The Netherlands). Raw coil signals were filtered in hardware (90-Hz low-pass Butterworth), digitized (1,000 Hz), and saved on computer for later analysis. Targets were rear-projected using a 2-mm-diameter mirror-controlled laser beam projected onto a translucent screen located 1 m in front of the subject with a 15° step response of >10 ms. The room was otherwise dark. The head of the subjects was stabilized with a bite bar of dental impression material.
Experimental paradigms
Saccade adaptation was induced using the standard double-step paradigm (McLaughlin 1967
). The experiment began with a sequence of error-clamp trials, then adaptation trials, and concluded with another set of error-clamp trials, as illustrated in Fig. 1.
|
ERROR-CLAMP TRIALS. Trials (Fig. 1B) began with a fixation target. At a random time in the range of 500–1,000 ms, the fixation target was turned off and target T1 was displayed at 15° horizontal. Once the saccade began, T1 disappeared and at 500 or 800 ms later (Groups 1 and 2, respectively) it reappeared at the position where the eye was located at 10 ms prior (i.e., 490 or 790 ms). In this way, first no and then a zero visual error were present after the saccade. Due to slight amplitude asymmetry between leftward and rightward saccade, occasionally a drift away from the center developed over time. We restrained eye position inside a ±20° range by resetting the fixation point to ±5° whenever the eye landed out of bounds. The mean intertrial time was 1,250 and 1,550 ms (for Groups 1 and 2, respectively).
CONTROL TRIALS. At the start of each experiment, subjects performed 20 trials in which the visual target at 15° horizontal remained present throughout the saccade and postsaccadic periods.
GROUP 1: GAIN-DOWN THEN GAIN-UP. The sequence of trials is illustrated inFig. 1C. Subjects performed 180 error-clamp trials consisting of 90, 60, and then 30 trials. They then had 480 gain-down trials followed by 60 gain-up trials. The gain-up trials were followed by a sequence of 30, 60, and 90 error-clamp trials. A brief rest period (30 s) was inserted between sets as shown in Fig. 1C.
GROUP 2: GAIN-UP THEN GAIN-DOWN. For this group, the pattern of training was the same as in Group 1 except that gain-down training followed gain-up training.
Data analysis
The duration of saccades was determined by a 20°/s speed threshold. Discriminating criteria were used to dismiss abnormal saccades. Saccades were rejected 1) if they did not reach a peak velocity >90°/s; 2) if they had a latency <100 ms; 3) if they displayed multiple peaks in their speed profile; and finally 4) if they were <50% of the target displacement. Most subjects had <5% of their saccade falling under one or more of these aforementioned criteria, with none of them exceeding 10% of all saccades.
System identification problem
We assumed that the learner's behavior (saccade amplitude) on any given trial depended on the values of two hidden states (effectively, the states of the memory). We represent these states with vector x (a 2 x 1). The states were affected by three factors: visual error
at the end of a saccade, passage of time between trials, and Gaussian noise
x. If we assume that the intertrial interval is constant, then we can write the change in states from trial to trial as
![]() | (1) |
y. This is written as
![]() | (2) |
The error on trial n is due to the intrasaccadic displacement that we imposed on the target. If we write that displacement as u(n), then the error on that trial is
![]() | (3) |
![]() | (4) |
In summary, the mathematical problem consists in finding the parameters of the adaptive system of Eq. 4, given a sequence of inputs u(n) (target displacements) and measurements y(n) (saccade amplitudes).
A recent breakthrough called subspace identification (van Overschee and De Moor 1996
) provides elegant, closed-form solutions for identification of stochastic linear systems. [A tutorial on the subject is available in the lecture notes of the last author's web page: www.shadmehrlab.org].
What we did in the preceding derivation is to transform our adaptive system equations into a form that can easily be solved by this approach. Once the parameters of the system are identified, we can estimate the state vector (i.e., the memory state) at each time step using a Kalman filter.
It is important to note that there is an infinite space of solutions to our problem, i.e., the same set of input and output data can be generated by an infinite number of systems that, from the perspective of Eq. 4, are indistinguishable. A particularly useful solution among these is one in which the matrix A is diagonal, assigning a unique time constant to each state. After estimation of the parameters, we transformed the system to one where A was diagonal. This produced a time constant of forgetting for each state. To interpret the timescales, we translated the state update equation of the discrete system of Eq. 4 to continuous time
![]() |
–1(A – bcT – I), bc =
–1b, and
is the intertrial interval, set to 1,250 ms. If we represent vector x as [xf, xs]T, i.e., the fast and slow states, then
s and
f refer to the time constant of the solution to this differential equation. |
|
RESULTS |
|---|
|
Figure 2, A and B shows representative and average saccade amplitude data for the two groups. In Group 1, saccades were hypometric (13.2 ± 0.47°, mean ± SE) in the initial set of error-clamp trials. The ensuing gain-down adaptation induced a rapid decrease in amplitude in the first set, followed by a slower decrease in the seven remaining sets. By the last set of gain-down training, saccade amplitude had dropped to 10.0 ± 0.32°. It took only 60 trials in the following gain-up training to bring the saccade amplitudes to near baseline (13.0 ± 0.30° for the last six trials). However, in the subsequent error-clamp trials saccade amplitudes did not remain stationary. Rather, they sharply declined (P = 0.002, one-sided t-test, within-subject comparison of the last six trials of gain-up vs. last six trials of the immediately following set of error-clamp trials). In the last two sets of the error-clamp trials amplitudes were 11.4 ± 0.52° (P < 0.0001, one-sided t-test, within-subject comparison of first and last two sets of error-clamp trials). Therefore when gain-down training was followed by a rapid period of gain-up training, in the following error-clamp trials the amplitudes reverted back toward the values achieved in the initial gain-down period.
|
Effect of error-clamp trials
Saccades in the initial error-clamp trials were hypometric, as is typical for saccades made in the dark to single sources of light (Collewijn et al. 1988
). However, no visual feedback was available in error-clamp trials for 500–800 ms. Did this delay in feedback influence saccade amplitudes? A recent report suggests that for large saccades (22–34°), lack of visual feedback at saccade termination may result in an increase in saccade amplitudes (Bonnetblanc and Baraduc 2007
). To check for this, we compared the amplitudes of saccades in the initial sets of error-clamp trials to a set of control saccades immediately preceding these trials: Before the main experiment began, subjects in both groups performed 20 trials in which the visual target at 15° remained lit throughout the saccade and postsaccade intertrial periods. For Group 1, amplitudes in the control period (13.3 ± 0.13°) were indistinguishable from the amplitudes in the following error-clamp trials (13.2 ± 0.47°, t-test, P > 0.5). To examine this question further, we increased the delay period in Group 2 to 800 ms from the value of 500 ms in Group 1. Similar to Group 1, amplitudes in the control period in Group 2 (13.3 ± 0.24°) were not reliably distinguishable from the amplitudes in the following error-clamp trials (12.9 ± 0.51°, P > 0.1). Therefore error-clamp trials did not appear to significantly bias the amplitudes of saccades.
Effect of the 30-s break between sets
The group data shown in Fig. 2, A and B suggest that the brief set breaks might have had a highly repeatable effect on saccade amplitudes. That is, in Group 1 the set breaks appeared to coincide with an increase in the amplitudes during the gain-down adaptation blocks, whereas in Group 2 the set breaks coincided with a decrease in the amplitudes during the gain-up adaptation blocks. To quantify this effect, for each subject we compared the last four saccades in each set with the first four saccades in the following set during the gain-down trials in Group 1 and gain-up trials in Group 2. On average, subjects in Group 1 showed an increase of 0.6 ± 0.17° [F(7,1) = 14.9, P < 0.01] and subjects in Group 2 showed a decrease of 0.6 ± 0.21° [F(8,1) = 7.84, P < 0.05] during the set breaks. Therefore the 30-s breaks between sets produced a significant forgetting. The amount of forgetting was indistinguishable between the two groups.
Multiple states of motor memory
Spontaneous recovery is a signature of an adaptive system that is supported by multiple states, each learning at a different timescale (Kording et al. 2007
; Smith et al. 2006
). To estimate these states and their timescales, we fitted the group data to a linear stochastic state-space model of learning (Eq. 4). In this system, the motor output (saccade amplitude) was supported by two hidden states. The changes in these states were a function of time between trials and endpoint errors observed on each trial.
Figure 2, C and D shows the fit of the model and plots the estimated contribution of each state to motor output, i.e., cTx = csxs + cfxf, in which the subscripts denote the slow and fast states. In the initial error-clamp trials, both states were near zero and were driven only by noise. In the subsequent adaptation stage, the fast state learned rapidly whereas the slow state lagged behind. During the subsequent 30-s rest period between sets, the fast state showed a large decay whereas the slow state showed little or no decay. This model exhibited two fundamental properties of memory: a fast system that learned quickly but showed poor retention with passage of time and a slow system that learned slowly but had significantly better retention.
As the adaptation trials continued, the slow state overtook the fast state so that by the end of gain-down training in Group 1 or gain-up training in Group 2, the slow state's contribution was twice that of the fast state. Therefore by the end of 480 adaptation trials (just before start of reverse-adaptation block) performance was dominated by the slow state.
In the subsequent set break, the fast state once again decayed toward baseline. However, in the subsequent training block (reverse-adaptation training) the errors were in the opposite direction of the previous training, forcing the fast state to rapidly learn and acquire values that now competed with the values of the slow state. By the end of the reverse-adaptation block, the sum of the two states was near baseline (Fig. 2C, top subplot, line labeled "Model"), but of course the two states had not returned to baseline.
In the following error-clamp trials (note that there was no set break here), the fast state rapidly declined to baseline, whereas the slow state gradually declined. The sum effect was spontaneous recovery. That is, in Group 1 motor output reverted back toward the gain-down pattern and in Group 2 it reverted back toward the gain-down pattern.
The fit to the group data produced a decay time constant of
f = 28 s for the fast state and
s = 7 min for the slow state. Error sensitivity of the fast state was 18-fold larger for the fast state, i.e., bf/bs = 18. Finally, saccade amplitudes relied almost twice as much on the slow than on the fast state, i.e., cs/cf = 1.7.
We next examined the distribution of these estimates by fitting the model to each subject and condition. The fit was satisfactory in 14 of 17 subjects and their parameter values are plotted in Fig. 3. The data for the three subjects that we could not fit exhibited large variability in saccade amplitudes during the initial error-clamp trials. For the remaining 14 subjects, we found that
f = 28 ± 7 s (mean ± 95%CI),
s = 7.8 ± 3 min, bf/bs = 18 ± 6, and cs/cf = 1.7 ± 0.5. Therefore the timescales of the fast and the slow systems, as well as their sensitivity to error, were an order of magnitude apart.
|
|
|
DISCUSSION |
|---|
|
Is there evidence that in short-term paradigms (
1 h) the memory that supports generation of saccades is affected by both the passage of time and errors? Seeberger et al. (2002)
reported that when a block of gain-down adaptation trials (in which the target was moved intrasaccadically and remained visible in the postsaccadic period) was followed by trials in which the postsaccadic target was turned off, the reduced saccade amplitudes caused by gain-down adaptation gradually returned back to baseline. Therefore passage of time produced a decay in the memory acquired during adaptation. However, the return to baseline was faster if the target remained stationary and the endpoint errors encouraged extinction. This is consistent with a model in which both error and passage of time affect the state of the learner.
If the saccades of the learner are supported by a memory that can be represented as a single state, then adaptation followed by extinction will not produce spontaneous recovery (Smith et al. 2006
). Rather, in the postextinction period saccade amplitudes simply return to baseline. However, if memory is effectively supported by two or more states, then the differential sensitivity of the purported states to error and passage of time can produce spontaneous recovery. That is, in trials in which errors are eliminated, the fast state should rapidly return to baseline, leaving behavior that reflects the slow state. Indeed, in the 30 error-clamp trials postextinction we observed a roughly 50% recovery of the saccade amplitudes. We estimated that during these trials the learning from the fast state had completely dissipated.
We found that during the 30-s break that separated each adaptation set, saccade amplitudes changed by about 0.6°, with a direction that was opposite to the direction of adaptation. The model explained that this forgetting was due to the same fast process that produced spontaneous recovery in the postextinction error-clamp trials. Therefore it was crucial not to introduce a break between the extinction block and the error-clamp block. Such a break would prevent us from observing the rapid postextinction change in saccade amplitudes.
How accurate is our assumption that error-clamp trials eliminated the error signal? The errors that drive saccade adaptation are derived mainly from visual information during the postsaccade period (Noto and Robinson 2001
; Wallman and Fuchs 1998
). When this information is delayed, the resulting adaptation is reduced or eliminated. For example, a 750-ms delay reduces the error-dependent adaptive response by 90% (Shafer et al. 2000
). In our paradigm, we not only included a delay in presenting the postsaccadic visual information, but we also presented the visual target at the current eye position. Therefore the combined techniques should reduce any error-driven adaptive response in a given trial even further.
Because saccades in the dark are hypometric, it is conceivable that when a postsaccadic target appears at the current eye position the brain interprets this as an error and attempts to reduce amplitude of the subsequent saccade. This would predict that error-clamp trials should increase hypometria. We checked for this by comparing saccade amplitudes in response to targets that did not disappear (control trials before start of the error-clamp trials) versus targets that disappeared and then were shown after a delay at the current eye position (error-clamp trials) and did not find a significant difference. If error-clamp trials introduced inadvertent errors, the effect must be quite small because the same model that assumed zero error in error-clamp trials and accounted for spontaneous recovery also explained the forgetting during the 30-s rest periods during which subjects were in complete darkness.
Perhaps the most significant contribution of the model is that it suggests the possibility that even in brief adaptation experiments, changes in behavior are a function of not only error, but also passage of time. The initially rapid and then gradual changes in the postextinction trials are largely due to the effect of time passage on the states that supported the memory during error-dependent adaptation. The model predicts that, functionally, these states have both fast and slow timescales, with values that are an order of magnitude apart. An order of magnitude also differentiates the sensitivities that these memory states express with respect to error. If one imagines that there are not two but many more states that support a memory, and their sensitivity to passage of time and error is distributed along a logarithmic scale, then such models may be able to account for saccade adaptation data on timescales of weeks and months (Kording et al. 2007
). Therefore such motor memory becomes "cemented" only if it is repeated, causing adaptation in the slowest of the slow states.
The adaptive behavior that we examined here is grossly impaired when there is damage to the oculomotor vermis in the cerebellar cortex (Barash et al. 1999
; Takagi et al. 1998
) or fastigial oculomotor region in the cerebellar deep nuclei (Robinson et al. 2002
). In other cerebellar-dependent paradigms it has been hypothesized that the initially rapid changes in behavior during adaptation may be due to changes in the cerebellar cortex, whereas the retention of memory over the long-term (days) may depend on the cerebellar nuclei (Bracha et al. 2000
; Medina et al. 2001
; Shutoh et al. 2006
). Of course, saccade adaptation may also engage structures other than the cerebellum. Therefore one possibility is that the two timescales reflect changes in distinct anatomical substrates that all contribute to the motor commands that move the eyes.
This line of thinking would be bolstered if there was evidence for the two states in the trajectory of saccades. In a recent examination of cross-axis adaptation, we indeed found that the commands that initiated the saccade adapted more slowly than commands that arrived later in the same saccade (Chen-Harris et al. 2008
). For example, the 30-s break between sets produced dramatic forgetting in the late-acting motor commands, yet it had essentially no effect on the motor commands that initiated the saccade. If the late-acting motor commands can be attributed to a "steering" mechanism of the eyes via the cerebellum (Kojima et al. 2008
), then it is possible that the fast states are a reflection of a changing contribution from this structure.
However, the fact that we have observed two functional states of memory in no way implies involvement of two distinct neural structures. In principle, the two states may reflect mechanisms of plasticity in a single neuron. From the perspective of the two-state model, spontaneous recovery is closely related to "savings": when adaptation is followed by extinction, subjects exhibit faster relearning in a subsequent readaptation block. The two-state model explains both phenomena using the concept of fast and slow systems (Smith et al. 2006
). Recently, Jirenhed et al. (2007)
demonstrated that behavioral changes during adaptation, extinction, and savings in a classical conditioning task that lasted about 16 h were correlated with discharge of single Purkinje cells in the cerebellar cortex. Therefore the multiple states that produced savings in that paradigm did not appear to require two or more distinct neural substrates, but were perhaps part of the molecular machinery that supported synaptic plasticity in single Purkinje cells.
|
|
FOOTNOTES |
|---|
Address for reprint requests and other correspondence: R. Shadmehr, Johns Hopkins School of Medicine, 410 Traylor Bldg., 720 Rutland Ave., Baltimore, MD 21205 (E-mail: shadmehr{at}jhu.edu)
|
|
REFERENCES |
|---|
|
Bonnetblanc F, Baraduc P. Saccadic adaptation without retinal postsaccadic error. Neuroreport 18: 1399–1402, 2007.[CrossRef][Web of Science][Medline]
Bracha V, Zhao L, Irwin KB, Bloedel JR. The human cerebellum and associative learning: dissociation between the acquisition, retention and extinction of conditioned eyeblinks. Brain Res 860: 87–94, 2000.[CrossRef][Web of Science][Medline]
Chen-Harris H, Joiner WM, Ethier V, Zee DS, Shadmehr R. Adaptive control of saccades via internal feedback. J Neurosci 28: 2804–2813, 2008.
Collewijn H, Erkelens CJ, Steinman RM. Binocular co-ordination of human horizontal saccadic eye movements. J Physiol 404: 157–182, 1988.
Jirenhed DA, Bengtsson F, Hesslow G. Acquisition, extinction, and reacquisition of a cerebellar cortical memory trace. J Neurosci 27: 2493–2502, 2007.
Kojima Y, Iwamoto Y, Robinson FR, Noto CT, Yoshida K. Premotor inhibitory neurons carry signals related to saccade adaptation in the monkey. J Neurophysiol 99: 220–230, 2008.
Kojima Y, Iwamoto Y, Yoshida K. Memory of learning facilitates saccadic adaptation in the monkey. J Neurosci 24: 7531–7539, 2004.
Kording KP, Tenenbaum JB, Shadmehr R. The dynamics of memory as a consequence of optimal adaptation to a changing body. Nat Neurosci 10: 779–786, 2007.[CrossRef][Web of Science][Medline]
McLaughlin S. Parametric adjustment in saccadic eye movements. Percept Psychophys 2: 359–362, 1967.[Web of Science]
Medina JF, Garcia KS, Mauk MD. A mechanism for savings in the cerebellum. J Neurosci 21: 4081–4089, 2001.
Myers KM, Davis M. Behavioral and neural analysis of extinction. Neuron 36: 567–584, 2002.[CrossRef][Web of Science][Medline]
Noto CT, Robinson FR. Visual error is the stimulus for saccade gain adaptation. Brain Res Cogn Brain Res 12: 301–305, 2001.[CrossRef][Medline]
Robinson DA. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans Biomed Eng 10: 137–145, 1963.[Medline]
Robinson FR, Fuchs AF, Noto CT. Cerebellar influences on saccade plasticity. Ann NY Acad Sci 956: 155–163, 2002.[Web of Science][Medline]
Scheidt RA, Reinkensmeyer DJ, Conditt MA, Rymer WZ, Mussa-Ivaldi FA. Persistence of motor adaptation during constrained, multi-joint, arm movements. J Neurophysiol 84: 853–862, 2000.
Seeberger T, Noto C, Robinson F. Non-visual information does not drive saccade gain adaptation in monkeys. Brain Res 956: 374–379, 2002.[CrossRef][Web of Science][Medline]
Shafer JL, Noto CT, Fuchs AF. Temporal characteristics of error signals driving saccadic gain adaptation in the macaque monkey. J Neurophysiol 84: 88–95, 2000.
Shutoh F, Ohki M, Kitazawa H, Itohara S, Nagao S. Memory trace of motor learning shifts transynaptically from cerebellar cortex to nuclei for consolidation. Neuroscience 139: 767–777, 2006.[CrossRef][Web of Science][Medline]
Smith MA, Ghazizadeh A, Shadmehr R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol 4: e179, 2006.[CrossRef][Medline]
Takagi M, Zee DS, Tamargo RJ. Effects of lesions of the oculomotor vermis on eye movements in primate: saccades. J Neurophysiol 80: 1911–1931, 1998.
van Overschee P, De Moor B. Subspace identification for linear systems. Boston, MA: Kluwer Academic, 1996. May be accessed at http://homes.esat.kuleuven.be/
smc/sysid/software/
Wallman J, Fuchs AF. Saccadic gain modification: visual error drives motor adaptation. J Neurophysiol 80: 2405–2416, 1998.
This article has been cited by other articles:
![]() |
M. Xu-Wilson, H. Chen-Harris, D. S. Zee, and R. Shadmehr Cerebellar Contributions to Adaptive Control of Saccades in Humans J. Neurosci., October 14, 2009; 29(41): 12930 - 12939. [Abstract] [Full Text] [PDF] |
||||
![]() |
J.-Y. Lee and N. Schweighofer Dual Adaptation Supports a Parallel Architecture of Motor Memory J. Neurosci., August 19, 2009; 29(33): 10396 - 10404. [Abstract] [Full Text] [PDF] |
||||
![]() |
V. S. Huang and R. Shadmehr Persistence of Motor Memories Reflects Statistics of the Learning Event J Neurophysiol, August 1, 2009; 102(2): 931 - 940. [Abstract] [Full Text] [PDF] |
||||
![]() |
V. Ethier, D. S. Zee, and R. Shadmehr Changes in Control of Saccades during Gain Adaptation J. Neurosci., December 17, 2008; 28(51): 13929 - 13937. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. E. Criscimagna-Hemminger and R. Shadmehr Consolidation Patterns of Human Motor Memory J. Neurosci., September 24, 2008; 28(39): 9610 - 9618. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |