HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) All Versions of this Article: 97/1/451    most recent 01243.2005v1 Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (3) Citing Articles via Google Scholar Google Scholar Articles by Glasauer, S. Articles by Ivanenko, Y. P. Search for Related Content PubMed PubMed Citation Articles by Glasauer, S. Articles by Ivanenko, Y. P.

Space–Time Relativity in Self-Motion Reproduction

Stefan Glasauer^1,2, Erich Schneider¹, Renato Grasso^3, and Yuri P. Ivanenko³

¹Center for Sensorimotor Research, Department of Neurology, Klinikum Grosshadern, Ludwig-Maximilians University; ²Julius Bernstein Center for Computational Neuroscience, Munich, Germany; and ³Department of Neuromotor Physiology, Istituto di Ricovero e Cura a Carattere Scientifico Fondazione Santa Lucia, Rome, Italy

Submitted 28 November 2005; accepted in final form 18 October 2006

	ABSTRACT

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
Experiments on reproducing imposed self-motion showed that not only final distance or angle of motion, but also the temporal profile are reproduced. Reproduction errors have been attributed to sensory inputs, inaccurate memorization of the motion variable, or motor errors. However, another possible source of error has so far been neglected. The internal time base for path integration or movement memorization may be distorted and thus not reflect physical time. Because additional cognitive load was previously shown to affect subjective estimation of duration, we used a dual-task paradigm during either the stimulation or reproduction phase of three different movement reproduction tasks. We asked subjects 1) on a rotating chair to reproduce imposed passive whole body rotations by controlling the chair with a joystick, 2) on a treadmill to actively reproduce locomotion with respect to the treadmill, and 3) while blindfolded to reproduce a previously walked straight trajectory. The cognitive load changed the distance of reproduced self-motion by about 25% depending on whether the mental task was performed while experiencing or reproducing the motion. Although imposed velocity was reproduced accurately in all conditions, reproduced movement duration was affected in the same way as distance. This result implies that for the perception of distance traveled, perceptual space and time are closely interrelated. The findings are consistent with shared processing of temporal and spatial information. A computational model of motion reproduction including a discrete path integrator is proposed that is able to explain the experimental results within one coherent framework.

	INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
"When a man sits with a pretty girl for an hour, it seems like a minute. But let him sit on a hot stove for a minute and it's longer than any hour. That's relativity" (Einstein 1938).

Although there is hardly any link to the physics of Einstein's special theory of relativity, this humorous aphorism truly reflects our commonsensical knowledge that perception and processing of the flow of time are not constant but rather depend on "the state of mind of the observer" (Einstein 1938). This so-called interference effect, i.e., the effect of nontemporal tasks on perceived time, is consistently found in the literature of time perception (for review, see Brown 1997). The effect is explained by attentional models of timing in terms of allocation of processing resources. Accordingly, during a dual task, less attention can be devoted to "timekeeping," which leads to the observed interference.

Time plays a role not only for temporal tasks, such as playing the piano, but also for many tasks that, at first glance, would be called "spatial." The successful reproduction of a previously experienced movement requires the accurate memorization of the movement and, subsequently, the formation of an adequate motor program. However, for accurate reproduction it is not sufficient to memorize the movement, e.g., as successive spatial locations. Rather, the time intervals between the memorized locations are of critical importance. This becomes evident when considering simple examples: we follow our skiing teacher downhill to then attempt to reproduce this elegant motion on our own; or we try to repeat a winning serve that our tennis coach demonstrates. In any case, temporal distortion of the reproduced motion will render our efforts unsuccessful. Reproduction of movement, which can be physically characterized by the instantaneous values of position, velocity, and time, is thus a spatiotemporal task. It is, however, not known whether the interference effect, described above for the perception of time, also affects the perception of space in tasks such as movement reproduction.

In the case of self-motion, the position, or distance traveled, is often not directly perceived and is thus unavailable for memorization, e.g., when it cannot be inferred from a salient, reliable landmark. Studies on the reproduction of linear self-displacement using robots (Berthoz et al. 1995; Grasso et al. 1999; Israël et al. 1997) or simulated visual flow (Bremmer and Lappe 1999) showed that not only final distance can be reproduced, but also the spatiotemporal profile of the experienced motion. As mentioned above, to achieve accuracy in movement reproduction, a biological system evidently needs to store both spatial and temporal aspects of the motion profile. For the storage of the spatial aspects of a movement profile either self-position values derived by path integration (Mittelstaedt and Mittelstaedt 1980) or self-velocity values without the need for path integration might be used (Berthoz et al. 1995). Path integration, which is inevitable in navigation tasks such as homing (for review, see Etienne and Jeffery 2004), ensures updating of positional parameters (distance and orientation) and is operating even in simple tasks such as goal-directed locomotion (Glasauer et al. 1994, 2002) or keeping track of self-orientation during passive displacement (Ivanenko et al. 1997). It may be performed by temporal integration of velocity signals, such as optic flow or angular velocity signals from the vestibular system, thus also involving temporal processing. Alternatively, spatial integration of instantaneous displacement, e.g., accumulation of step length during locomotion, may be used (Mittelstaedt and Glasauer 1993) and was shown to occur in ants (Wittlinger et al. 2006). For accurate processing and memorization of time either an explicit linking of spatial values to time stamps might be used, or the time might be coded implicitly by sampling and reproducing spatial values on an assumed identical time base, i.e., by using or assuming the same sampling time while both experiencing and reproducing a movement. But what happens if the assumed equivalent time base is distorted?

In the present study, we sought to answer whether and how an internal representation of time plays a role in human movement reproduction. We used concurrent mental arithmetic as a "tool" to modify the internal representation of time either while experiencing or while reproducing a movement. This mental task was previously shown to interfere with the reproduction of temporal intervals (Brown 1997; Burnside 1971; Wilsoncroft and Stone 1975). If movement reproduction would rely on the same internal time base as temporal interval reproduction, we expected that the reproduced distance would depend on whether mental arithmetic is performed either during stimulus presentation or during reproduction. We chose three experiments to evaluate whether mental arithmetic affects temporal information for movement reproduction at all, whether dual-task interference is found both during acquisition and reproduction, and whether the results of one sensorimotor modality can be generalized to others. The experimental results were compared with predictions from different versions of a mathematical model of self-motion reproduction to evaluate which of the various temporal processes involved may be affected by the dual-task interference.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
In all three experiments, subjects had to reproduce a previously experienced motion under three conditions. In the control condition, no mental task was performed. In MTE (Mental Task during Experienced motion), mental arithmetic (counting backward by 7) was performed while experiencing the motion; in MTR (Mental Task during Reproduction), mental arithmetic was performed while reproducing the motion. Twenty-seven naive subjects participated in the four experiments, which are described below in detail. Subjects experienced a motion and were asked to subsequently reproduce the motion by different methods. In each experiment, the instruction to the subjects was to "reproduce the experienced movement as accurately as possible." In the experiments passive rotation and treadmill locomotion (see Fig. 1), six different motion profiles (triangular velocity profile; three durations of 10, 15, or 20 s x two maximal velocities) were presented randomly and tested under the three experimental conditions: control, MTE, and MTR. The distances d are thus computed by d = v x (t/2), where v is the peak velocity and t is the duration of the triangular profile. The experimental protocol for rotation and treadmill thus yielded 18 trials for each subject. For the mental task, the subject started to count backward in steps of 7 as soon as an experimenter said a three-digit number immediately before the movement onset. To ensure that subjects continuously performed this arithmetic task, they had to count aloud. There was no feedback on successful reproduction. In the third experiment, overground locomotion, we used the same experimental protocol (three durations of 10, 15, or 20 s) but asked the subjects to walk at normal speed, yielding nine trials per subject.

View larger version (14K): [in this window] [in a new window]   FIG. 1. Experiments. A: rotation. Reproduction of self-motion on a rotating chair: drawings of a subject being rotated and then reproducing the rotation together with raw velocity data (gray: stimulus; black: reproduction). Black rectangles indicate the optokinetic drum, which was visible and stable in space during the whole experiment. B: treadmill. Reproduction of locomotor movement on a treadmill. Drawing shows the experimental paradigm of walking on a treadmill while experiencing movement and during reproduction. During reproduction, a position sensor connected to the subject ensured that the treadmill velocity (v) was proportional to the subject's position (x). Initial position was 0.5 m from the rear end of the treadmill.

To estimate whether subjects correctly reproduced the imposed triangular velocity profiles, each reproduced velocity profile was normalized in duration. The resulting time-normalized velocity profiles were then analyzed by two methods. First, a principal components (PC) analysis was performed to assess variability between profiles. The principal components were extracted using the correlation matrix. Second, to estimate whether reproduced profiles were triangular, each reproduced profile was correlated with a normalized triangle. Statistics were performed on z-transformed correlation coefficients.

Post hoc tests were performed using Scheffé's test. Values of P < 0.05 were considered significant. In the text, means ± SEs are reported.

Rotation: memorization and reproduction of self-rotation in a rotating chair

Seven naive subjects (age 26–65 yr; three females, four males) participated. The subjects were seated in a rotating chair (Toennies turntable, Jaeger/Toennies, Freiburg/Höchberg, Germany) surrounded by a space-fixed immobile drum (see Fig. 1A) lined with vertical black and white stripes (visual angle 7.2°). Thus subjects received both vestibular and visual self-velocity information, but no information about their position. The imposed chair rotation (always to the right) was computer controlled (triangular velocity profile, 10-, 15-, or 20-s duration; 25 or 50°/s maximal velocity). The distance ranged from 125 to 500°. After receiving instructions, the subjects familiarized themselves for about 5 min with the use of the joystick to control the rotating chair. After each experienced velocity profile, the subjects tried to reproduce the movement using the joystick. The inclination angle of the joystick was proportional to chair velocity (maximum velocity 100°/s). The chair velocity, position, and the joystick position were recorded digitally at a sampling rate of 200 Hz. For data analysis, movement onset was defined as the time when chair velocity exceeded 2.5°/s; the end, when chair velocity first fell below 2.5°/s after reaching its maximum.

Treadmill: memorization and reproduction of a short path by treadmill locomotion

Seven naive subjects (age 15–40 yr; one female, six males) were asked to reproduce an imposed treadmill movement by treadmill locomotion. Full vision of the laboratory and the treadmill was allowed. During imposed treadmill movements, subjects had to compensate the treadmill motion by walking. After familiarization with the treadmill (Model XELG 70; Woodway, Weil am Rhein, Germany), six imposed movements of the treadmill belt (triangular velocity profile, 10-, 15-, or 20-s duration; 0.7 or 1.4 m/s maximal velocity) were delivered at random, resulting in distances from 3.5 to 14 m. After the treadmill stopped, the subject was asked to reproduce this movement by walking on the treadmill. During reproduction, subjects controlled treadmill velocity (maximum 2.3 m/s) by their distance from their initial starting position (50 cm from the rear end of the treadmill, Fig. 1B), i.e., treadmill velocity was proportional (sensitivity 1.5 s^–1) to the subject's displacement (Ivanenko et al. 2000). Thus to generate a treadmill velocity of 1 m/s, subjects had to adopt a position 0.67 m from their initial starting position. Treadmill velocity was recorded by an optical encoder (resolution 0.005 m/s) and was computer controlled (30 Hz). For treadmill control, the subject's position was recorded with a potentiometer (accuracy 2 mm) that measured the position of a lightweight, stiff thread attached to the subject. Data analysis was done as described for rotation, except that the threshold velocity for detecting movement onset and end was set to 0.05 m/s.

Locomotion: overground locomotion at normal walking speed

Seven naive subjects (age 19–44 yr; two females, five males) were asked to walk in a large horizontal open space (100 x 17 m) at constant normal speed with eyes closed throughout the experiment. An experimenter was always beside the subject to hold him/her in case of falls (which never happened). Nine trials were performed by each subject: three durations of 10, 15, or 20 s x three tasks (control, MTE, and MTR). In each trial, an experimenter verbally provided start and stop signals for forward walking. Immediately after stopping, the subject was asked to reproduce the movement. The distance was measured with a measuring tape and the duration with a stop watch. The total distance and duration during the reproduction phase ranged from 6.5 to 49 m and from 6 to 39 s, respectively, across all conditions. Walking speed was estimated as distance divided by duration (overall mean 1.25 ± 0.05 m/s).

Models of movement reproduction

To better understand the effects of a modification of internal time on movement reproduction, we compared the predictions of four different computational models of movement reproduction (Fig. 2; see APPENDIX for equations) based on previous work (Grasso et al. 1999). Common to all models was memory storage for one of the spatial variables, either distance derived from path integration or self-velocity available from sensory information. The spatial variable was stored in memory during acquisition and retrieved during reproduction. Reproduction was modeled as a feedback loop generating the motor command from the error between the retrieved (desired) and the actual variable. Mental arithmetic was supposed to affect either the time base for memorization, the time base for path integration, or both. The temporal distortion was modeled as factors _t for memorization and _x for path integration. If internal and physical time matched, the factors would equal unity. The distinction between internal time bases for memorization (_t) and path integration (_x) was introduced because path integration may be an automated low-level process operating on a short temporal range and running independently of a possible distortion of perceived time.

View larger version (27K): [in this window] [in a new window]   FIG. 2. Models for the effect of a mental task on encoding and reproduction of a self-motion. Four different versions are proposed (rows 1–4; see METHODS and APPENDIX). Black flash indicates the influence of the mental task at different stages of processing. Left column: imposed motion is stored in spatial memory either as velocity profile (1) or as distance profile (2, 3, 4). Right column: during reproduction, an on-line comparison of the measured stored variable (velocity or distance) and the estimated variable is used to issue a motor command, which generates the motion and thus forms a feedback loop. Mental arithmetic is assumed to affect memory storage and retrieval (1, 2, 4) and/or path integration (3, 4).

Four major versions of this model are possible (Fig. 2):

1. ) Storing a velocity profile in which mental activity affects memory access during encoding or retrieval by dilating the time interval between successively stored or retrieved velocity values (no _x; _t < 1)
   
2. ) Storing a distance profile in which mental activity affects memory access, but not path integration (_x = 1; _t < 1)
   
3. ) Storing a distance profile in which mental activity affects path integration by dilating time for path integration, but not for memory access (_x < 1; _t = 1)
   
4. ) Storing a distance profile in which mental activity affects both memory access and path integration in the same way (_x = _t < 1), the rationale being that the time needed for one step of path integration determines the memory access rate.
   

Numerical models were simulated using Matlab (The MathWorks) with a fixed time step of t = 1 ms and Euler-forward integration (e.g., distance x_i+1 at time t_i+1 is computed as x_i+1 = x_i + v_i · t, where x_i and v_i are, respectively, velocity and distance at the previous time step t_i). This integration method is simple but accurate if the involved time constants are much longer than the time step t. Model predictions were derived analytically (see APPENDIX), verified by numerical simulation, and are presented in the RESULTS section.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
In the first experiment (self-rotation), subjects reproduced passive whole body motion (see Fig. 1A) by controlling a rotating chair with a joystick. Subjects received only visual and vestibular self-velocity information. The second experiment (treadmill) tested whether the results from the first experiment could also be generalized to a reproduction of an active locomotor movement on a treadmill (Fig. 1B). In this experiment, vestibular information was not useful in determining treadmill velocity. Instead, subjects could use proprioception, efference copy, or visual information about treadmill speed to infer their relative motion with respect to the treadmill. In the last experiment (locomotion), we tested whether natural overground locomotion would yield similar results. Table 1 summarizes the three experiments and their main properties. The dependent variables' distance, duration, and peak velocity were extracted from the raw data. The statistical analysis was performed on the gains (reproduced variable divided by experienced variable).

To determine whether reproduced distance depended on experiment, condition, or the independent variables' duration and velocity, we normalized each reproduced distance by the respective experienced distance (distance gain). Distance gain (Fig. 3A) showed a highly significant effect of condition [F(2,17) = 44.5, P < 0.001], which resulted from smaller distance gains for MTE (0.76 ± 0.03, mean ± SE) and larger gains for MTR (1.38 ± 0.07) compared with control (1.03 ± 0.04).

View larger version (17K): [in this window] [in a new window]   FIG. 3. Average distance, duration, and velocity gains. Gains are shown for all experiments and conditions (error bars: ± SD). Black: control; dark gray: MTE (mental task during experienced motion); light gray: MTR (mental task during reproduction). Dashed lines indicate a gain of unity as required for perfect reproduction. A: distance gain. Reproduced distance divided by experienced distance. Distance gain was lower than that in the control condition when mental arithmetic was performed while experiencing the motion (MTE) and higher in MTR. B: duration gain. Reproduced duration divided by experienced duration. Reproduced duration depended on condition in all experiments. C: velocity gain. Reproduced peak velocity divided by experienced peak velocity. Velocity gain did not depend on condition.

View larger version (15K): [in this window] [in a new window]   FIG. 4. Gains over experienced variable. Gains are plotted over the respective experienced variable for all experiments (rows) and conditions (columns). Each point represents one single experimental trial. Bold dashed lines are linear least-squares fits; the normal dashed lines indicate a gain of unity as required for perfect reproduction. Gray dots represent trials with high peak velocity (rotation 50°/s; treadmill 1.4 m/s); black with low peak velocity (rotation 25°/s, treadmill 0.7 m/s). A: distance gain over experienced distance. Distance gain was lower in MTE than in the control condition and higher in MTR. Additionally, distance gain was higher for low peak velocity (corresponding to lower distance) than for high peak velocity, except for locomotion, when subjects were required to walk at normal speed. B: duration gain over experienced duration. Reproduced duration depends on experimental condition in all experiments. C: velocity gain over experienced velocity. Velocity gain did not depend on the condition.

Reproduction of duration

To determine why reproduced distance depended on the experimental condition, we analyzed reproduced duration gain. The repeated-measures ANOVA showed main effects for all dependent variables [experiment: F(2,18) = 11.0, P < 0.001; condition: F(2,17) = 60.2, P < 0.001; duration: F(2,17) = 44.9, P < 0.001]. The effect of condition (Fig. 3B) arose from lower-duration gains during MTE (0.71 ± 0.03) compared with control (0.97 ± 0.02) and higher gains during MTR (1.25 ± 0.05). Duration gains were different for each experiment, with the lowest gain for rotation and highest for locomotion (rotation: 0.84 ± 0.04; treadmill: 0.97 ± 0.04; locomotion: 1.12 ± 0.04). A significant interaction between experiment and condition [F(4,34) = 5.4; P = 0.002] was mainly the result of experiment-specific differences in MTR, where the increase in duration gain compared with control was most visible for locomotion and almost absent for rotation (Fig. 3B). The dependency of duration gain on experienced duration resulted from longer experienced durations leading to shorter reproduced durations (Fig. 4B). This was found only for rotation and treadmill, but not for locomotion, as shown by a significant interaction of experiment and duration [F(4,34) = 8.1, P < 0.001]. For rotation and treadmill (separate ANOVA), duration gain also depended on velocity [F(1,12) = 14.0, P = 0.001] with longer reproduced duration for high velocity.

In summary, reproduced duration decreased when mental activity was required while experiencing motion (MTE), compared with the control condition, and increased in MTR for all experiments.

Reproduction of the velocity profile

To ascertain how well subjects reproduced the experienced triangular velocity profile, we performed a principal components analysis (PCA) on the time-normalized profiles (Fig. 5) separately for the first two experiments (rotation, treadmill) and all conditions. The first principal components (Fig. 6), which can be interpreted as the average shape of the reproduced velocity profiles, accounted for >70% of the variance for rotation and >85% for treadmill. The triangular form of the principal component together with the equally high values of explained variance shows that subjects roughly reproduced the angular velocity profile for rotation and treadmill regardless of condition. Therefore the observed effects on distance did not arise from different reproduction of velocity profiles.

View larger version (20K): [in this window] [in a new window]   FIG. 5. Average velocity profiles. Average velocity data for each experiment. Data were normalized to the mean duration of each condition to show the approximately triangular velocity profiles. Rows: 1st: low velocity; 2nd: high velocity. Columns: durations 10, 15, and 20 s. Black: control; dark gray: MTE (mental task during experienced motion); light gray: MTR (mental task during reproduction). A: rotation. 1st row: 25°/s peak velocity, 2nd row: 50°/s peak velocity. Subjects reproduced peak velocity and profile, but the duration of movement was always shortest for MTE. For the high-velocity condition (bottom row), the duration was longest for MTR. Peak at the end of the top right red trace is attributed to one outlier that was not removed from the data. B: treadmill. 1st row: 0.7 m/s, 2nd row: 1.4 m/s. As in A, velocity profiles were roughly triangular with the shortest duration for MTE.

View larger version (15K): [in this window] [in a new window]   FIG. 6. Principal components analysis (PCA) of reproduced velocity profiles. First (solid) and second (dashed) principal component for each condition (Black: control; dark gray: MTE; light gray: MTR) are shown. Raw data from movement reproduction were first normalized in time and then analyzed by PCA. Legend shows how much of the overall variance is explained by the first principal component (PC). A: passive rotation. B: treadmill locomotion.

Our finding that subjects in the rotation experiment could faithfully reproduce the velocity profile contradicts that of a previous study that also used a rotating chair but in darkness (Siegler et al. 2000). The authors failed to find any reproduction of the velocity profile. It is possible that our more natural condition, which provided both vestibular and visual information, helped subjects to store and reproduce the motion profile.

We further analyzed whether subjects accurately reproduced the experienced peak velocity (Fig. 3C). Repeated-measures ANOVA on peak velocity gain (reproduced peak velocity divided by experienced peak velocity) showed no significant effects or interactions, confirming that velocity gain was nearly unity, independent of condition or experiment (Fig. 3C). For rotation and treadmill, velocity gain depended on imposed velocity [F(1,12) = 42.8, P < 0.001], showed a two-way interaction of velocity and condition [F(2,11) = 14.1, P = 0.001], and a three-way interaction of condition–duration–velocity [F(4,9) = 5.4, P = 0.017]. These effects arose from lower velocity gains for high velocity, specifically for mental task conditions. When the mental task was performed during movement reproduction (MTR), velocity gain also depended on duration and the gains were larger for longer duration.

In summary, the subjects reproduced the velocity profile and the peak velocity independently of the experiment and the condition.

Comparison with model predictions

A comparison of how reproduced distance depends on experienced distance reveals a uniform picture across the experiments. In the control condition, distance was accurately reproduced, whereas in MTE the reproduced distance was smaller and in MTR larger. This condition-dependent change in reproduced distance was caused by a respective change in reproduced duration, but not in velocity.

To better understand the effects of a modification of internal time on movement reproduction, we compared the predictions of four different computational models of movement reproduction (see METHODS and Fig. 7) based on previous work (Grasso et al. 1999). Model versions 1 (storage of a velocity profile) and 4 (storage of a distance profile) yield exactly the same predictions (see APPENDIX for proof): velocity is accurately reproduced in all conditions, but reproduced duration and distance are decreased relative to control for MTE and increased for MTR. The equivalence of models 1 and 4 shows that it may be difficult, if not impossible, to determine from the experimental data whether the internally stored spatial variable signifies distance or velocity. Versions 2 and 3 differ in the predicted effects on distance, velocity, and duration reproduction (see Fig. 7).

View larger version (14K): [in this window] [in a new window]   FIG. 7. Simulation of motion reproduction. For a description of the 4 different models of movement reproduction, see METHODS and Fig. 2. A: simulated motion reproduction for models 1 or 4 with t = 0.75 (cf. Table 2). Top plot: distance; bottom plot: velocity. Black: control; dark gray: MTE; light gray: MTR. In the control condition, the model successfully reproduces the motion together with its velocity profile. Mental task conditions are simulated by changing the internal time, which leads to dilation or contraction of the reproduced duration and thereby to longer or shorter reproduced distances. B: predicted gains of reproduced velocity, distance, and duration for all 4 models (see METHODS for description). For models 2 and 3, either distance or duration is accurately reproduced independently of the condition, i.e., gain equals unity. Consequently, reproduced velocity is inaccurate. In contrast, for models 1 and 4, velocity is accurately reproduced, but contraction or dilation of subjective time yields inaccurate distance reproduction.

View this table: [in this window] [in a new window]   TABLE 2. Estimates for the factor t which describes the distortion of subjective time, derived from the two mental task conditions (MTE, MTR) and from distance and duration gains

In summary, the model simulations show that our experimental results are compatible with either 1) storage of a velocity profile with the dual task affecting the internal time base for encoding or retrieval or 2) storage of a distance profile with the dual task affecting both path integration and the temporal aspect of memory.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
Our results show for the first time that a dual task systematically influences movement reproduction. After experiencing motion stimuli without explicit distance information, reproduced duration and distance decreased when a mental task was imposed during stimulus presentation (MTE) and increased when mental activity was imposed during reproduction (MTR). Reproduced velocity was not affected by the concurrent mental task and thus the dual task did not change perceived movement speed.

More than 30 yr ago, it was shown in experiments on the reproduction of time intervals (Burnside 1971; Wilsoncroft and Stone 1975) that mental arithmetic while experiencing a temporal interval led to an underestimation of the experienced duration; when the dual task was performed during reproduction, the experienced duration was overestimated. The results obtained in the present study are not simply a logical extension of this interference effect (for review, see Brown 1997) because the time base for purely perceptual temporal tasks, such as reproduction of time intervals, and the time base for spatial behavior, especially when expressed as motor response, need not necessarily be the same. Storing and reproducing a specific velocity or distance profile may involve timescales that are much smaller than those necessary to experience and reproduce a duration. For example, it was shown that backward counting only minimally influences gait parameters such as walking velocity or stride duration (Beauchet et al. 2005). Because timing in the shorter range is assumed to be "automatic" (Lewis and Miall 2003), movement reproduction may not have been influenced by mental arithmetic.

The interference effect was previously attributed to a disruption of the allocation of attentional resources by the dual task (e.g., Brown 1997) or to increased working memory load (e.g., Fortin and Breton 1995). The allocation of attention and working memory, necessary to perform mental arithmetic, may thus be the reason for the observed effects on movement reproduction. Accordingly, the observed distortion of reproduced movements could be the consequence of allocation of attention rather than of a distortion of an internal representation of time. However, in all our experiments the dual task caused movement reproduction to be changed not only in distance but also in duration. Therefore assuming that allocation of attentional resources and working memory causes the interference effect for time perception, it also affects the internal representation of time for movement reproduction (but not necessarily in an identical fashion).

Previous studies using mental arithmetic during a locomotor task (Takei et al. 1997) or the estimation of self-position after passive rotation (Yardley and Higgins 1998; Yardley et al. 1999) found only that the accuracy of spatial performance decreased. They attributed this decrease in accuracy to attentional demands, but did not further investigate the reasons for this decrease. Specifically, they did not report whether temporal aspects of motion reproduction were affected.

Comparison of experiments

Results of the three different experiments are comparable with respect to the effect of the mental task on duration and distance reproduction. This independence, which demonstrates the generalization of the effect across sensory and motor modalities, is not a trivial finding. In the rotation experiment, subjects had access to angular self-velocity from the visual and vestibular senses. Therefore temporal aspects played a major role when reproducing a given distance. In the treadmill and locomotion experiments, vestibular cues were not useful in determining self-velocity. In the locomotion experiment, visual information was not available at all. Instead, in these two experiments, proprioception and efference copy information were important. Consequently, subjects could have relied solely on spatial cues. In the treadmill experiment, changes in walking velocity could have been produced by changing step length (Mittelstaedt and Mittelstaedt 2001). Because step length and step duration are closely coupled in human walking (e.g., Glasauer et al. 1994), memorization of a sequence of step lengths would be sufficient for accurate reproduction. In the locomotion experiment, in which subjects were asked to walk at constant speed, storing either final distance derived from spatial summation of step lengths or even just a number of steps would have been sufficient. Because mental arithmetic did not significantly change walking velocity, an alternative is that only the duration of the walk was memorized and reproduced. However, the similar results of treadmill, locomotion, and rotation suggest that all depend on the same mechanism, which is independent of sensory or motor modality.

The interdependence of perceptual space and time

Several previous experiments suggested that perceptual space and time are interdependent (for review, see Walsh 2003). In experiments in which subjects carried out tasks in spatially scaled down versions of an environment and had to stop when they estimated 30 min to have passed (DeLong 1981), subjective duration turned out to scale according to the environment. For short-duration whole body rotations, experimental changes in the visually displayed final distance also influenced perceived motion duration (Seemungal et al. 2002); for example, when subjects were shown a shorter distance than they actually had experienced, they also judged the duration of the motion to be shorter. In passive linear self-transport, subjects used counting (i.e., an estimate of time) to estimate distance (Israël et al. 2004). Finally, it was recently shown that the well-known spatial compression around the onset of saccadic eye movements is accompanied by temporal compression of events occurring around that time (Morrone et al. 2005).

This interdependence of perceived space and time is further supported by various findings documenting shared processing of temporal and spatial information, e.g., in the hippocampus (Burgess et al. 2002; Hölscher 2003; Kesner and Hopkins 2001; Redish et al. 2000), cortical motor areas (Macar et al. 2002), the parietal cortex (for review, see Berthoz 1997; Walsh 2003), or the cerebellum (for review, see Gibbon et al. 1997). Furthermore, impaired cognitive time-estimation ability in humans is often coupled with deficits in the motor-output system (Elsinger et al. 2003; Harrington et al. 2004; Perbal et al. 2003) as well as in integrating spatial and temporal attention (Snyder and Chatterjee 2004). However, only few studies have tested patients with cortical lesions on time perception and production tasks. Single- or multiunit recordings in brain regions involved in navigation, such as the hippocampus (O'Keefe 1976; Rolls 1999) or the head-direction cell system (Taube et al. 1990), do not give a conclusive answer as to which aspects of movement may be remembered. Firing of hippocampal place cells, which encode the position of the animal, can be closely related to elapsed time (Redish et al. 2000). Head-direction cells in the lateral mammillary nucleus not only encode the orientation of the animal, but also carry angular velocity signals (Stackman and Taube 1998).

Models of movement reproduction

Analysis of different models of movement reproduction (see METHODS and Figs. 2 and 7) revealed that storage of velocity versus distance profiles are indistinguishable possibilities, given that the controllers are appropriately chosen. Thus the conclusion that accurate reproduction of a velocity profile implies storage of that profile may be wrong. The predictions of two equivalent models of storing velocity or distance profiles were supported by our data, as shown by the model parameter for temporal distortion _t that was extracted from the experimental results (Table 2).

It was previously hypothesized that self-motion is reproduced by memorizing velocity (Berthoz et al. 1995) because the velocity profiles of experienced motions could be faithfully reproduced. A recent study (Israël et al. 2006) in which subjects were explicitly asked to reproduce the plateau velocity and plateau duration of passive self-rotations confirms this notion: subjects accurately reproduced plateau velocity and overall duration, as found in our study. Such a strategy does not require path integration, but only a temporally correct reproduction of the stored velocity profile. If only a velocity profile is stored, it is sufficient to assume that the dual task affects internal representations of duration the same way as it does for time perception. The temporal distortion either when encoding (MTE) or decoding (MTR) would then causally lead to the observed effects on reproduced distance.

If, instead, a distance profile is kept in memory, we have to assume that the dual task affects both the stored duration of the movement and the path integration process itself. Path integration—that is, the transformation of self-velocity signals into distance information—has been documented in lower animals (e.g., ants and spiders), mammals, and humans (for review, see Etienne and Jeffery 2004). Therefore to reproduce a previously experienced self-motion, path integration could be used together with storage of a memory trace of distance information. This hypothesis could explain all of our results without contradicting previous results on movement reproduction (Berthoz et al. 1995; Bremmer and Lappe 1999; Grasso et al. 1999; Israël et al. 1997). One possible scheme for such a model is a discrete path integration process, in which the termination of one path integration event triggers memory access. In other words, once a new distance value is computed, it is stored in spatial working memory. The increment in distance during a path integration event may be computed from velocity and the duration of the event, or may be directly available, e.g., as step length. Decreasing the frequency of path integration events by cognitive load (or allocation of attention) during encoding (MTE) leads to the shortening of stored distances while preserving peak velocity. During reproduction, distance is retrieved from memory and compared with on-line estimates of actual distance, again computed by path integration. Consequently, mental load during reproduction (MTR) also affects reproduced distance, but leaves peak velocity unchanged.

As suggested for other motor tasks (Conditt and Mussa-Ivaldi 1999), a path integration model that triggers memory encoding does not require an explicit internal clock, but it is driven by and relies on undisturbed path integration. If path integration is not required as a result of available distance information, mental activity during encoding would probably not affect the distance of the reproduced motion.

The current results also allow for simultaneous memorization of a distance and a velocity trace, which may be used as additional input for controlling movement reproduction. Combined control of two or more variables, such as distance and velocity, is widely used in technical systems and has been proposed for various motor control tasks, such as postural stabilization (Jeka et al. 2004). For other motor tasks, the use of velocity information is still under debate (De Lussanet et al. 2004). For vestibular stimuli in monkeys, neurons in the parietal cortex (which presumably plays a major role in spatial processing) were previously shown to carry simultaneous representations of distance, velocity, or acceleration (Klam and Graf 2003), suggesting that a combination of kinematic variables may be used for movement control.

Finally, the path integration model predicts that on-line distance estimates in experiments such as rotation should be disturbed by a concurrent mental task. It was shown previously that subjects who performed mental arithmetic during passive whole body rotation could correctly monitor their starting position (Yardley et al. 2002). This can be explained by assuming that outward and backward distance estimates were both affected by the mental task and therefore consistent errors in path integration should cancel out. Preliminary results from our laboratory on path integration during self-rotation suggest that path integration is indeed affected by mental arithmetic, as expected from our model (Glasauer 2006). If this finding is confirmed for locomotion, models of locomotor path integration such as the encoding-error model (Fujita et al. 1993) have to be revised. Our results show that errors arising from the dual task are made not only during encoding, but also during reproduction and suggest that an internal representation of time may play a crucial role for path integration.

In conclusion, concurrent mental arithmetic affected the reproduction of traversed distance and movement duration. The results can be explained by distortion of an internal representation of time. Modeling suggests that the space–time relativity is independent of whether velocity, distance derived by path integration, or both variables are stored in spatial working memory. Thus Donders' (1868) statement that "distraction during the appearance of the stimulus is always punished with prolongation of the process" holds for perception of motion and leads to the observed contraction of both reproduced duration and distance. Moreover, the conclusion posited by Einstein (1938) that "the state of mind of the observer plays a crucial role in the perception of time" can be extended by the perception of space during motion.

	APPENDIX

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
Model equations

Common to all models was memory storage for one of the spatial variables, such as the estimated distance x' derived from path integration or estimated velocity v', assumed to be directly available from sensory information (v' = v). In the following, internally coded variables are denoted by a prime; for example, internally coded distance is denoted by x' as opposed to the physical distance x. During encoding, discrete distance values x'_i or discrete velocity values v'_i were stored in internal memory with a sampling rate of 20 Hz (t = 50 ms) of internal time t'. Thus the memory trace of the motion was a list of distance or velocity values

(A1)

with the subscript index i denoting the ith value in memory. Note that time stamps t'_i were not stored in memory; instead, it was assumed that the temporal distance t between two successive values is known and remains the same for memorization and retrieval. In the following, the subscript i is suppressed and the memory trace is assumed to be continuous (this is justified because t is assumed to be negligibly small compared with stimulus duration).

Internal time t' was computed from physical time t by

(A2)

where _t is a factor to model distortion of internal versus physical time for memorization and retrieval (_t = 1 for no concurrent mental activity; _t = 0.75 if mental activity was required; see Table 2).

MODEL 1: MEMORY TRACE OF VELOCITY. The memory trace s of experienced velocity v_e can thus be written as

(A3)

Expressed in physical time, the memory trace is thus

(A4)

which is a temporally compressed version of the experienced velocity profile v_e(t) for _t < 1.

During reproduction, the memory trace s was retrieved and compared with the percept v'_r of the velocity v_r to generate a motor command m. Such a feedback loop was proposed earlier (Grasso et al. 1999) and, accordingly, an integrative controller was used for model 1

(A5)

where g is the controller gain. The motor command m was then low-pass filtered ( = 0.5 s), simulating inertia and delays, to generate the reproduced motion v_r

(A6)

The resulting velocity v_r was then used as input to the system.

MODELS 2, 3, AND 4: MEMORY TRACE OF DISTANCE. The memory trace s of experienced distance x'_e can be written as

(A7)

Expressed in physical time, it is

(A8)

Distance values x' (experienced or reproduced) were generated from velocity by path integration

(A9)

where _x is a factor to model the distortion of internal time for path integration. During reproduction, the memory trace s was retrieved and compared with the currently estimated value distance x'_r. Then the difference was used to generate a motor command m. Internal distance x' was reset to zero before reproduction started and a proportional controller was used to generate a motor command proportional to the difference between current distance x'_r and stored distance s

(A10)

where g is the controller gain. The reproduced velocity was then computed using Eq. A6.

Alternative models tested included storage of a velocity profile with a proportional instead of an integrative controller during reproduction, storage of two variables (e.g., velocity values together with an explicit time stamp), and distortion of internal time by shutting off memory access and/or path integration for certain time intervals during mental activity. These alternative models yielded predictions similar to those described above.

Equivalence of path integration and velocity models

To show the equivalence of models 1 and 4, we show in the following for mental activity while experiencing motion (MTE) that the motor command for the two models assumes the same values, i.e., that the resulting reproduced motion is the same. For further simplicity, the simulated inertia is ignored by setting = 0 in Eq. A6, yielding v_r(t) = m(t) during reproduction.

In the case of stored velocity values, the motor command (Eq. A5) is

with s(t) = v_e(t/_t) (Eq. A4) and v'_r = m follows that the differential equation for the motor command is

(A11)

This equation fully describes the motor command (and thus the reproduced motion) in the case of model 1.

If position values are stored, the motor command (Eq. A10) is

The memory trace s (Eq. A8) can be inserted, yielding

(A12)

According to Eq. A9, the memory trace x'_e(t/_t) is

(A13)

Differentiating Eq. A12 with respect to time and inserting Eq. A13 yields (with v_r = m)

(A14)

By setting _x = _t = 1, this equation is equivalent to Eq. A11. Thus if the dual tasks equally affect path integration and memory storage, models 1 and 4 are equivalent. The same rationale holds for mental activity during reproduction.

Model predictions

For the predictions of the models, the transfer functions of controller and inertia are ignored for simplicity.

• Models 1 and 4: The reproduced velocity is equal to the stored velocity profile. For MTE, this is a contracted version of the original velocity profile. The reproduced duration t_r is t_r = _t · t, therefore the reproduced distance is x_r = v · t_r/2 = x · _t. For MTR, the stored velocity profile is accurate, but is retrieved using a dilated internal time base, i.e., the reproduced duration is t_r = t/_t. Because the reproduced peak velocity is accurate (see above), x_r = v · t_r/2 = x/_t.
  
• Model 2: The stored distance profile is accurate, but only the durations are changed. Thus for MTE, t_r = _t · t, and for MTR, t_r = t/_t. This leads to inaccurately reproduced velocities: for MTE, v_r = 2 · x/t_r = 2 · v · t/2/t_r = v/_t; for MTR, v_r = v · _t.
  
• Model 3: The reproduced duration is accurate in all cases (t_r = t), but the distance is x_r = _t · x for MTE and x_r = x/_t for MTR. Therefore reproduced velocity depends on task: for MTE, v_r = 2 · x_r/t = 2 · _t · x/t = v · _t; for MTR, v_r = v/_t.
  

Although the models described above assume that a feedback loop controls reproduction, subjects can just as well ignore on-line estimates of current distance or velocity and use only the stored values to generate a motor command for open loop performance. In this case, the prediction for model 1 would not change significantly because the effect of distortion of the internal time on the feedback loop is negligible: it results in only a modest change in feedback gain g in Eq. A5. For the models with stored distance profiles, it is a different matter: open loop performance may render path integration during reproduction unnecessary. Thus the prediction for model 2 also remains unchanged because path integration was assumed to be unaffected anyway. In contrast, the predictions for the MTR condition (but not the MTE condition) of models 3 and 4 differ in the open loop condition. For model 3, mental activity would not have any effect on the MTR condition. For model 4, the predicted MTR result would be equivalent to that of model 2.

	ACKNOWLEDGMENTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
We thank J. Benson for copyediting the manuscript, T. Eggert and J. McIntyre for helpful comments, and A. Kleiser for fruitful discussions. E. Schneider was supported by Deutsche Forschungsgemeinschaft GL342/1.

	FOOTNOTES

 
Deceased October 6, 2000.

The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

^* Address for reprint requests and other correspondence: S. Glasauer, Center for Sensorimotor Research, Klinikum Grosshadern–NRO, 81377 Munich, Germany (E-mail: sglasauer{at}nefo.med.uni-muenchen.de)

	REFERENCES

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
Beauchet O, Dubost V, Herrmann FR, Kressig RW. Stride-to-stride variability while backward counting among healthy young adults. J Neuroeng Rehabil 11: 2–26, 2005.

Berthoz A. Parietal and hippocampal contribution to topokinetic and topographic memory. Philos Trans R Soc Lond B Biol Sci 352: 1437–1448, 1997.[CrossRef][ISI][Medline]

Berthoz A, Israël I, Georges-Francois P, Grasso R, Tsuzuku T. Spatial memory of body linear displacement: what is being stored. Science 269: 95–98, 1995.[Abstract/Free Full Text]

Bremmer F, Lappe M. The use of optical velocities for distance discrimination and reproduction during visually simulated self motion. Exp Brain Res 127: 33–42, 1999.[CrossRef][ISI][Medline]

Brown SW. Attentional resources in timing: interference effects in concurrent temporal and nontemporal working memory tasks. Percept Psychophys 59: 1118–1140, 1997.[ISI][Medline]

Burgess N, Maguire EA, O'Keefe J. The human hippocampus and spatial and episodic memory. Neuron 35: 625–641, 2002.[CrossRef][ISI][Medline]

Burnside W. Judgement of short time intervals while performing mathematical tasks. Percept Psychophys 9: 404–406, 1971.

Conditt MA, Mussa-Ivaldi FA. Central representation of time during motor learning. Proc Natl Acad Sci USA 96: 11625–11630, 1999.[Abstract/Free Full Text]

DeLong AJ. Phenomenological space-time: toward an experiential relativity. Science 213: 681–683, 1981.[Abstract/Free Full Text]

de Lussanet MH, Smeets JB, Brenner E. The quantitative use of velocity information in fast interception. Exp Brain Res 157: 181–196, 2004.[ISI][Medline]

Donders FC. Die Schnelligkeit psychischer Processe (On the speed of mental processes, translated by Kostner WG. Acta Psychol 30: 412–431, 1969.). Arch Anat Physiol wissensch Medicin 6: 657–681, 1868.

Einstein A. On the effects of external sensory input on time dilation. J Exotherm Sci Technol 1, 1938. (Cited after: Mirsky S. Einstein's hot time. Sci Am 287: 102, 2002.)

Elsinger CL, Rao SM, Zimbelman JL, Reynolds NC, Blindauer KA, Hoffmann RG. Neural basis for impaired time reproduction in Parkinson's disease: an fMRI study. J Int Neuropsychol Soc 9: 1088–1098, 2003.[CrossRef][ISI][Medline]

Etienne AS, Jeffery KJ. Path integration in mammals. Hippocampus 14: 180–192, 2004.[CrossRef][ISI][Medline]

Forti