INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Humans often interact with and manipulate objects that contain hinges, pivots, flexible attachments, or other non-rigid elements where the movements of the object are different from the movements of the hand manipulating that object. For example, when carrying a bucket of water or a briefcase, the bucket moves relative to the handle in a way that cannot be directly controlled by the hand. Similar examples include balancing a pole on the tip of one's finger, carrying a cup of coffee, or even using a lasso or bullwhip. The goal in these object-manipulation tasks is usually to affect some particular motion of the object being manipulated rather than of the hand itself. When a rigid object is held firmly in the hand, controlling the movement of the object is equivalent to controlling the movement of the hand. For non-rigid objects, however, the motion of the object is governed only indirectly through the interaction of the motions of the hand with the internal dynamics (i.e., degrees of freedom) of the object. Anecdotal evidence suggests that humans and other primates can and do learn to manipulate such dynamically complex objects efficiently but does not answer the question of how this process occurs. It is not yet known if humans develop and use internal representations of the dynamical behavior of non-rigid objects during manipulation. These questions were addressed in the present work.

When planning and executing reaching movements, the human motor control system must account for the mechanical properties of the arm. The results of a number of experiments suggest that this process is accomplished using a detailed "internal model" of arm dynamics. In general, an internal model can be defined as a neural representation of a dynamical system that predicts the consequences of the neural commands acting on that system (Imamizu et al. 2000; Wolpert et al. 1995). In this context, a forward model is one that predicts the next state of the system (i.e., its positions and velocities) based on the current state and a particular motor command, whereas an inverse model is one that estimates the value of the motor command required to move the system into a particular desired state (Imamizu et al. 2000; Miall and Wolpert 1996; Wolpert and Kawato 1998; Wolpert et al. 1995). In the present paper, the term "model-based controller" is used to describe control schemes that may include forward or inverse models or both. It must be emphasized that the CNS could execute movements without making use of any internal model by relying on feedback error-correction alone. Such a strategy could work for executing movements at low speed and with limited interaction forces between the limb and its environment. However, the inherent noise and time delays in sensory feedback signals makes accurate control of rapid or complex movements by feedback alone very difficult (Humphry and Reed 1983; Hogan 1988; Hogan et al. 1987).

Mechanical robots can be programmed to control un-actuated joints (Lynch and Mason 1999; Lynch et al. 2000; Schaal and Atkeson 1994). This demonstrates that such tasks are theoretically possible. When humans learn to manipulate similar objects with un-actuated degrees of freedom and unknown dynamics, they must either guess the dynamics of the object (i.e., form an internal model of those dynamics) based on their experiences or employ some strategy that does not depend on knowledge of object dynamics. The simplest model-independent strategy that subjects might adopt would be to slow their movements to a speed sufficient to allow visual feedback to become a viable source of information for on-line control. This strategy would be applicable to a broad class of statically stable objects (such as the bucket, briefcase, or coffee cup). However, this strategy would also be slow and inefficient and would fail outright when attempting to control statically unstable objects such as an inverted pendulum.

A second possible model-insensitive strategy that subjects might adopt would be to globally increase arm impedance to enforce a specific kinematic trajectory for the hand (Shadmehr and Mussa-Ivaldi 1994) regardless of the forces imposed on the hand by the object. This strategy corresponds to a nearly ideal position control of hand movements and still depends on acquiring a feasible hand trajectory that satisfies the task constraints. Such a trajectory could be found by trial and error without invoking any explicit model of the object's dynamics. Once a feasible hand trajectory was found, increasing arm impedance about this nominal trajectory would allow the motor controller to enforce the same hand trajectory in the presence of a wide range of interface forces encountered at the hand. For certain tasks, such as manipulating a mass on a spring, where subjects must control the mass as well as the hand, each enforced hand trajectory would produce an equivalent object trajectory only for a limited class of "kinematically equivalent" mass-spring objects; i.e., objects with different inertia and stiffness parameters but with the same resonant frequency (see METHODS). The present experiments exploited this feature of mass-spring dynamics to determine if subjects used such a position control strategy when manipulating mass-spring objects. Specifically, the effects of unexpectedly changing object mechanical parameters were examined and experimental results were compared with predictions made by different hypothesized control laws.

Numerous studies have shown that when making point-to-point reaching movements in various external force fields, such as viscous fields (Conditt and Mussa-Ivaldi 1999; Conditt et al. 1997; Gandolfo et al. 1996; Shadmehr and Brashers-Krug 1997; Shadmehr and Moussavi 2000; Shadmehr and Mussa-Ivaldi 1994; Thoroughman and Shadmehr 1999) or Coriolis fields (Cohn et al. 2000; Dizio and Lackner 1995; Lackner and Dizio 1994), humans adapt their motor commands to precisely cancel out the effects of these fields. This adaptation process can take up to several hundred movements to complete, depending on the nature of the field. Similar adaptive responses are found when humans manipulate rigid objects that impose inertial force fields on the arm, although the adaptation typically takes place much faster. Humans can adjust grip force to accommodate the simple inertial loads imposed by typical rigid objects within as little as 135 ms during a single movement (Bock 1990, 1993). When subjects lift objects of unusually high densities, they adapt their responses within fewer than five movements (Gordon et al. 1993). Even for more complex modifications of the arm's inertial properties, adaptation is typically completed within 40-50 movements (Sainburg et al. 1999). Furthermore, precise predictive grip force modulation requires that the feedback forces produced by a manipulated rigid object mimic those of a real rigid object (Blakemore et al. 1998; Witney et al. 2000).

These studies have demonstrated that humans maintain an internal model of the dynamics of their arm that adapts when the physical (i.e., inertial, viscous, and/or Coriolis) properties of their arm are altered. However, in each of these contexts, the kinematics of the system being controlled by the CNS (i.e., the arm) remained unaltered. This mechanical system possessed the same number of degrees of freedom whether these force fields were present or not. It is not known if the adaptive mechanisms discussed in the preceding text can be extended to situations involving manipulation of non-rigid objects where the CNS must learn to control not only the behavior of the arm but also of those additional degrees of freedom introduced by the object. Indeed, a control strategy that is successful for controlling one mechanical system may fail when that system is coupled to a second mechanical system and the resulting combined system being controlled is therefore changed (Hogan et al. 1987). Furthermore, the degrees of freedom of the object are not directly actuated by muscles the way that limb segments are and therefore cannot be controlled directly. Instead, the controller must learn to act through the physics imposed by the object. When an object introduces degrees of freedom external to the body, observation of the states (i.e., positions and velocities) associated only with subject's limb is not generally sufficient to predict the hand-object interaction forces. In such a context, these interaction forces can only be represented as time-dependent forces. However, in experiments that applied strictly time-dependent force fields to the arm, subjects did not learn the time-dependent field but instead attempted to compensate for the forces imposed as if they were dependent on arm state variables (Conditt and Mussa-Ivaldi 1999). This observation that subjects do not construct appropriate representations of time-dependent forces raises the possibility that despite having an accurate and adaptable model of their own arm, humans may not be able to use similar models when dealing with environments or objects that introduce additional state variables external to the body.

The purpose of the present study was to determine if the strategies humans use to successfully manipulate non-rigid objects are consistent with a controller that attempts to predict the dynamical behavior of the object (i.e., one that constructs an internal model of object dynamics). This objective was addressed by having subjects perform a goal-directed reaching task where they learned to manipulate an object with one internal degree of freedom: a mass on a spring. It was hypothesized that subjects would learn to perform this task by implementing a model-dependent control strategy that relied on an internal representation of the dynamics of the mass-spring object being manipulated. The present experiments tested for evidence of the alternative model-independent hypotheses that subjects would slow down to minimize perturbations imposed by the hand-object interaction forces or might globally increase arm impedance to resist those perturbations and enforce a prespecified kinematic plan. The results indicate that subjects did not adopt either of these two alternative model-independent strategies. Although subjects moved more slowly in the object-manipulation task than when reaching with the hand alone, they did exhibit systematic reductions in total movement time, which indicated that they were acquiring the capacity to predictively control the behavior of the object. Subjects also exhibited statistically significant and systematic deviations in hand trajectories when unexpectedly exposed to kinematically equivalent objects (see METHODS). Analysis of subjects' responses to these unexpected changes in object parameters refuted the possibility that they were controlling the object simply by increasing the impedance of the hand about some nominal trajectory. Instead, the present findings were consistent with the hypothesis that humans control non-rigid objects using low-impedance force control laws similar to those used when the arm is subjected to force perturbations (e.g., Shadmehr and Mussa-Ivaldi 1994). These findings expand on previous work on the use of internal model strategies for controlling arm movements in external force fields by demonstrating that similar control strategies might also apply to the control of dynamical systems that extend beyond our own bodies.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Data collection

Six young healthy subjects (3 male and 3 female; age = 30.7 ± 4.6 yr; height = 1.75 ± 0.11 m; weight = 77.6 ± 23.4 kg) with no known neuromuscular disorders affecting their upper extremities participated in the experiment after providing written informed consent. Experiments were performed using a 2-degree-of-freedom robotic manipulandum (Fig. 1A), described in detail elsewhere (Conditt and Mussa-Ivaldi 1999; Conditt et al. 1997; Scheidt et al. 2000). Visual feedback of the relevant task variables was presented on a video monitor mounted above the robot's motors (Fig. 1A). Subjects made one-dimensional 20- to 25-cm reaching movements (Fig. 1B) with their dominant arm while firmly holding the handle of the manipulandum. Reaching movements were directed away from the subject's body along a line passing through the center of rotation of the shoulder and parallel to the sagittal plane (the positive y axis). Each subject's arm was supported by a sling attached to the 8-ft. ceiling and adjusted so that movements were made approximately in the horizontal plane. The manipulandum was programmed to produce forces that simulated a one-dimensional undamped mass on a spring (Fig. 1C) that oscillated in the y direction only. During movements made with the mass-spring object, subjects received on-line real-time visual feedback of both their hand position and the position of the virtual mass (Fig. 1B).

View larger version (24K): [in this window] [in a new window]   Fig. 1. A: representation of a subject using the 2 degree-of-freedom robotic manipulandum. Visual feedback of relevant task variables was displayed on a video monitor mounted above the robot's motors. B: representation of the screen display used in the experiment. Subjects saw the start and goal targets and a moving cursor that represented the current positions of both the hand and the mass (the text shown in this figure was not displayed on the screen). C: schematic diagram of the simulated mass-spring object attached (virtually) to the manipulandum handle (see Eqs. 1 and 2). D: quiver plot of the forces imposed by the mediolateral force field used to ensure that subjects made reasonably straight-line movements along the positive y axis.

The second-order equation of motion for this mass-spring object was

(1)

where M_O was the object mass K_O was the object spring stiffness, and y_O and y_H were the y positions of the object and the hand, respectively. The first-order state space form of this equation was obtained by defining the object state variables as q₁ = y_O and q₂ = _O

(2)

This equation was integrated in real-time to compute the instantaneous position of the object (q₁) for each corresponding position of the hand (y_H). The forces exerted on the hand by the motors were then computed from

(3)

The control input to the system, as defined in Eq. 2, was the y position of the handle; y_H(t). These equations assumed a zero rest length for the spring, such that subjects experienced zero anterior-posterior forces when the handle and object were in the same y position (i.e., F_y = 0 when y_H = y_O). Reaching movements were always initiated with the hand and object both at rest so that on any given trial, subjects could not tell prior to the onset of movement whether the object was present or not. To ensure that reasonably straight-line movements were made along the positive y direction, a mediolateral force field (Fig. 1D) was imposed along the y axis by generating forces acting on the handle in the x direction that mimicked a damped cubic spring (i.e., F_x = A · x³  B; where A = 50 × 10⁶ Nm¹ and B = 50 Nsm¹). This cubic spring produced restoring forces of F_x  0.4 N at lateral displacements of x  ±2 mm, which rose sharply to F_x = 50 N at x = ±10 mm, thus providing a narrow "dead zone" in which subjects could move freely. The forces imposed by this force field were completely orthogonal to the forces imposed by the mass-spring object and were computed completely independently of those forces. After the first few trials, subjects made straight enough movements that this mediolateral force field went effectively unnoticed.

Each subject completed five blocks of 150 movements each. Subjects were instructed to reach from the start location to the target (Fig. 1B) in 0.8 ± 0.2 s. Subjects were given visual feedback on the computer monitor about their movement duration after each trial (too fast vs. just right vs. too slow). The first block of 150 movements was made primarily with the virtual object turned off (i.e., F_y = 0 and q₁ = y_H). This was done to familiarize the subject with the reaching task and the desired timing of the movement. During 10 randomly selected trials of the last 100 trials of the first block, the object was unexpectedly turned on (i.e., q₁ defined from Eq. 2 and F_y from Eq. 3) to assess each subject's initial response to the task. Trial blocks two through five were all completed with the object turned on during all trials as subjects learned the new task. During these trials, subjects were instructed to bring both their hand and the mass to a complete stop within the target zone within the specified time frame (0.8 ± 0.2 s). Success in the task was defined as achieving a y velocity of  < 0.02 ms¹ for both the hand and mass within the target zone for >0.15 s. Subjects were given a maximum time limit of 2.5 s per trial to complete the task successfully, after which trials were terminated automatically. Subjects learned to perform the object-manipulation task with an object of M_O = 3 kg and K_O = 120 Nm¹, which had a resonant frequency of f₀ =  ·   1 Hz.

Equation 2 shows that the dynamics of the mass-spring object were defined exclusively by the ratio of the object's mass to its spring stiffness: K_O/M_O. Thus any "kinematically equivalent" object (i.e., any object with different K_O and M_O values, but the same K_O/M_O ratio and thereby the same resonant frequency) would exhibit the same output kinematics (i.e., [q₁, q₂]^T) when subjected to the same kinematic input, y_H(t). In other words, the same hand trajectory, y_H(t), would produce the same object trajectory, y_O(t). To determine if the subjects in the present experiment were merely "stiffening up" to enforce a prespecified hand trajectory, y_H(t), the last 120 trials of the fifth and final block of 150 movements included 12 randomly selected "catch trials" in which the K_O/M_O = (120 Nm¹/3 kg) object was unexpectedly replaced with either a K_O/M_O = (40 Nm¹/1 kg) object (6 randomly selected trials) or with a K_O/M_O = (200 Nm¹/5 kg) object (6 randomly selected trials).

During all trials, the position of the handle was recorded from position encoders mounted in the manipulandum. Handle velocities were obtained from handle positions using a finite difference formula. These data were used in real time to compute the position and velocity of the virtual mass of the simulated object and to compute the reaction forces that were applied to the hand. The positions and velocities of both the hand and virtual object and the interface forces applied to the hand were saved to disk for further analysis. All kinematic data were sampled at 100 Hz and then low-pass filtered using a zero-lag 6th order Butterworth filter with a cutoff frequency of 10 Hz.

Data analysis

To quantify the relative rates of learning in each subject, plots of total movement time exhibited on each trial versus trial number were constructed. These data were fit with a single exponential function to quantify the trends in these learning curves

(4)

where T_i was the total movement time on trial i (where i = 1, 2, ..., 600 trials),  defined the time constant of adaptation (in number of trials), T_f defined the final adapted movement time (i.e., the movement time that each subject asymptotically approached across the duration of the experiment), and A defined the gain of the adaptation process. Only those trials performed with the 3-kg mass-spring object were included in these calculations. The free parameters of Eq. 4 (A, , and T_f) were fit using a nonlinear least squares procedure (function "nlinfit" in Matlab). One model-independent strategy that subjects might have adopted in the present experiment would be to move slowly enough to minimize oscillations of the mass-spring object, thus allowing the use of on-line feedback control. If subjects adopted such a strategy, then one would not expect to see significant changes in overall movement times beyond the first few trials.

The task requirements were to reach to the target and bring both the hand and object to a stop. Subjects were allowed to choose any specific trajectory they wished to satisfy these requirements. However, the additional un-actuated degree of freedom introduced by the mass-spring object restricted the choice of hand trajectories that subjects could adopt while remaining successful at the task. Because accurate control of the object could only be achieved through the choice of an appropriate hand trajectory, subjects were not expected to return to their preexposure hand kinematics. To examine the nature of the specific hand and object trajectories chosen by each subject, average y direction velocity profiles for the hand, _H(t), in the preadaptation condition and for both the hand (_H) and object (_O) in the postadaptation condition were plotted and compared. For each average velocity profile, 95% confidence intervals (CI) were computed from ensembles of 20 trials. These velocity profiles were used for these comparisons because differences between test conditions were more readily apparent in these curves. Similar comparisons were made between movements of both the hand and object.

As discussed in the INTRODUCTION, another model-independent control strategy that subjects could use would be to globally increase arm impedance to enforce a specific kinematic trajectory for the hand (Shadmehr and Mussa-Ivaldi 1994). The present experiment directly tested for evidence of this possibility. If subjects had merely "stiffened up" to enforce a prespecified hand trajectory, y_H(t), then unexpectedly replacing the 3-kg object with either of the kinematically equivalent 1- or 5-kg objects would not be expected to substantially alter task performance. y-direction hand-velocity profiles, _H(t), obtained from the individual catch trials with the 1- and 5-kg objects were compared with 95% CI of the average hand-velocity profile obtained for the last 20 postadaptation trials for the 3-kg object. Deviations in the catch-trial hand-velocity profiles beyond the 95% CI for the postadaptation velocity profiles constituted statistically significant differences from the acquired postadaptation behavior. Accordingly, any such differences would provide evidence that subjects were not simply globally increasing hand impedance to enforce a specific hand trajectory.

Simulating model-dependent control

The goal of these simulations was to determine if the exhibited catch-trial trajectories were consistent with a controller that relied on a prespecified feed-forward command to explicitly compensate for the forces imposed on the hand by the mass-spring object. This hypothesis was compared with the alternative possibility that the exhibited catch-trial trajectories could have been produced by a controller that relied on globally increasing hand impedance. The existence of such a predictive feed-forward control command would be consistent with a control strategy in which subjects were computing that command based on some prediction of the object's subsequent behavior (i.e., an internal model). The simplified one-dimensional model was determined to be adequate for obtaining a first-order approximation of this effect while maintaining computational efficiency.

Equation 1 takes as its control input the position of the hand as a function of time, y_H(t). If the arm was an ideal position controller (i.e., one with infinite impedance), subjects could execute any desired y_H(t) with infinite precision regardless of the interface forces experienced at the hand. Because humans are not infinite-impedance position controllers, some degree of kinematic deviation was expected even if subjects were globally increasing their arm impedance. In previous experiments where subjects adapted their reaching movements to viscous force fields (Shadmehr and Mussa-Ivaldi 1994), subjects did not use such a strategy but instead exhibited behavior consistent with a low-impedance controller that incorporated an explicit internal model of arm dynamics. To determine if the behavior exhibited by the subjects in the present experiment was similarly consistent with this form of controller, catch-trial movements were simulated for each subject using a simplified one-dimensional model of the arm plus object (Fig. 2A). The arm was modeled as a single point mass attached to the hand (M_H) that was controlled by a single time-varying control force, C(t). The second-order differential equations of motion for this double mass-spring system were

(5)

Defining a new set of state variables for this arm-plus-object model as [q₁  q₂  q₃  q₄]^T = [y_O  O  y_H  H]^T, the equivalent first-order state space equations for this system were

(6)

Following the same form as the controller postulated by Shadmehr and Mussa-Ivaldi (1994; their Eq. 9), the control input for this model, C(t), was defined to include three terms. The first of these terms was a feed-forward term, (t), defined to be an estimate of the inverse dynamics of the arm-plus-object system. The remaining two terms were error-feedback terms that accounted for the passive stiffness and viscosity of antagonist muscles and their associated segmental reflexes (Shadmehr and Mussa-Ivaldi 1994). Thus C(t) was defined as

(7)

where

(8)

defined the estimate of the inverse of the arm-plus-object dynamics (i.e., the internal model) for movements made along the desired trajectory, [y^*_H(t)  y^*_O(t)]. It should be noted that this system could be equivalently modeled by including the passive stiffness and damping in the plant (Fig. 2B) rather than in the feedback terms of the controller. The resulting equations for this equivalent system were

(9)

where the control input then became

(10)

A straightforward substitution of terms demonstrates that the system defined by Eq. 6 with C(t) defined as in Eq. 7 is identically equivalent to the system defined by Eq. 9 with U(t) defined as in Eq. 10. The resulting system (Eqs. 9 and 10) was a linear time-invariant (LTI) system subject to a single time-varying input, U(t). Note that both formulations of the model control system contain an estimate of the inverse dynamics of the arm-plus-object system, (t), in their control input.

View larger version (23K): [in this window] [in a new window]   Fig. 2. A: schematic representation of the simulated 2-mass arm-plus-object model (Eqs. 5 and 6). The arm was modeled as a single point pass located at the hand (MH). The forces applied to the arm mass were produced by a controller consisting of an approximation of the inverse model of the arm-plus-object dynamics, (t) (Eq. 8): plus feedback terms necessary to provide stability (Shadmehr and Mussa-Ivaldi 1994). B: schematic diagram of a mathematically equivalent model in which the passive stiffness and damping of arm are included in the plant, rather than the feedback control loops (Eqs. 9 and 10). In this model, the control input is a single feed-forward command. Both models are mathematically identical and both include an estimate of the arm-plus-object inverse dynamics (i.e., a predictive internal model) in their feed-forward control inputs.

For these simulations, arm masses (M_H) were estimated for each subject as a fixed percentage (5%) of their total body mass based on standard anthropometric data (Winter 1990; Table 3.1). Arm mass values ranged from 2.8 to 5.8 kg across the six subjects. Gomi and Kawato (1996, 1997) estimated endpoint (i.e., measured at the hand) stiffness ellipses for subjects making normal (i.e., no objects or perturbing force fields) anterior-posterior reaching movements. The anterior-posterior components of these stiffness ellipses shown in their results (their Fig. 4) had magnitudes ranging from approximately K_H = 100 Nm¹ at proximal arm configurations to more than K_H = 1,000 Nm¹ at the distal positions (Gomi and Kawato 1996, 1997). Studies examining arm stiffness properties during similar movement paradigms have reported either similar values (Bennett 1994; Bennett et al. 1992; Burdet et al. 2000), somewhat larger values (Xu and Hollerbach 1999), or somewhat smaller values (Mah 2001). When subjects actively co-contract to resist perturbations, postural endpoint stiffness can increase by as much as five to six times these values (Gomi and Osu 1998; Perrault et al. 2001). Estimates of hand viscosity ellipses have also been reported for postural tasks by Tsuji et al. (1995). The anterior-posterior components of the viscosity ellipses shown in their results (their Fig. 11) had magnitudes ranging from approximately B_H = 5 Ns·m¹ for proximal arm configurations to upwards of B_H = 25 Ns·m¹ for distal arm configurations (Tsuji et al. 1995).

In the present study, reaching movements were initiated from a posture in which the hand was located proximal to the body. Therefore values of K_H = 100 Nm¹ and B_H = 5 Ns·m¹ were used to simulate the catch-trial behavior that would be expected if subjects had not globally increased their arm impedance. Likewise, values of K_H = 500 Nm¹ and B_H = 25 Ns·m¹ were used to simulate the catch-trial behavior that would be expected if subjects had co-contracted to enforce a position control strategy. Although in vivo arm stiffness and damping in humans varies as a function of arm posture (Gomi and Kawato 1996, 1997; Tsuji et al. 1995), it was assumed that this range of variation would be adequately captured within the bounds established by these values. It should be emphasized that the present experiments were neither designed nor intended to estimate actual hand stiffness and viscosity values during the mass-spring-object-manipulation task. These simulations were only used to provide a first-order estimate of the likelihood that the kinematic trajectories exhibited during the catch trials were consistent with a predictive control strategy that compensated for the interface forces imposed on the hand by the mass-spring object rather than with a strategy that relied on globally increasing hand impedance to resist those perturbations.

Desired hand and object trajectories, [y^*_H(t)  y^*_O(t)], were defined for each subject from the average trajectories obtained from the last 20 postadaptation (i.e., M_O = 3 kg; K_O = 120 Nm¹) trials prior to the introduction of the catch trials (i.e., trials 11-30 in the 5th block of movements). To obtain sufficiently smooth hand trajectories from the inverse dynamics (Eq. 8), these average trajectories were first interpolated from 100 to 1,000 Hz and then zero-lag low-pass filtered at a cutoff frequency of 10 Hz. Controller trajectories, U(t), were then computed directly from these data using Eq. 10 and the original learned object parameters (M_O = 3 kg and K_O = 120 Nm¹). These control trajectories were then used as the input to the forward dynamic simulation (Eq. 9), where the K_O/M_O = (120 Nm¹/3 kg) learned object parameters were replaced with either the K_O/M_O = (40 Nm¹/1 kg) or the K_O/M_O = (200 Nm¹/5 kg) catch-trial object parameters. Simulations were generated in Matlab using the "lsim" command. Simulated catch-trial hand-velocity profiles were compared directly to experimental catch-trial hand-velocity profiles to determine if the kinematics exhibited by the subjects in the present study were consistent with the hypothesis of a controller that incorporated a predictive model of the arm-plus-object dynamics.

To quantify the quality of the fits between each of the two model realizations and the data, an "index of relative fit" (IRF) was derived. First, the root-mean-square (rms) difference between the average catch-trial hand-velocity profiles, (t) (where  ·  denotes the average over n = 6 catch trials), and the model-predicted hand-velocity profile, (t), was computed. This difference measured how well each predicted catch-trial hand trajectory fit the actual catch-trial hand trajectories. If the model produced a perfect fit, this difference would be zero. The rms difference between the average catch-trial hand-velocity profiles, (t), and the average postadaptation hand-velocity profiles, (t) (where  ·  denotes the average over the last 20 postadaptation trials), was also computed. This difference quantified the average magnitude of the hand-velocity deviations exhibited by each subject during the catch trials and was used to control for the fact that because different subjects exhibited different postadaptation kinematics, they were also expected to exhibit somewhat different catch-trial behavior. The IRF was then defined as the ratio of these two rms differences

(11)

Thus IRF  0 if the predicted catch-trial hand movements closely matched the actual catch-trial hand movements, whereas IRF  1 if the predicted catch-trial hand movements closely matched the postadaptation hand movements. IRF values much more than 1 indicated poor model fit (i.e., either that the model predicted catch-trial deviations much bigger than those actually observed or in the opposite direction to those observed). For each subject and catch-trial condition (1 vs. 5 kg), the value of IRF was computed over the first 500 ms of the movements for both the low- and high-impedance realizations of the arm-plus-object model. Differences in IRF between low- and high-impedance model predictions were tested using a one-sided paired t-test for the IRF data computed for both 1- and 5-kg catch trials, where the data were paired within subjects.

Simulating model-independent control

It is reasonable to ask whether or not a control strategy that did not include an explicit representation of the inverse dynamics of the arm-plus-object system, (t), might also be capable of reproducing the catch-trial behavior observed in the present experiment. To test this possibility, simulations of the arm-plus-object system (Eq. 6) were also performed by assuming an alternative controller, C_A(t), that did not include an internal model [i.e., (t) = 0], but instead relied solely on setting the overall gains (K_H and B_H) for the net hand stiffness and viscosity feedback terms

(12)

This controller translated into an equivalent control input that could be applied to the equivalent LTI model with time-varying input (Eq. 9)

(13)

Simulations based on this alternative controller were generated in a manner similar to the process described in the preceding text. Control inputs, U_A(t), were computed directly from the interpolated and filtered postadaptation data for each subject using Eq. 13. To determine what feedback gains (K_H and B_H) were required to allow this controller to adequately reproduce the original learned behavior, these controller trajectories were first used to drive the forward dynamic simulation (Eq. 9) with the learned object parameters (i.e., M_O = 3 kg and K_O = 120 Nm¹). A wide range of arm impedance gains (i.e., 100 Nm¹ < K_H < 1,000 Nm¹ and 5 Ns·m¹ < B_H < 50 Ns·m¹) were tested. To simulate the corresponding catch-trial behaviors, the learned object parameters were replaced with either set of catch-trial object parameters, and the forward dynamic simulations were repeated. Simulated postadaptation and catch-trial hand-velocity profiles based on this alternative controller were compared directly to experimental hand-velocity profiles and to the catch-trial trajectories predicted by the original internal model controller (Eq. 7). These comparisons were made to determine if the kinematics exhibited by the subjects in the present study could have been generated by a controller that relied solely on adjusting hand stiffness and damping rather than trying to learn a predictive model of the arm-plus-object dynamics.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Subjects exhibited learning behavior

The trial-to-trial total movement times (T_i) computed from the object-manipulation trials demonstrated that most subjects learned to perform the task with reasonable skill (Fig. 3). However, time constants of learning (Eq. 3) varied widely between subjects (18 <  < 442 trials). Because the mass-spring object itself contained no damping, completing the reaching task successfully in any amount of time required subjects to effectively dampen out the oscillations of the object. While no subject successfully completed the task within the desired time limit (0.8 ± 0.2 s) during the duration of the experiment, all subjects exhibited lower movement times with practice. This suggests that these reductions in movement time were an expression of learning.

View larger version (31K): [in this window] [in a new window]   Fig. 3. Movement time results for each of the 6 subjects tested. Dashed horizontal lines indicate desired movement time (1.0 s.). Thick gray lines represent exponential (Eq. 3) curve fits to the data. Equations for each curve fit are shown at the bottom of each subplot. Although learning rates varied, all subjects exhibited some capacity to improve their overall performance over the course of the experiment.

These improvements in total movement time might indicate that subjects were learning the appropriate feedback gains to use in a control scheme based exclusively on feedback error correction. Alternatively, they might reflect the fact that subjects were first acquiring and then gradually improving their capacity to control the behavior of the object by learning an internal model of object behavior. In either case, these results demonstrate that these subjects did indeed learn (by some means) to improve their performance on the designated task.

Postadaptation kinematics

Preexposure (no object) reaching movements performed by the subjects in the present study exhibited the bell-shaped velocity profiles (Fig. 4A) typically associated with such unperturbed reaching movements (Flash and Hogan 1985; Morasso 1981). As anticipated, unexpectedly exposing subjects to the mass-spring object caused severe disruption of these reaching kinematics (Fig. 4B). Figure 4B shows that the object oscillated at its resonant frequency of 1 Hz and that these oscillations were slowly damped out. Because the object was unanticipated in these trials, this damping behavior was likely the result of passive mechanisms related to mechanical properties of the arm and/or to the fact that subjects were not able to hold onto the handle with a perfectly rigid grasp.

View larger version (23K): [in this window] [in a new window]   Fig. 4. Reaching kinematics for a typical subject (subject 4) prior to learning. A: y-direction velocity profiles for the hand (H) during preexposure trials exhibited the expected bell-shaped profile. B: y-direction velocity profiles for both the hand (H) and object (O) during initial exposure trials exhibited substantial disruption of the movement pattern. For each plot, dark lines represent averages and shaded areas represent ±95% confidence intervals, each computed from ten trials. Dashed vertical lines indicate the start of movement (t = 0.0 s) and the targeted movement time (t = 1.0 s). All subjects exhibited similar patterns of behavior on initial exposure to the mass-spring object.

By the end of the learning phase of the experiment, each subject had adopted a new pattern of movement (Fig. 5). Although different subjects exhibited different hand-velocity profiles, these movement patterns were quite consistent for each subject, as demonstrated by the low trial-to-trial variability. In contrast to previous studies where subjects adapted to force fields that depended only on arm state variables (e.g., (Lackner and Dizio 1994; Shadmehr and Mussa-Ivaldi 1994), no subject in the present study regained their original unperturbed kinematics. For every subject in the present experiment, final velocity profiles of the hand and object were both substantially different from their unperturbed hand kinematics. Although velocity profiles for the object were roughly bell-shaped for most subjects, they exhibited consistently lower peak velocities and were extended over a longer time frame compared with these subjects' preexposure hand movements. Hand-velocity profiles were distinctly non-bell-shaped in all subjects and exhibited different shapes for different subjects (Fig. 5). These changes in hand kinematics provide evidence of predictive control. They demonstrate that subjects were able to predict that if they moved their hand along one specific trajectory, then the object would follow another specific trajectory, the combination of these two trajectories leading to successful completion of the task goal. If subjects had been unable to learn these new appropriate hand trajectories, then they would not have been able to improve their overall task performance and thereby their overall movement times (Fig. 3).

View larger version (25K): [in this window] [in a new window]   Fig. 5. Postadaptation reaching kinematics for all 6 subjects. y-direction velocity profiles for both the hand (H) and object (O) are shown. In each plot, dark lines represent averages and shaded areas represent ±95% confidence intervals, each computed from the last 20 object-manipulation trials prior to the introduction of the catch trials. Dashed vertical lines indicate the start of movement (t = 0.0 s) and the targeted movement time (t = 1.0 s). All subjects exhibited postadaptation kinematics that were dramatically different from both their preexposure and initial-exposure kinematics (compare Fig. 4). These movement patterns were highly consistent for each subject, as indicated by the narrow 95% confidence intervals (CI).

Subjects did not use high-impedance control

Although Fig. 5 demonstrates that each subject adopted a control strategy that relied on moving their hand along a specific trajectory to successfully complete the task, such movements could be executed in a number of ways. One way would be to globally increase arm impedance to neutralize the effects of the interface forces generated at the hand by the mass-spring object. An alternative approach would be for subjects to learn the temporal sequence of hand-object interface forces and adjust their feed-forward commands to compensate for them directly. The present study directly tested for evidence of these two possibilities by introducing kinematically equivalent objects (see METHODS) during randomly selected catch trials toward the end of the learning phase of the experiment. If subjects had globally increased their hand stiffness enough to enforce a specific hand trajectory regardless of the perturbing forces encountered at the hand, then only minimal deviations would be expected during these catch trials. However, all subjects exhibited statistically significant and substantial deviations from their acquired postadaptation hand paths when making reaching movements with both the 1- and 5-kg "catch-trial" objects (Fig. 6). Catch trials with the 1-kg object always exhibited initially greater velocities during the first half of the movements (t  approximately 500-750 ms). With the exception of subject 1, catch trials with the 5-kg object always exhibited initially lower velocities during the first half of the movements.

View larger version (35K): [in this window] [in a new window]   Fig. 6. y-direction velocity profiles for the hand (H) after learning the 3-kg object (95% CI shown in black) and for the individual catch trials with the 1 kg (light gray lines) and 5 kg (dark gray lines) objects for all subjects. Dashed vertical lines indicate the start of movement (t = 0.0 s) and the targeted movement time (t = 1.0 s). All subjects exhibited statistically significant deviations in their hand kinematics when exposed to either catch-trial object.

Model-based control predicts exhibited behavior

To obtain first-order estimates of the kinematic deviations that would be expected during the catch trials if subjects had adopted a model-based control strategy, simulations were performed with a simplified one-dimensional model of the arm-plus-object (Fig. 2; Eqs. 4-6). The controller for this system was assumed to incorporate an explicit model of the arm-plus-object dynamics (Eqs. 7 and 8). By varying the magnitudes of arm stiffness (K_H) and viscosity (B_H) in this model, the relative effects of these parameters on the resulting catch-trial kinematics were assessed. Simulated catch trials were generated for each subject (e.g., Fig. 7. A and B) based on values of arm stiffness and viscosity that were in the low to moderate range of those reported in the literature (i.e., 100 Nm¹ < K_H < 500 Nm¹ and 5 Ns·m¹ < B_H < 25 Ns · m¹) (Burdet et al. 2000; Gomi and Kawato 1997; Tsuji et al. 1995). However, only when the arm model was assumed to incorporate low impedance ("LI model" in Fig. 7; K_H = 100 Nm¹ and B_H = 5 Ns · m¹), did the model provide reasonably good predictions of the catch-trial behavior exhibited by these subjects. When the arm model was assumed to incorporate higher impedance ("HI model" in Fig. 7; K_H = 500 Nm¹ and B_H = 25 Ns · m¹), the model predicted much smaller kinematic deviations than observed experimentally. Still higher values of arm stiffness and damping lead to even smaller predicted deviations, as expected from the definition of these kinematically equivalent objects.

View larger version (28K): [in this window] [in a new window]   Fig. 7. A: example hand-velocity data (shaded areas) from a typical subject (subject 6) for 1-kg catch-trial data and predicted catch trials (black lines) based on the arm-plus-object model with either low impedance (LI) or high-impedance (HI) assumptions about the feedback gains in the internal model based controller (Eq. 7). B: example data (shaded areas) and associated model predictions (black lines) from the same subject for 5-kg catch trials. Light gray shaded areas in both subplots represent ±95% CI for postadaptation trials (n = 20), whereas dark gray shaded areas represent ±95% CI for catch trials (n = 6). Note that in both cases, the LI controller better predicts that actual catch-trial behavior, whereas the HI controller predicts much smaller deviations in hand kinematics. Similar results were obtained for all subjects. C: index of relative fit (IRF) scores for model predictions based on both LI and HI controllers for all subjects tested. In nearly all cases, the LI controller predictions more closely fit the actual catch-trial behavior (smaller IRF), whereas the HI controller predictions more closely fit the postadaptation behavior (IRF  1).

These findings were confirmed by the IRF scores computed over the first 500 ms of these movements for all subjects (Fig. 7C). In nearly all cases, the low-impedance (LI) model resulted in lower IRF scores (mean IRF = 0.460 excluding the 5-kg catch trials for subject 1), whereas IRF scores for the high-impedance (HI) model were almost always much closer to 1 (mean IRF = 0.896). These differences were highly statistically significant by the one-sided paired t-test (T = 5.43; P = 1.03 × 10⁴). This finding indicates that the low-impedance predictions were a better fit to the actual catch trials, whereas the high-impedance model predicted catch-trial kinematics that would have more closely fit the postadaptation kinematics. The primary exception to these findings was in the 5-kg catch trials for subject 1 where, as shown in Fig. 6, these catch-trial trajectories exhibited deviations in the direction opposite to (i.e., higher rather than lower velocities) those of the other five subjects and of the model predictions. While it is not clear what exactly caused this subject to exhibit this particular behavior for these catch trials, it is clear that even these catch-trial kinematics could not have been generated by a control strategy that relied on globally increased arm impedance.

Feedback alone cannot predict exhibited behavior

The simulations shown in Fig. 7 indicate that the experimental catch-trial results were well predicted by a model system driven by a low-impedance controller that included an internal representation of the arm-plus-object dynamics. However, this finding does not preclude the possibility that the postadaptation and catch-trial trajectories exhibited by the subjects in the present study might be equally well predicted by a controller that did not include such an internal model. To address this question, simulations of the arm-plus-object system were performed using an alternative controller that relied solely on adjusting the overall gains for hand stiffness and viscosity for control (Eqs. 12 and 13). Two primary findings were evident from these simulations. In the first step of the modeling process, forward simulations of the postadaptation trajectories were generated by the arm-plus-object model using the original learned object properties (M_O = 3 kg and K_O = 120 Nm¹) and the alternative controller. These simulations clearly demonstrated that the arm-plus-object model driven by the alternative controller was incapable of reproducing the original desired postadaptation behavior unless very large feedback gains (K_H  1000 Nm¹ and B_H  50 Ns·m¹) were employed (Fig. 8A). However, the adoption of such large feedback gains leads to predictions of only minimal kinematic deviations during the catch trials (Fig. 8B). Thus if subjects had been using a control strategy based on setting the appropriate gains for a feedback controller that incorporated hand stiffness and viscosity alone, they could not have exhibited the catch-trial kinematics that they did in the present experiment.

View larger version (29K): [in this window] [in a new window]   Fig. 8. A: example postadaptation hand-velocity data (shaded areas) from a typical subject (Subject #6) and predicted postadaptation catch trials (black lines) based on an arm-plus-object model