Abstract
The generation of goaldirected movements requires the solution of many difficult computational problems. Among these are transformations from extrinsic to intrinsic reference frames, specifying solution paths, removing underspecification due to excess degrees of freedom and path multiplicity, constraint satisfaction, and error correction. There are no current motorcontrol computational models that address these issues in the context of realistic arm movement with redundant degrees of freedom. In this paper, we conjecture there is a geometric stage between sensory input and physical execution. The geometric stage determines movement trajectories independently of forces. It uses a gradient technique that relies on the metric of the space of postures to resolve endpoint path selection, posturechange specification, error correction, and multiple constraint satisfaction online without preplanning. The model is instantiated in an arm with seven degrees of freedom that moves in threedimensional space. Simulated orientationmatching movements are compared with actual human movement data to assess the validity of several of the model's behavioral predictions.
INTRODUCTION
Voluntary movements such as reaching and grasping are easy to do, but the computations and brain processes involved are hard to explain. Complicated problems of this kind can be simplified by decomposing them into subtasks. Most studies assume a division into sensory and motor stages, but further decomposition is needed to fully explain movement generation and to map it to specific brain areas. In this paper, we propose an additional stage that computes movement paths independent of forces or the speed of movement. We call it the “geometric stage.” The input to the geometric stage consists of a description of the target for movement and of the current posture. Its output represents posture movement paths.
Converting movement paths into actual limb movements requires additional computation. Here we refer to the systems that do these computations collectively as the “motor implementation stage.” The task of this stage is to implement the movement paths specified by the geometric stage. This implies that observed movement paths are independent of speed and are determined by the geometric stage without reference to motor implementation. We leave the issue of the temporal coupling between the geometric and motor stages to the discussion.
A major advantage of introducing a geometric stage is that difficult problems such as excess degrees of freedom, path underdetermination, error correction, and constraint satisfaction can be solved geometrically in a way that is biologically plausible and that simplifies the job of the motor implementation stage. We provide an explicit computational model of the geometric stage. The model's behavior is illustrated by simulating a sevendegreeoffreedom arm making reach to grasp movements in three dimensions. Comparison of this behavior with experimental results demonstrates the biological plausibility of the model.
Previous work
The motivation for introducing a separate geometric stage is the difficulty of solving the problem of under determinacy inherent in specifying the movements of many jointed limbs (Bernstein 1967). Two important sources of movement under determinacy are excess degrees of freedom and path redundancy. If there are more degrees of freedom in the limb than in the space in which it moves, there are infinitely many postures corresponding to each hand location and orientation. There are also infinitely many paths to the same goal whether or not there are excess degrees of freedom. It remains unknown how the brain generates the possible solution paths or how it chooses one path from all the possible ones.
These problems are studied in fields other than biological motor control such as robotics and computer animation, and techniques from these fields have proved useful. The excess degreeoffreedom (df) problem in robotics is often solved using a technique called resolved motion rate control (Whitney 1969). This method assumes that the desired hand trajectory is known. It then uses some optimization method, such as minimizing kinetic energy, to find a corresponding posture trajectory. This solves a subset of the reach to grasp problem, but the need for a hand trajectory and the complexity of kinematics computations make this approach unattractive from a biological point of view. Our model uses a computational strategy that comes from computer animation (Witkin et al. 1987) but modifies it to fit the task of reaching to grasp and to be biologically plausible. This is described in detail later. First we review some previous work.
The most common computational approach in biological motor control has been to consider the problem as analogous to one in classical mechanics. In this formulation, the trajectory that minimizes an energylike quantity must be found. This typically involves integrating a complicated function along all possible trajectories to find the one that is optimal according to some preset criterion. These integrals generally require that the final posture and the elapsed time be known prior to movement, making the computations daunting when the tasks are underdetermined. Because of this, almost all the work has been done with simplified systems having 2 df and moving in a plane, i.e., with no excess degrees of freedom. Under very simplified conditions, the minimizing integral equations can be solved, but in most cases they cannot and many numerical integrations are required to find the minimizing path. Models of this general type such as minimum jerk (Flash and Hogan 1985), minimum torque (Uno et al. 1989), minimum metabolic energy (Alexander 1997), and minimum variance (Harris and Wolpert 1998) have played a useful role in the study of planar arm movement.
Sometimes the problem can be simplified and realistic solutions found for multijointed arms in three dimension (3D). For example,Soechting et al.(1995) address the task of finding a realistic posture for a specified target hand location starting from different initial postures. They use an arm with 4 df, i.e., one extra degree of freedom, and make a series of reasonable assumptions about the geometry and physics of arm movements. The most realistic final postures were found to be the ones that minimize peak work. A search is required, using the simplex method, to find the posture that minimizes peak work. Theses postures are found to correspond quite closely to those found experimentally. This is a dynamic model since it deals with forces. It ignores path generation, and the computations involved do not seem to be biologically plausible.
The idea of separating geometry from dynamics in biological motor control was broached by Hinton (1984) but has not been widely used despite experimental data showing that there are brain areas with forceindependent representations of movement (Kalaska et al. 1990).
Wang and Verriest (1998) do make clever use of the geometry of the arm to solve the same problem as Soechting et al. (1995). Realistic postures are achieved apparently because joint angular velocities are minimized. This model does take joint angle limits into account but requires tests that may not be biologically plausible. The algorithm, like most inverse kinematics algorithms, needs to be supplied with a path. A straightline path is used as a default, but realistically curved arm paths also give good results.
A full review of the vast literature in this field is beyond the scope of this paper. However, we feel that explaining the experimental data on unconstrained arm motions may require a new strategy: one that is able to find complex paths, correct errors online, and satisfy multiple constraints in a biologically plausible way when the arm has redundant degrees of freedom. There is experimental evidence showing that reaching paths remain invariant under moderate changes in such physical variables as speed and loading (Atkeson and Hollerbach 1985; Nishikawa et al. 1999). This suggests that geometry might be a basic component in guiding the movement.
Geometric stage
The geometric stage is a computational module between sensory input and motor output. It transforms information about the goal of a movement, such as an object's location and orientation, into a postural path that brings the hand to the target in the correct configuration to accomplish the goal. For example, grasping an object. The geometric stage requires information about the current posture as well as about the target.
The postural path output by the geometric stage is represented in terms of variables that unambiguously describe postures and can be controlled by the motor implementation stage. It is not yet known just what postural variables are controlled by the motor system. Joint angles and muscle lengths are possibilities. The computational strategy we propose here for the geometric stage will work with any set of postural variables. There are two important features of the geometric stage to keep in mind. First, it computes the geometrical aspects of movement but not the forces required to bring these changes about. And second, it specifies movement paths, independent of the speed of movement. If our conjecture about the geometric stage is correct, then movement paths should be independent of speed, and they should be accurately accounted for by a model of the geometric stage without reference to motor implementation.
The transformation from an extrinsic description of a goal for movement to a postural path is underdetermined. This means that an optimization technique must be used to compute it. The computations used should be biologically plausible as well as able to solve the excess df problem, select realistic postural paths, and respect joint angle limits. Because we don't know the actual variables controlled by the motor system, it should also be possible to parameterize the transformation, using experimental data, so that veridical paths are generated. We have found that a modified form of a gradient descent strategy, originally developed for computer graphics (Witkin et al. 1987), has all these properties so we use gradient descent to model the geometric stage.
Gradient descent requires a function that maps all the variables specifying both the current posture and the target for movement to a positive scalar that decreases as the posture approaches its target. Various disciplines use different names for this kind of function such as cost, error, principle, andenergy function. We will use cost function. The distance remaining between the current hand configuration, and its target is a positive scalar that decreases to zero as the target is approached. We use this distance as the scalar for the cost function in the geometric stage. The gradient of the cost function is a vector whose components specify the relative amount to change each postural variable to get the hand closer to the target position and orientation. The gradient descent procedure for the reachtograsp task consists of repeatedly computing the gradient of the cost function and changing the posture a small amount each time until the hand is at the target in the correct orientation. In this way, it generates a postural path from the initial to the target posture.
We illustrate the gradient technique using an arm with 7 df that does reachtograsp movements in three dimensions. For simplicity, we first use a posture configuration space consisting of a particular set of joint angles. There is no a priori reason to think that the paths generated by this choice of posture configuration space will give realistic movement paths, and, in fact, they do not. Later we show how to generalize the gradient method to respect joint limit constraints and use any set of postural variables. In particular we use parameters obtained from experimental observation to generate realistic paths with the model.
Model: computing the gradient
The model deals with two phases of grasping: transport, moving the hand to the object, and orientation, rotating the hand to match the object axis. Therefore the cost function has to map joint angles to a single value that combines distance to the target with the difference in orientation between hand and target. We first describe using gradient descent for transport.
Consider an arm consisting of constant length segments linked end to end by n flexible joints. One end of the arm is attached to a fixed point in a 3D reference frame and the other end, i.e., the hand, remains unconnected, Fig. 1. LetX, the workspace, be the set of points that can be reached by the hand in 3D space. Let Q, the configuration space, or specifically here, the joint angle space, be an open subset ofR ^{7}. The components of Q are any set of variables, i.e., joint angles, which can specify all armpostures. For our example we use the Euler angle sequence given in Fig. 1 to describe a 7 df arm.
There is a vector valued function, f that maps Qonto X, i.e., every hand location x ∈X can be written as f(q) for at least one q ∈ Q, and every q maps to some x ∈ X
where
A path in Q is a directed line that connects two posturesq
^{initial} andq
^{final}. The path in Xtraced by the hand corresponding to a path in Q is called its image in X. A solution to the transport problem is obtained by moving along the path given by the negative of the gradient of the cost function
Next we consider using gradient descent to reduce the difference in orientation between the hand and the target object. We need to find a single value that decreases with this difference in orientation and is a function of q and the orientation of the target. Our strategy is to first find the matrix that represents a rotation that will align the target with the hand. This rotation can then be represented by a single angle φ about the principal axisn⃗ (Altman 1986). This angle can be used in the cost function. The general scheme is given in the following text.
Solving the orientationmatching problem
Consider three reference frames: one fixed at the shoulder S, one on the hand H, and one on the object O, Fig.2. The relative orientation of these frames can be represented with orthogonal rotation matrices:H for the rotation from shoulder to hand, O from shoulder to object, and R =O ^{−1} H, from object to shoulder to hand, where O ^{−1} =O ^{T} is the rotation from object to shoulder. Note that H depends on all seven joint angles of the posture, q, while O is given as input to the geometric stage. The matrix R represents the orientation difference we want to reduce and depends on both the current posture and the object's orientation.
To get a single scalar value for the orientation difference, we use the fact that for any rotation matrix, such as R, there is a principal axis n⃗ about which a single rotation φ has the same effect as the rotation matrix. A cost function for reachingtograsp can be obtained by combining φ with the distance term for transport derived in the preceding text
It is important to note that each term in Eq. 4 depends on all seven joint angles. This means that the wrist and upper arm arenot treated as separate modals. Posture changes are made in a way that simultaneously changes hand position and orientation. This is important because rotation of any joint angle can potentially change either the location or the orientation of the hand (Jeannerod 1988).
The gradient descent algorithm starts atq ^{initial}, takes a small step in the direction of the negative gradient to a new pointq ^{current}. This procedure is repeated using successive values of q ^{current}until the value of r goes to zero, at which pointx ^{current} equalsx ^{target}, and the hand orientation matches the object's. In practice, computation is stopped whenr is less than some small tolerance. Ideally the steps should be infinitesimal, but the gradient is approximated quite well using moderately large steps. An important feature of the gradient paradigm is that the trajectories obtained in Q provide a complete description of all arm configurations as movement proceeds from initial to final posture.
An important property of Eq. 4 is that r goes to zero if and only if the hand is at the target in the correct orientation. This enables the system to avoid the problem of local optima that can plague gradient descent. If the gradient ofr goes to zero and r is not zero, then a perturbation can be made in the trajectory and gradient descent continued until r is zero. This is not a trivial feature of the exact form of the cost function (Eq. 4 ). Rather it comes from the fact that distance and orientation difference are geometrical properties of the system that go to zero only when the reachtograsp task is complete.
Generalization of the model
Generalizing the model enables it to produce realistic postural paths and obey constraints such as joint angle limits. Generalization is based on the fact that the paths traced out by the gradient depend on how posture configuration space is represented. Thus by transforming configuration space, we can get veridical gradient paths.
Our initial description of the gradient method used joint angles as axis of a Cartesian system to represent posture configuration space. This representation was chosen for illustrative simplicity. We now explore how to transform this Cartesian system to generate arbitrary gradient paths. We can do this either by transforming configuration space and then computing a new gradient or by computing the original gradient and then operating directly on the gradient vector.
These techniques are illustrated by using the gradient method to generate paths for the tip of an arm with only two joint angles moving in a plane. In this system, there is no excess df. However, the paths are underdetermined. We first find the path using a posture configuration space with each joint angle as an axis of a Cartesian coordinate system. This gives curved paths. Then we transform posture configuration space so that the paths are straight lines.
The arm is modeled with two unitlength segments and two joints,q
_{1} at the shoulder andq
_{2} at the elbow. The forward functionx = F(q) in Cartesian configuration space is given by
The translation cost function in this case is
If we want the tip of the arm to always follow straight lines we can, find a transformation, q̃ = G̃ ·q, from the original to new coordinates such that the gradient in the new system gives straight paths in X. The gradient with respect to the new coordinates is
In the following sections, we use the techniques for generalizing the gradient to develop a more realistic model of the geometric stage.
Constraint satisfaction in goaldirected motions
There are many constraints imposed on arm movements. Some limit joint rotation others save effort by biasing the upper arm to remain closer to its resting posture. We implement these constraints using the properties of the gradient operator described in the preceding text.
The general idea is to premultiply the gradient by a positivedefinite matrix, denoted by C instead ofG̃ ^{−1}, to represent the constraints. This yields a new gradient vector, dq ^{C}, guaranteed to decrease the original cost because positivedefinite implies that dq ·C · dq^{t} > 0 so the new gradient vector dq ^{C} is within 90° of the old dq. In addition, dq ^{C} = C · dq = 0 if and only if dq = 0, so all minima of the cost function are still minima, and no new minima are introduced.
Constraints are implemented using diagonal C's. Each nonzero component of C acts on a single joint angle because it affects only one of the components of the gradient vector. The constraints use sums of products of logistic functions that approximate step functions. This allows the logical requirements of the constraint to be implemented using continuous functions.
Simulated constraints
SAVING EFFORT.
The upper arm requires more effort to elevate than the more distal arm segments because it is heavier. Effort can be saved by discriminating against elevating the upper arm away from its resting position and enhancing movement back toward rest. The matrix C in this case has all diagonal components equal 1.0 except for the one associated with the elevation angle of the upper arm. This component is set to
MOVING COMFORTABLY.
The limitations on joint angle flexion can also be expressed using a diagonal matrix, but now each diagonal component is a function that goes to zero when the associated change in joint angle moves it outside its allowed range and stays at one otherwise. To get such a function, we combine a function having partial derivatives of 1.0 in range and 0.0 out of range multiplied by a direction discriminating function of the kind used in the preceding text. An example is
Fitting the model to experimental data
Realistic gradient paths can be obtained by fitting the matrixG̃ ^{−1} to experimental data. The task used to get data to demonstrate this was a reachtograsp movement with a handheld cylinder to match target position and orientation. The resulting sevenjoint angle postural path data were recorded using the Polhemus Fastrak motiontracking system (Torres 2001). Constant components forG̃ ^{−1} were found by optimization (appendix ). In some cases, the paths could be fit with symmetrical G̃ ^{−1}; but in general, this was not the case.
The matrix G̃ ^{−1} is computed across the entire data set for each subject. Table1 lists the mean absolute deviations (in radians) of the difference between the data and the model for each subject. The results of simulating paths with the model using the experimentally derived G̃ ^{−1} are superimposed on the actual data paths from the hand sensor are plotted in Fig. 5 The close correspondence between model and observed paths shows that the model can be parameterized to fit the data with a constant linear transformation of the gradient vector given by an arbitrary joint angle representation.
Comparison of the model with behavioral observations
The arm trajectories computed by the geometric stage will be reflected in observed behavior if, as we conjecture, the motor stage implements these trajectories. This means that various features of the trajectories computed using the gradient paradigm should be comparable to actual arm movements. For example, simulated orientationmatching motions using this method reflect the dependence of final posture on the starting posture and show the coarticulation of hand orientation with arm movement observed in reachtograsp movements (Desmurget et al. 1998; Soechting et al. 1995). Differences in posture paths and hand position and orientation paths due to two initial postures are shown in Fig.6, A–C, respectively (see also Table 2). Simulated paths are superimposed on the actual paths from one of our subjects.
Paths resulting from movements from a single starting posture to targets located in the same position but oriented differently are shown in Fig. 7 (see also Table3). The effect of target orientation on the position hand paths, the orientation hand paths, and the postural paths are shown in Fig. 7, A–C, respectively. They are also superimposed on the motion paths of one subject. A detailed analysis of these experiments will be presented elsewhere.
Error correction
There are many sources of error that can cause deviations from the desired movement path. An important task for any real sensorymotor system is to compensate for these errors. An advantage of the gradient method is that some of these errors are automatically corrected at the geometric stage. We simulate two sources of error due to perturbations. Figure 8 A illustrates the hand position paths resulting from perturbations applied in configuration space. At each time step, two different levels of random noise are used to affect the incremental directional signal output by the gradient. Because the geometric stage continually receives input regarding the current posture and visual goal, the hand moves toward the target irrespective of where the perturbation has put it. The paths are deformed but still reach the target. The real paths observed from this kind of noise would be smoother due to inertia in the physical arm. In Fig. 8 B, we simulate a sudden change in target position during the movement. The model also handles these kinds of online perturbations and achieves the target because it receives visual input related to the current goal position and orientation at each time step (Prablanc and Martin 1992).
Arbitrary trajectories
Real hand movements are often complex with many rapid changes in direction. Examples are paths that avoid obstacles and tasks such as drawing and writing. The gradient method makes it possible to generate these arbitrary trajectories without preplanning. This is because both the x ^{target} and theo ^{target} input to the geometric stage can be updated continually during movement. The path followed by the hand will closely approximate the changing target position and orientation in the input stream. One source of this stream of targets could be the visual system. Generating a stream ofx ^{target} ando ^{target} values from vision is not trivial, but it is a clearly defined problem for which the visual system is well endowed. The key point is that we have separated the task of generating a hand path in an extrinsic 3D reference frame from the job of finding intrinsic values e.g., joint angle changes, that can be used by the motor stage to implement movement. Figure9 gives an example of a complex hand path generated using this method.
DISCUSSION
A major advantage of treating geometry independently of forces is that differential optimization techniques that are relatively simple to implement in a biologically plausible way can be used. This allows incremental changes to be computed as movement proceeds and eliminates the need to compute the whole trajectory ahead of time. This facilitates online error correction, consistent with experimental observations (Castiello et al. 1998;Desmurget and Prablanc 1997; Prablanc and Martin 1992), and simplifies complex tasks that require rapidly changing trajectories such as obstacle avoidance and writing. Models that use a differential approach are not new to motor control (Bullock and Grossberg 1988; Hoff and Arbib 1993). However, we specifically address the problem of redundancy and extend our model to handle the issue of coordinate representations in relation to task constraints.
The gradient method used to implement the geometric stage uses only continuous functions to compute paths, correct errors and enforce constraints. This is biologically significant because simple neural networks can compute these functions. We have trained feed forward networks to do the gradient computation and found that the model neurons in these network have response properties very similar to those of real neurons in brain areas involved in arm movements (Torres 2001). This shows that the gradient method is consistent with current experimental data.
A candidate for the location of the geometric state in the brain is posterior parietal cortex. This area serves as an interface between the sensory and motor systems, and it is known for playing a central role in transforming information from sensory to motor reference frames. In particular, reaching experiments using loads find populations of neurons in this region with forcefree movement representations, in contrast to M1 populations that are generally modulated by force (Kalaska et al. 1990).
The temporal coupling between the geometric stage and motor implementation stage is an important issue for the biological plausibility of the model. The geometric stage can supply the motor stage with a continuous stream of joint angle changes. This could account for rapid responses to unanticipated perturbations during movement. Compensation for feedback delays, when there is tight temporal coupling between stages, requires a feed forward dynamical model. Such a model undoubtedly exists, perhaps in the cerebellum.
It is not necessary for the geometric stage to operate in a tightly coupled mode. It can also generate postural paths without movement taking place. This is done by continuously adding the changes in posture given by the gradient to the current posture, which is what is done in our simulations and is just numerical integration of Eq.1. In this way, partial or complete paths can be computed before any movement starts. This would allow the motor implementation stage to carry out the predictive preparations known to be made before movements start (Gribble 1999). The geometric stage can also be invoked at any point during a preplanned movement to adjust for unexpected perturbations. If the geometric stage is located in a posterior parietal area, this could account for the observation that this part of the brain is involved in correcting movement when target positions change (Desmurget et al. 1999).
Considerations about coupling between stages suggest that there is a need for some kind of buffer memory for path information. Using such a buffer the geometric stage could first provide a path and then monitor ongoing movement to correct for deviations from the path or for changes in target position. Such a buffer would be needed by most other motor control mechanisms and probably should be looked for by physiologists.
The gradient implementation makes the geometric stage a type of error correction mechanism, so it can handle errors online. This occurs as long as the geometric stage receives feedback on the current status of the parameters of interest. There is experimental evidence suggesting the sensorymotor pathway is capable of online error correction (Prablanc and Martin 1992).
Our model does not disambiguate redundancies in the way targets can be grasped. Instead a particular grasp orientation must be given as the orientation input. However, the model is able to deal with all grasp orientations. This was demonstrated by simulation in the paper where we show how postural paths are affected by target orientation. How and where this disambiguation is carried out in the brain is unknown.
The model presented here is oversimplified to provide clarity. However, the general principles are not limited to the simplified form of the model but can be generalized to more realistic situations. The gradient method, in particular, is very flexible. It is not limited to joint angle space but will work with any posture configuration space. So long as the postures expressed in one space can be related to the parameters in another space, it is possible to compute the corresponding transformation rule in the gradient computation. This has biological significance because we do not yet know what space(s) the brain uses to encode postures but can be reasonably confident that the model can be adapted to the representations used in the brain.
The control of speed independently of trajectory is advantageous because it allows either to be changed rapidly during the course of movement. The conjecture that trajectories and speed are specified independently has some experimental support (Atkeson and Hollerbach 1985; Nishikawa et al. 1999). It also raises some interesting questions such as how andwhere speed is specified. One possibility is that speed information is fed to the geometric stage where it is used to determine the step size, or length, of the gradient vector passed to the motor stage. Changing the step size would not significantly affect hand trajectories but could be used to modulate speed.
Reaching for moving and rotating targets fits nicely into this paradigm. Delays can be compensated for by providing target locations that lead actual target positions. There is evidence that the visual system does advance the locations of moving objects because we perceive them ahead of their actual locations (Kowler 1989).
Acknowledgments
We thank E. Todorov, J. Mitchell, and H. C. Gromoll for very useful suggestions and comments.
This work was supported by National Institute of Neurological Disorders and Stroke Grants 3 F31 NS1010905S1 (to E. Torres) and ICS9820 (systems developments to D. Zipser).
Footnotes

Address for reprint requests: E. B. Torres, Div. of Biology, California Institute of Technology, 333 Beckman Labs, M/C 21676, Pasadena, CA 91125 (Email: etorres{at}vis.caltech.edu).
 Copyright © 2002 The American Physiological Society
Appendix
Let Y = M · X, where Y = dq ^{Data} andX = dq ^{Model}.
We define the error E = ∑