Prevailing views on how we time the interception of a moving object assume that the visual inputs are informationally sufficient to estimate the time-to-contact from the object's kinematics. Here we present evidence in favor of a different view: the brain makes the best estimate about target motion based on measured kinematics and an a priori guess about the causes of motion. According to this theory, a predictive model is used to extrapolate time-to-contact from expected dynamics (kinetics). We projected a virtual target moving vertically downward on a wide screen with different randomized laws of motion. In the first series of experiments, subjects were asked to intercept this target by punching a real ball that fell hidden behind the screen and arrived in synchrony with the visual target. Subjects systematically timed their motor responses consistent with the assumption of gravity effects on an object's mass, even when the visual target did not accelerate. With training, the gravity model was not switched off but adapted to nonaccelerating targets by shifting the time of motor activation. In the second series of experiments, there was no real ball falling behind the screen. Instead the subjects were required to intercept the visual target by clicking a mousebutton. In this case, subjects timed their responses consistent with the assumption of uniform motion in the absence of forces, even when the target actually accelerated. Overall, the results are in accord with the theory that motor responses evoked by visual kinematics are modulated by a prior of the target dynamics. The prior appears surprisingly resistant to modifications based on performance errors.
Interception requires precise timing, whether to catch or hit a ball, to jump on land or dive in water. How do we estimate time-to-contact (TTC) with an approaching object? Most extant theories posit that all relevant information is present in the visual stimulus without resorting to high-level computations or internal representations (Gibson 1966; Lee and Reddish 1981; Regan and Gray 2000; Tresilian 1999). However, there are limitations in the visual system that raise questions about the general validity of these theories (Lacquaniti et al. 1993a). Although several monocular and binocular cues can contribute to TTC estimates, they generally provide only first-order approximations related to the object's distance and velocity (Lee and Reddish 1981; Regan and Gray 2000; Rushton and Wann 1999; Tresilian 1995). In fact neurophysiology shows that cortical neurons specialized for visual motion processing accurately encode target velocity and direction, but contain only partial information about acceleration (Perrone and Thiele 2001). Moreover, psychophysics indicates that the human visual system poorly estimates image accelerations over short viewing periods (Bozzi 1959; Brouwer et al. 2002; Regan et al. 1986; Runeson 1975; Werkhoven et al. 1992). Accordingly, at the motor level arbitrary accelerations generally are not taken into account in timing interceptions (Bootsma et al. 1997; Engel and Soechting 2000; Gottsdanker et al. 1961; Lee et al. 1997; Port et al. 1997). Another problem is that, because of sensorimotor delays, interceptive actions must be anticipatory, based on information available ≥100 –200 ms before contact. Extrapolation of visual motion information into the future can lead to severe spatial and temporal misjudgments (Tresilian 1995).
To overcome these problems, it has been hypothesized that on-line visual information is combined with an a priori representation (called reference model) of target dynamics that depends on the context of the specific visuomotor task (Lacquaniti et al. 1993a). In other words, the brain could make the best guess about visible or hidden causes of object's motion at any time, and use a predictive model to extrapolate TTC from expected dynamics (kinetics) rather than from measured kinematics. In Newtonian dynamics a free particle moves with constant velocity, unless acted on by an accelerative force. If it is difficult to discriminate arbitrary accelerations visually, gravitational acceleration might be internalized based on lifelong exposure (Hubbard 1995; Lacquaniti et al. 1993a; Tresilian 1993; von Hofsten and Lee 1994; Watson et al. 1992). Visual signals about target position and velocity could be combined with an internal estimate of earth's gravity, yielding a second-order dynamic model of TTC for interception of objects whose motion is expected to be affected by gravity (Lacquaniti and Maioli 1989; Lacquaniti et al. 1993a; McIntyre et al. 2001). In a similar vein, it has been suggested that targeted movements are coupled to an intrinsic second-order guide similar to gravity (Lee 1998). The idea that an internal model of gravity can be used to disambiguate sensory information has also been developed in the field of vestibular physiology. Vestibular otoliths (as all linear accelerometers) measure gravitoinertial forces, but cannot distinguish linear accelerations from gravity. Based on eye movement responses to combinations of tilts and rotations, it has been proposed that the brain combines canal signals with otolith signals and uses an internal model of gravity to estimate linear accelerations and disambiguate tilt from translation of the head (Hess and Angelaki 1999; Merfeld et al. 1999).
Here we show that the timing of responses aimed at intercepting a visual target is very different depending on the dynamic context. Virtual targets moving vertically downward with different laws of motion were intercepted with a timing consistent with the assumption of uniform motion in the absence of forces. In contrast, when the same virtual targets were used to punch a hidden real ball arriving in synchrony, response timing was consistent with the assumption of gravity effects on an object's mass. With training, the gravity model was not switched off, but adapted to nonaccelerating targets by shifting the time for motor activation. The results indicate that responses evoked by low-level visual inputs change with the context established by dynamic predictive models from higher-level processing.
In total 20 healthy subjects (12 women and 8 men, 29 ± 5 yr old, mean ± SD) participated in the study receiving modest monetary compensation. All but one subject participated in one experiment only. The subjects were right-handed (as assessed by a short questionnaire based on the Edinburgh scale), had normal vision or vision that was corrected for normal, and were naïve to the purpose of the experiments (except subject G.B.). Experiments were approved by the Institutional Review Board and conformed to the Declaration of Helsinki on the use of human subjects in research.
First we describe the setup common to both the punching and the button-press experiments. Subjects sat on a chair placed in front of a wide vertical screen attached to the ceiling (Fig. 1). The screen (Video Spectra 1.5-gain) had a pearlescent frontal projection surface (3.94 m wide, 2.13 m high). The rear part of the screen was made of black, nontransparent material, preventing the view of any object through it. The back of the chair supported the head and torso of the subjects without preventing their motion. The vertical inclination of the back was adjusted at about 40° so as to allow a comfortable view of the screen with the subjects' eyes located at a horizontal distance of 0.5 m from the screen, 1.82 m below the top. In that position subjects could easily reach below and beyond the lower border of the screen by protracting their arm forward. Images were generated by a PC and displayed on the screen by a BARCO Graphics 808s (1,024 × 768 pixels, 85-Hz refresh frequency). A black 9-cm-wide box was constantly displayed at the top of the screen against a white background. In each trial a red 9-cm-diameter target sphere moved vertically downward; it was displayed initially as emerging progressively from within the start box at a predefined speed and acceleration (see Experimental procedures and protocols) and then shifted with the prescribed law of motion to finally disappear progressively at the lower border of the screen. The visual angle subtended by the target increased from 0.8°, when it was first fully visible, to 10.3° when it passed at eye level. The rest of the room was dimly illuminated, the only light sources being the BARCO and a PC monitor. Next we describe the setup specific for each set of experiments.
punching experiments. A precision laminated electromagnet (G.W. Linsk, Clifton Springs, NY), attached behind the screen, held an inflated rubber ball (diameter, 9 cm; weight, 70 g). This was a soft ball bought in a children's toyshop, and never caused any discomfort to the subjects even after several punching trials. A small steel washer (outer diameter, 1.4 cm) was attached to the ball surface to ensure magnetic contact with the electromagnet. The electromagnet release accuracy was better than 1 ms. Spatial trajectories of the center of the virtual sphere on the screen and of the real ball were aligned in the vertical plane orthogonal to the seat of the chair, offset by 7.5 cm in depth. The release time of the real ball and the start time of virtual motion were set by the computer so as to result in synchronous arrival at an interception point located below the lower border of the screen. The time of synchronous arrival will be denoted interception time in the following. Synchronous timing of virtual and real motions was determined by means of an optic calibration procedure that involved 1-kHz shuttered videorecordings axially centered on the interception point. The timing error was always less than the refresh rate of image display for all tested laws of simulated motion. Because the release of the ball from the electromagnet and its subsequent fall were noiseless and invisible, the visual information about the virtual motion provided the only perceptually available TTC information before the ball contact with the arm.
button-press experiments. Here there was no real ball falling behind the screen. Subjects sat on the chair as before, grabbing a computer mouse in the right hand. To provide visual feedback of late interceptions, the screen surface was prolonged downward with a strip 0.9 m long and 0.33 m wide. A black start box and a blue end box were constantly displayed at the top and bottom of the regular screen, respectively, the red target-sphere moving downward from the start to the end box.
punching experiments. 3D motion of selected body points was recorded at 200 Hz by means of the Optotrak 3020 system (Northern Digital, Waterloo, Ontario; ±3SD accuracy better than 0.2 mm for x, y, z coordinates). Three infrared emitting markers were attached to the skin overlying the shoulder, elbow, and wrist joint on the right arm. Three additional markers were fixed to the lower border of the screen to determine the screen plane. A miniature accelerometer (Isotron Endevco), fixed on the wrist, was sampled at 1 kHz. Electromyographic (EMG) activity of the clavicular portions of pectoralis (PE) and trapezius (TP), anterior deltoid (AD), posterior deltoid (PD), biceps (BC), triceps (TR), and flexor carpi radialis (FC) was recorded by means of surface electrodes (SensorMedics 650414; diameter of detection surface, 2 mm) in bipolar configuration (interelectrode distance, 1 cm). EMG signals were preamplified at the recording site using a pair of matched FET operational amplifiers to reduce noise, and then differentially amplified (120-dB CMRR), high-pass filtered (20 Hz), low-pass filtered (200-Hz, 4-pole Bessel), and sampled at 1 kHz. Sampling of kinematic and EMG data was synchronized with trial start.
button-press experiments. The analog voltage signal from the mouse button was A/D converted and sampled at 1 kHz synchronized with trial start.
Experimental procedures and protocols
Before the experiment, subjects received general instructions and familiarized with the setup in front of the screen (they could not see behind it). They were informed that in each trial a visual target sphere would move vertically downward from the start box at different randomized speeds.
punching experiments. Subjects were asked to intercept the target just below the lower border of the screen by punching the real soft ball that would arrive there at the same time as the visual target. They kept a free relaxed posture between trials. Before trial initiation, an alert signal instructed the subjects to look at the box on the top of the screen and to recoil their arm in the starting posture: with the adducted shoulder, the upper arm was roughly vertical, the forearm horizontal, the wrist midpronated, the hand and fingers clenched in a fist. At the beginning of the trial, an audible attention signal sounded and, after a random delay ranging from 1.2 to 1.7 s, the virtual sphere exited from the black box. The movement required to punch the ball involved the forward protraction of the hand by about 20 cm, along a roughly straight-ahead, horizontal direction. Data were acquired for 5 s, starting 0.4 s before sphere exit. The intertrial interval was about 14 s. Total duration of each session was about 1 h 20 min, with 10 min of rest allowed halfway.
In any given trial the virtual sphere moved from the starting box with randomly assorted initial speeds (v0 = 0.7, 1, 1.5, 2.5, or 4.5 m s–1) at a constant acceleration (either A = 9.81 ms–2 = 1g or A = 0 = 0g). The randomization procedure always avoided identical combinations of v0 and A in consecutive trials. Subjects were assigned to one of 3 groups (with 5 subjects in each group). In each group, subjects were exposed to identical sequences of trials.
Subjects of the first 2 groups performed 2 different experiments 24 h apart. Each experiment consisted of 4 identical blocks of 55 trials. In each block, there were 50 test trials and 5 pseudorandomly interspersed “catch trials” with unexpectedly altered kinematics. For test trials, v0 was pseudorandomly assigned to one of 5 values, and A was assigned to one fixed value for day 1 experiment (either 0 or 1g depending on the protocol) and the other fixed value on day 2. Thus A = 1g on day 1 and A = 0 on day 2 for group 1 (1g–0g protocol), whereas A = 0 on day 1 and A = 1g on day 2 for group 2 (0g–1g protocol). Each catch trial had the same v0 as that of the previous test trial (catch 1), but different A (A = 0 if test trial was 1g and A = 1g if test trial was 0g), except for the catch trials of day 1 in group 1 that were all trials with A = 1g, v0 = 0 m s–1. Each experiment therefore consisted of 40 repetitions per 5 test conditions, and 4 repetitions per 5 catch conditions. The sequence of target v0 was identical in day 1 and day 2. The first through the fifth catch trials occurred in the sequence as the 7th, 14th, 27th, 34th, and 54th trials, respectively. Catch trials were used to reveal the presence of aftereffects attributed to adaptation (Shadmehr and Mussa-Ivaldi 1994).
Subjects of group 3 (Random protocol) performed a 1-day experiment involving 200 test trials without catch trials. For each trial, both v0 and A were randomized (20 repetitions per 10 conditions).
button-press experiments. Subjects sat in the same posture as for punching, and kept the arm still on an armrest with the computer mouse grabbed in the right hand. Before trial initiation, an alert signal instructed the subjects to look at the box on the top of the screen. At the beginning of the trial, an audible attention signal sounded and, after a random delay ranging from 1.2 to 1.7 s, the virtual sphere exited from the black box. Subjects were asked to click the mouse button with the index finger so as to intercept the descending target just as it reached the center of the end box (interception point). If they intercepted in the allotted time (±15 ms relative to the exact interception time), the target sphere “exploded” blue with a pleasing reward sound. If they intercepted too early or too late, the target sphere was flashed red at the point of incorrect interception. Six subjects performed a 1-day experiment involving only test trials whose v0 and A were randomized (as in Random above). One subject had participated in a punching experiment (Random protocol) about 1 yr earlier.
General data analysis
punching experiments. Rectified EMG and raw kinematic data were numerically low-pass filtered (bi-directional, 20-Hz cutoff, 2nd-order Butterworth filter) to eliminate impact artifacts. Instantaneous tangential velocity of the hand was computed by numerically differentiating the recorded x, y, z coordinates of the wrist marker. We measured motor timing based on the time of occurrence of the first positive peak of hand acceleration because this parameter was very reliably derived from the data (see Fig. 3 for examples of acceleration records). The time of this acceleration peak relative to the interception time will be denoted TTC. We also estimated the onset time (the time when the acceleration first reached 10% of the first positive peak), and the time of zero-crossing of hand acceleration. Movement duration was defined as the interval between onset time and zero-crossing time.
Nominally subjects had a timing tolerance of about ±15 ms around the interception time to contact the falling ball with the finger knuckles. However, to compute the interception rate we allowed a wider margin of error because subjects could effectively contact the ball also with other parts of the hand (such as the top of the fist). Thus to determine whether the ball was intercepted in each trial, the time of any measurable contact between the ball and the limb was determined by the occurrence of specific high-frequency oscillations in the accelerometer signal (the raw signal was filtered with a bidirectional 25-Hz high-pass Butterworth filter). The ball was considered intercepted if the contact oscillations occurred between the first positive peak of acceleration and the first negative peak. Typically, this time interval was about 120 ms independent of the conditions, implying that we considered as intercepted those trials in which the contact occurred within ±60 ms relative to the interception time. The interception rate was then computed as the percentage of all intercepted trials over the total number of trials of a given condition. We also verified that subjects intercepted close to the expected interception point by computing the vertical spatial coordinate of the hand at the time of contact oscillations. This position was not significantly different from the expected value in any subject. On average it was 0.7 ± 5.7 cm (mean ± SD, across all trials and subjects) lower than the expected value, equivalent to a time lag of 1.0 ± 11.4 ms relative to the interception time. As a measure of the spatial error, we computed the anteroposterior horizontal spatial coordinate of the hand at the interception time and compared with the corresponding coordinate of the screen markers.
An exponential function was fitted to the series of repetitions of TTC values for each condition (v0, A) to characterize the rate at which subjects adapted. The function has 3 free parameters: offset b0, gain b1, and time constant b2 (1) where TTCi is the TTC value on repetition i.
button-press experiments. TTC was measured as the time of the button press relative to the interception time. To calculate interception rate we used 2 different time windows relative to the interception time: ±15 or ± 60 ms. The first window corresponds to the time interval used to provide positive or negative feedback of success to the subjects during the experiments. The second window corresponds to the time interval that was used to compute the interception rate in the punching experiments. For both time windows, the interception rate was derived as the percentage of trials whose TTC was included in the window.
statistics. Differences between conditions were assessed using ANOVA followed by Bonferroni–Dunn tests, and using Wilcoxon signed ranks or t-statistics (P < 0.05, level).
Few trials (0.2% of all trials) were excluded from the analysis attributed to the presence of artifacts or lack of subject's attention during the trial as marked in the experiment notebook.
Models of interception timing
Figure 2 defines the critical timing variables for both punching and button-press experiments. The instantaneous height h(t) of the virtual target above the interception point is given by the standard equation of motion (2) where h0 and v0 are the initial height and velocity, respectively, and
To intercept the target, subjects implicitly solve for h(t) = 0 at t = expected interception time (IT). IT does not necessarily coincide with the true interception time (flight duration of the target) because the subject's estimate of target motion might be erroneous. We hypothesize that a motor response is programmed at a given lead time LT relative to IT, necessary to compensate for signal transmission delays. In general, LT will depend on neural and mechanical delays. Thus LT = PT + MT, where PT represents the processing time and MT represents the movement time (or the time epoch of the movement segment under consideration). For causal systems, the future motion of the target is unknown and can be extrapolated based only on current estimates of h(t) and its time derivatives. The order of the model used for motion extrapolation refers to the time derivatives that appear in the describing equation. Thus a zero-order model includes only position, a first-order model also includes velocity, and a second-order model also includes acceleration. We have considered models that incorporate visual information about target distance and velocity because it is known that these variables can be measured accurately by the visual system (Orban et al. 1984; Regan 1997). Visual acceleration generally is poorly measured (Regan et al. 1986), and we have considered a second-order model that incorporates an internalized estimate of gravitational acceleration (Lacquaniti et al. 1993a). Our models assume that the response is programmed after a given visual or internalized variable has reached a critical threshold value at time TT = IT – LT. This assumption is customary in models of fast interceptions (Lee et al. 1983; Michaels et al. 2001; Port et al. 1997; van Donkelaar et al. 1992; Wann 1996). A critical test for threshold-based models is that the threshold value should remain invariant across a number of initial conditions; otherwise, these models would furnish only an ad hoc fitting to a data subset.
In the following, we provide an expression for the time of the motor response from trial onset predicted by each model (RT*, asterisk denotes prediction) as a function of the initial conditions of target motion (h0, v0, A) and of the model parameters (PT, and hTT or τ or λ, depending on the model; see following text). In addition, we provide an expression for predicted by each model as a function of . For these models, is linearly related to , with an intercept (c0) and slope (c1) that differ according to the model and the initial conditions, as follows (3) All expressions are derived from Eq. 2 in a straightforward manner.
distance model. It postulates that the response is programmed when the target has reached a certain threshold height hTT at t = TT (van Donkelaar et al. 1992; Wann 1996). The distance model provides a zero-order approximation of target motion because it incorporates information about current distance of the target, but ignores velocity and acceleration.
In the case of 0g motion (4) and c0 = hTT and c1 = –PT in Eq. 3.
In the case of 1g motion (5) and c0 = hTT + 0.5gPT2 and c1 = –PT.
τ-model. It postulates that the response is programmed when IT – TT has decreased below a certain threshold time τ (Lee and Reddish 1981; Lee et al. 1983). By definition (6) The τ-model provides a first-order approximation of target motion because it incorporates information about target distance and velocity, but ignores acceleration. Thus it represents a 0g model.
In the case of 0g motion (7) and c0 = 0 and c1 = (τ – PT).
In the case of 1g motion (8) and c0 = (0.5PT – t)gPT and c1 = (τ – PT).
1g model. It postulates that the response is programmed when IT – TT has decreased below a certain threshold time λ (Lacquaniti and Maioli 1989; Lacquaniti et al. 1993a; McIntyre et al. 2001). By definition (9) where ĝ is an internalized estimate of gravitational acceleration. In the following we assume that ĝ = 1g (the acceleration of earth's gravity at sea level). The λ-model provides a second-order approximation of target motion, because it incorporates information about target distance and velocity, and always assumes that the target is accelerated by gravity. Thus it represents a 1g model.
In the case of 0g motion (10) and c0 = 0.5gλ2 and c1 = (λ – PT).
In the case of 1g motion (11) and c0 = 0.5g(λ – PT)2 and c1 = (λ – PT).
The experimental values of RT (either the times of the positive peak of hand acceleration or the times of the button press relative to trial onset) were (least-squares) fitted using Eqs. 4–11 to obtain estimates of the model parameters (PT, and hTT or τ or λ). To assess the robustness of the estimates, we followed different fitting procedures. First, the results for 1g targets and 0g targets were fitted simultaneously, using the global model obtained by coupling Eqs. 4 and 5 for the distance model, Eqs. 7 and 8 for the τ-model, and Eqs. 10 and 11 for the λ-model. Second, the results for 1g targets were fitted separately from those of 0g targets. This procedure could be used for both 1g and 0g targets in the case of the distance model, only 1g targets for the τ-model, and only 0g targets for the λ-model.
An additional analysis was performed for the punching experiments based on the height–velocity (h–v) phase plots introduced by Port et al. (1997). Here the experimental values of hRT were linearly regressed against the corresponding values of vRT using Eq. 3 to obtain estimates of the intercept (c0) and slope (c1). The results for 1g targets were fitted separately from those for 0g targets. Several different temporal landmarks were used for RT, based on the time when hand acceleration first reached a given percentage level (10, 25, 50, 75, and 100%) of its positive peak. This analysis allows discrimination among the various models based on the quadrant where the intercept–slope values (determined from the experimental data) fall.
Intercepting 0g versus 1g targets by punching a ball
In all experiments a virtual target sphere moved vertically downward on the projection screen with a randomized law of motion. In the first series of experiments, subjects were asked to intercept this target by punching a real soft ball (of the same size as the virtual one) that fell hidden behind the screen. Because the virtual target arrived at the interception point at the same time (interception time) as the real ball, subjects had to estimate the TTC of the former to punch the latter correctly.
On the first day, one group (group 1) of subjects was presented with visual targets moving with 1g acceleration and randomly assorted initial velocities (v0). All subjects punched the ball by propelling the hand forward with a ballistic, stereotypical movement. The responses were correctly time-locked to the interception time from the first attempt and varied little in successive trials. The 1st, 2nd, and 11th presentations of the same condition from one such experiment are superimposed in Fig. 3 (left, in red, green, and blue, respectively). Acceleration and EMG profiles are essentially indistinguishable among these repetitions.
On average, hand acceleration reached the first positive peak at –64 ± 31 ms (mean ± SD across all 1g test trials of 5 subjects, n = 997). Negative (positive) values indicate times before (after) the interception time. The zero-crossing of hand acceleration occurred at –2 ± 24 ms, indicating that subjects generated maximum momentum to punch the incoming ball at the right time. Limb propulsion was determined by a burst of muscle activity of shoulder flexors and elbow extensors, and it was subsequently braked by shoulder extensors and elbow flexors.
The same group of subjects was trained 24 h later with targets at 0g acceleration and the same randomly assorted v0 as the previous day. Limb movement and arm muscles activation were similar to those of the previous day, but their timing was very different (Fig. 3, right). When subjects were first exposed to each 0g condition, their responses started too early and the hand passed the interception point well before the interception time. On average, hand acceleration reached the first positive peak at –158 ± 83 ms (mean ± SD across the first presentations of each 0g test trials of 5 subjects, n = 25), 94 ms earlier than for the 1g targets of the previous day. The zero-crossing of acceleration occurred at –93 ± 78 ms. As a measure of the spatial error, we computed the difference between the anteroposterior horizontal spatial coordinate of the hand at the interception time and the corresponding mean value for the 1g targets of the previous day. On average, in the first 0g trial the hand was 7.3 ± 6.6 cm more anterior than in the 1g trials. Consequently, it was hit by the ball at the metacarpus or carpus. With training, the early inappropriate responses were progressively delayed (compare the 1st, 2nd, and 11th presentations in Fig. 3, right), but they remained premature as compared with 1g responses.
The timing but not the waveform of hand acceleration depended on the law of target motion. All presentations of the same condition from one experiment are superimposed in Fig. 4. Acceleration profiles for 1g trials are essentially superimposable for most trials. The positive component of the acceleration profiles for 0g trials is very similar to that of 1g trials, staggered in time as a function of presentation order. The negative component (braking the movement) tends to be slightly greater for 0g trials than that for 1g trials. Several spatiotemporal landmarks of the motor responses systematically covaried across trials and conditions. Thus the time of onset, first positive peak, and zero-crossing of hand acceleration were always strongly correlated (P < 0.001) between each other. Moreover, movement duration varied little across all conditions (on average, its SD was 32 ± 13 ms). In the following, we mainly concentrate on the TTC of the acceleration peak, considered as the motor correlate of the time-tocontact of the target estimated by the subject.
The effect of training was tested using 3 different protocols. Two groups of subjects were trained with 1g test trials on one day (the first and second day in group 1 and group 2, respectively) and 0g test trials on the other day. Group 3, instead, performed both 1g and 0g trials within the same session, the value of target acceleration being randomized along with the value of initial velocity (Random protocol). The mean TTC values (±SE, n = 5), averaged across all subjects of the latter group, are plotted as a function of repetition number for each initial velocity in Fig. 5 (red and blue points correspond to 1g and 0g trials, respectively). The ensemble average of the data of all groups is plotted in Fig. 6 along with the best-fitting surfaces.
The general trend of the results was independent of the order with which 1g and 0g trials were practiced. In all groups, TTC of 1g trials varied little with either repetition or v0. Two-factor ANOVAs showed no significant effect of repetition in any group (P = 0.3), a small but significant effect of v0 in groups 1 and 2 (P < 0.05), and a nonsignificant effect in group 3 (P = 0.09). The trend was that TTC for v0 = 4.5 m s–1 was slightly shorter than that for the other v0. The mean range of variation of TTC across v0 was 16 ± 8 ms (mean ± SD over all groups).
The TTC values of 0g test trials were significantly (t-test, P < 0.001) longer than those of 1g test trials: on average, by 114 ± 23 ms (means ± SE across all v0 conditions and subjects) in the first repetition and by 56 ± 9 ms over the last 5 repetitions. Two-factor ANOVAs showed that TTC of 0g trials shortened (the time of acceleration peak shifted closer to contact) significantly with both increasing v0 (P < 0.001) and repetition (P < 0.001) in all groups. These changes were substantial, resulting in a mean range of variation of 237 ± 44 ms. In general, TTC shortened rapidly with repetition (within the first 4–5 repetitions) in the conditions with low v0 (0.7, 1 m s–1). Exponential fitting (Eq. 1) of these changes yielded a time constant of 1.8 ± 0.6 (mean ± SE).
We can rule out the hypothesis that subjects synchronized their movements to the start of target motion by assuming that the real ball was dropped at the same time as the virtual one. This hypothesis would predict a constant time of the motor response (RT) relative to trial onset. This hypothesis was rejected because 3-factor ANOVAs showed a highly significant effect of acceleration (P < 0.001), velocity (P < 0.001), and repetition (P < 0.005) in all groups.
In contrast with the clear dependency of the timing (TTC) of arm movements on the law of target motion, their maximum speed showed no systematic relation with target motion. Thus peak tangential velocity of the hand (vP, occurring close to nominal interception time; see above) did not depend significantly on target acceleration or v0 in 2 groups, whereas it was slightly but significantly (P < 0.001) higher for 0g targets than for 1g targets in group 1 (on average, vP = 3.54 ± 0.96 and 3.25 ± 0.61 m s–1 for 0g and 1g targets, respectively; mean ± SD, n = 1,000). However, the overall range of variation of vP across conditions (v0 and acceleration changes) was always <13% of the mean vP.
Consistent with the accurate timing, the interception rate of 1g trials was high from the outset and improved only slightly with practice. On average, it was 80 ± 16% (mean ± SD) in the first repetition and 92 ± 10% in the last 5 repetitions of each condition for all subjects and groups. Instead, the interception rate of 0g trials increased substantially with practice but was always significantly (t-test, P < 0.001) smaller than that of 1g trials. On average, it was 20 ± 10% in the first 0g repetition and 59 ± 13% in the last five 0g repetitions.
Occasional (9% incidence) trials with unexpectedly altered kinematics, termed “catch trials,” were interspersed among the test trials of protocols 1 and 2 to verify for the presence of aftereffects attributed to adaptation. The 1g catch trials with v0 = 0 m s–1 unexpectedly presented during immersive training with 1g test trials with random v0 (day 1 in group 1) did not cause aftereffects. TTC of these catch trials did not differ significantly from that of the preceding trial.
In contrast, the unexpected occurrence of 1g catch trials during immersive 0g training did cause significant (P < 0.001) aftereffects in groups 1 and 2 (Fig. 7). The size of the aftereffects did not change significantly as a function of the time of occurrence during 0g training, probably because of the fast time constant of adaptation (the first 1g catch trial occurred in the sequence after most adaptation had occurred). Two-factor ANOVAs showed no significant effect of repetition in any group and a slight significant effect of v0 (P = 0.003) in group 1 but not in group 2. When present, the effect of v0 was small (on average, 6 ms TTC change per 1 m s–1 velocity change). On average, the interception rate for these 1g catch trials was 37 ± 21%, significantly (t-test, P < 0.001) lower than the interception rate of 1g test trials.
We also compared the TTC of the 0g test trial after a 1g catch trial (catch + 1) with the TTC of the previous 0g test trial with the same v0 (catch – 1). The difference between (catch + 1) and (catch – 1) did not deviate significantly from the difference between (catch – 1) and (catch – 2).
Modeling the data
In theory, subjects might time their responses on the basis of different kinds of TTC estimate of the visual target: exact estimates or approximations of different order. Exact estimates would lead to a correct motor timing for both 1g and 0g targets. Zero-order approximations of the distance model incorporate information about target distance, but ignore target velocity and acceleration. The application of this model would overestimate the TTC of targets with shorter flight duration and underestimate the TTC of targets with longer flight duration. First-order approximations of the τ-model incorporate information about target distance and velocity, but ignore target acceleration. Because the τ-model is equivalent to a 0g model, it would lead to correct TTC estimates for 0g targets, but would overestimate TTC of 1g targets. Finally, second-order approximations based on an internalized estimate of gravitational acceleration incorporate information about target distance and velocity, and always assume that the target is accelerated by gravity. Therefore the 1g model correctly estimates TTC of 1g targets, but underestimates TTC of 0g targets.
The finding that the motor timing was premature and the success rate was low in 0g test trial compared with that of 1g test trials indicates that the TTC of 0g targets was underestimated, whereas the TTC of 1g targets was estimated correctly. This finding apparently refutes both exact as well as zero-order and first-order mechanisms and is compatible instead with a second-order mechanism based on the 1g model. However, in line of principle a difference in timing of the motor responses to either the 0g targets or the 1g targets could result from the application of a low-order TTC estimate but with an offset of the motor responses. For instance, subjects could program the response when the target has traveled a given distance but the response would occur only after a given delay attributed to neural and mechanical transmission times. The consequences of a delay are not intuitive and need to be assessed quantitatively.
To account for transmission delays, we fitted Eqs. 4–11 to the experimental RT values and obtained the values of RT* and TTC* (= flight duration – RT*) predicted by each model. In a first analysis, the results for all 1g targets and for 0g targets were fitted simultaneously, under the assumption that the model parameters are invariant across all tested initial conditions. Moreover, a lower boundary value of 100 ms was imposed on PT, τ, and λ in the fitting procedure, to avoid unrealistic results (minimum delays for visuomanual responses are of that order of magnitude; Carlton 1981). The values of TTC* are plotted as a function of v0, along with the mean experimental values of TTC for the first repetition of each condition in group 2, in Fig. 8. The results were essentially identical for all groups.
The 1g model (thick curve in Fig. 8) reproduced well the trend of the experimental data for all speeds (v0) and accelerations (1g and 0g). The best fit was obtained with a processing time (PT) of 139 ms and a threshold time (λ) of 202 ms. The linear regression of all predicted TTC* values versus the corresponding experimental TTC values was highly significant (mean r2 = 0.87 across groups, P < 0.001), with a slope (0.9) and intercept (–18 ms) that did not differ significantly from unity and zero, respectively (as expected from a homogeneously good fit).
The distance model (thin curve) reproduced the experimental data for the lower speed (v0 ≤ 1.5 m s–1) 0g targets, but grossly overestimated (on average, by 106 ms) both the values of TTC experimentally obtained for higher speed 0g targets, as well as the values for all 1g targets. The best fit was obtained with a PT of 100 ms and a threshold height (hTT) of 0.39 m. The linear regression of all predicted TTC* values versus the corresponding experimental TTC values was significant (mean r2 = 0.86 across groups), but the slope (1.5) and the intercept (133 ms) were significantly (P < 0.05) greater than unity and zero, respectively.
The τ-model (dash-dotted curve) reproduced the experimental TTC values for 1g targets but did not reproduce those for 0g targets (mean r2 = 0.47 across groups). The best fit was obtained with a processing time (PT) of 267 ms and a threshold time (τ) of 476 ms.
Next, we fitted the 1g targets separately from the 0g targets. The results did not change markedly with the 1g model (mean r2 = 0.76, PT = 157 ms, λ = 208 ms). The distance model reproduced the mean TTC of 1g data better than before, but overestimated or underestimated the TTC of targets with shorter or longer flight durations, respectively (mean r = –0.31); its fit of 0g data did not change appreciably (mean r2 = 0.76, PT = 100 ms, hTT = 0.36). The τ-model fitted the data separately worse than before (mean r2 = 0.08).
Then we removed the boundary values in the fitting procedure. The results for both the 1g model and the τ-model remained identical to those obtained under constrained fitting. Instead, the unconstrained best-fitting of the distance model yielded negative values incompatible with a causal system (PT = –121 ms and hTT = –0.40 m).
The second analysis is related to the h–v phase plots introduced by Port et al. (1997). The models we are considering predict that the height of the target above the interception point at a given time after trial onset is linearly related to the corresponding value of target velocity, for all v0 at a fixed acceleration (either 1g or 0g). However, the intercept and slope of the h–v relationship differ according to the model and the initial conditions (see Models of interception timing in methods). Figure 9 shows the results of simulating the performance of the different models. Each curve in the figure corresponds to the locus of all pairs of intercept–slope values obtained by varying model parameters (see figure legend for details). Red and blue curves correspond to 1g and 0g targets, respectively. This graphic analysis allows discrimination among the various models on the basis of the quadrant where the intercept–slope values fall.
The distance model predicts h–v relationships with positive intercept and negative slope, for both 1g and 0g targets (orange quadrant in Fig. 9). Instead, the τ-model predicts a positive slope for all targets, a zero intercept for 0g targets, and a negative intercept for 1g targets (green quadrant). Finally, the 1g model predicts positive intercept and positive slope for all targets (yellow quadrant).
Predicted curves were compared with the experimental data in the following way. The experimental values of hRT for the first repetition of each condition were linearly regressed against the corresponding values of vRT using Eq. 2 to obtain estimates of the intercept and slope. Here the response time RT was varied in steps based on the time of occurrence of different percentage levels (10, 25, 50, 75, and 100%) of the positive peak of hand acceleration. This allowed generalization of the results to different temporal landmarks of the response. The results for 1g targets were fitted separately from those for 0g targets.
Experimental values of hRT generally were linearly related to those of vRT (mean r = 0.91). The estimated pairs of intercept–slope values (black circles in the yellow quadrant of Fig. 9) consistently satisfied the predictions of the 1g model. Each circle corresponds to a different RT. With decreasing values of RT, the experimental points shift progressively higher (arrows) along the curves predicted by the 1g model. Notice the good agreement between the path described by the experimental points and the theoretical curves.
The 1g model also predicts that the pairs of intercept–slope values for 0g targets becomes identical to that for 1g targets when TTC = λ (azure circle). Thus the time threshold λ can be estimated independently from 3 parameters of the model: 1) the TTC at which the intercept–slope curve for 0g targets intersects that for 1g targets, 2) the corresponding intercept value (c0 = 0.5gλ2), and 3) the slope (c1 = λ). In the first repetition, the 3 estimates of λ were 214, 210, and 202 ms, respectively. Notice the good agreement of the different estimates between each other and with those derived above. Training with 0g targets resulted in a systematic decrement of λ-values. This issue is addressed in the next section.
Adaptation of 1g model to repeated presentations of 0g trials
We previously showed that the responses to 0g trials were premature and that with training the performance improved but there was a consistent residual timing error (Figs. 4 and 5). Here we show that this behavior is compatible with the persistent use of the 1g model throughout practice, with a progressive adaptation of the threshold time λ.
Application of the 1g model leads to predictable differences in timing between 1g and 0g targets. A 1g target arrives at the time predicted by the 1g model, but a 0g target arrives later than expected (Fig. 10 left). Thus motor activity triggered by the 1g model starts earlier than necessary at 0g. For any given value of λ, the timing error (δ) between a 0g target and a 1g target decreases nonlinearly with increasing v0 (Fig. 10 middle). We tested the hypothesis that adaptation reduces δ by progressively decreasing λ by fitting the prediction of the 1g model (δ* = gλ2/2v0) to the set of experimental δ-values.
Figure 10 right shows examples of the mean δ-values derived from 3 successive presentations of each v0 in 0g test trials of group 2. Fitting the prediction of the 1g model yielded the following estimates of λ-values: 0.199, 0.180, and 0.132 s for the 1st, 2nd, and 6th repetition, respectively (r2 = 0.79, 0.91, and 0.97). Figure 11 shows the changes of the estimated λ-values in all repetitions of group 2. The results were similar in all groups. In general, each family of experimental δ-values from a given repetition was adequately fitted by the 1g model with a given λ-value (P < 0.05 for all subjects). As predicted by the hypothesis of 1g model adaptation, λ-values tended to decrease monotonically with 0g repetitions. λ was 134 ms and PT was 74 ms (r2 =0.89) in the last repetition. Exponential fitting (Eq. 1) of the changes of λ with repetition showed a rapid decrement (time constant = 1.8 ± 0.8) from an initial value of 205 ± 23 ms to a steady-state value of 135 ± 28 ms.
Although the 1g model fitted the data significantly, it did not always reproduce the data at the highest initial velocity (v0 = 4.5 m s–1). Thus the model predicts a monotonic nonlinear decrement of δ-values with increasing v0 (Fig. 10B), but in some cases the δ-values at v0 = 4.5 m s–1 were actually slightly greater than those at v0 = 2.5 m s–1 (see Fig. 5).
A striking result of the punching experiments was that, even after prolonged exposure, visual targets moving vertically downward at constant velocity were misrepresented as accelerated by gravity. To verify whether these illusions depend on the nature of the visual stimuli or they depend on the dynamic context of interception, we carried out an additional series of experiments. Here the subjects were presented with targets descending with randomly assorted initial velocities and accelerations as in the Random protocol of punching experiments, but there was no real ball falling behind the screen. Instead the subjects were required to “explode” the virtual target by clicking the mouse at the interception time.
For most conditions, the results were the reverse of those of punching, both at a population level and at an individual level (one subject participated in both sets of experiments 1 yr apart). Thus the responses to slower 0g targets (that were timed too early in punching) were correctly time-locked to interception time from the first attempt and varied little in successive trials in virtual interception (Fig. 12). On average, the TTC of the button press was 0 ± 31 ms (mean ± SD, n = 720) for all 0g targets at v0 ≤ 1.5 m s–1. Conversely, the responses to slower 1g targets (that were timed correctly in punching) were slightly but significantly (P < 0.001) late in virtual interception. On average, TTC was 21 ± 40 ms (mean ± SD, n = 958) for all 1g targets at v0 ≤ 2.5 m s–1. The timing differences between 0g and 1g targets at corresponding v0 were highly significant (P < 0.001). The interception rate computed within ±15 ms relative to the interception time was not significantly different between these 0g targets (39 ± 10%) and 1g targets (40 ± 12%). However, the interception rate computed within the time window (±60 ms) comparable to that used for punching experiments was significantly (t-test, P < 0.001) higher for 0g targets (95 ± 4%) than for 1g targets (84 ± 15%).
The responses to faster 0g targets (v0 ≥ 2.5 m s–1) were much more variable across trials than those of slower targets (the mean SD of the former was twice as large as that of the latter). On average, they were slightly but significantly (P < 0.001) early (TTC = –28 ± 62 ms, n = 479).
Three-factor ANOVA showed a significant effect of acceleration (P < 0.001) and v0 (P < 0.001), but no significant effect of repetition (P = 0.772) when all the trials were pooled together. However, the mean changes of TTC with repetition were significantly (P < 0.05) fitted by an exponential (Eq. 1) in a number of individual 0g and 1g conditions (see fitting curves in Fig. 12), indicating the presence of training effects. The changes were of limited amplitude and could involve either a decrement or an increment of TTC with repetition. They were rapid for slower targets (time constant of 1.6 ± 0.9, means ± SE), and more gradual for fast targets (15.3 ± 9.9).
The finding that the TTC of most 0g targets was estimated better than that of most 1g targets indicates the use of a dynamic model that assumes uniform motion in virtual interception. This hypothesis was supported quantitatively by the results of modeling (Fig. 13). The TTC* values predicted by different interception models are plotted as a function of v0, along with the mean experimental values of TTC using the same analysis previously applied to punching (see Fig. 8). The overall best fit (mean r2 = 0.61, P < 0.01) of the ensemble of all TTC data was provided by the τ-model (dash-dotted curve) with a processing time (PT) and a threshold time (τ) of 161 ms. As expected, this model did not accurately reproduce TTC values corresponding to high initial velocities (v0 = 3.5 or ≥2.5 m s–1 for 1g and 0g trials, respectively). The other 2 models (1g and distance model) failed to fit the ensemble of data (yielding low, nonsignificant correlations).
We showed that the timing of responses aimed at intercepting a visual target was very different depending on the dynamic context, virtual interception versus punching. Timing differences between the 2 conditions were present from the first trial, before any feedback of performance, and persisted throughout practice. They were observed both at the population level and at the individual level of the subject who participated in both sets of experiments. We will argue that they reflect the operation of predictive models from high-level processing that affect responses evoked by low-level visual inputs.
Although humans are able to analyze visual motion in general scenes, they can easily be deceived into seeing incorrect motion of simple stimuli (Weiss et al. 2002). We must then address the question of possible perceptual biases arising in the present experiments arising from the nature of the visual stimuli employed. The visual system normally integrates information from multiple sources to judge the relative position and motion of objects. Changes of retinal image size, binocular disparity, optical gap between target and interception point, and eye movements are known to contribute to TTC estimates under different conditions (Regan and Gray 2000; Tresilian 1999). These cues are probably combined to generate more robust estimates, as in the dipole model combining image size and disparity cues (Rushton and Wann 1999).
For bypass approaches such as those of the present experiments, both expansion and translation of the retinal image are critical cues (Bootsma and Oudejans 1993; Tresilian 1999). It is reasonable that to achieve best accuracy in ballistic interceptions, visual information is integrated from the onset up to the latest possible time when the motor response must be centrally triggered (Land and McLeod 2000). For our experimental conditions we estimate the viewing time (VT) available to integrate visual information as VT = TT = IT – LT (see Fig. 2). For punching, the best-fitting model (1g model) yielded an estimate for LT (λ) of about 200 ms (before adaptation). For virtual interception, the best-fitting model (τ-model) yielded an estimate for LT (τ) of about 160 ms. Thus VT ranged from about 150 ms (at v0 = 4.5 m s–1) to about 400 ms (at v0 = 0.7 m s–1) for 1g targets, and from about 300 ms to about 3 s for 0g targets (it was about 70 ms longer after adaptation). Based on the geometry of our experimental setup (see methods), we estimate that a number of the optical variables that are known to be critical for TTC estimates were in the range of reliable measurement. Thus the visual angle subtended by the target (θ) varied from 0.8° (at trial onset) to 10.3° (at eye level), greater than the 0.7° size reported to result in accurate estimates of absolute time to collision for constant velocity approach under binocular conditions (Regan and Gray 2000). We estimate that image expansion rate (dθ/dt) was always >0.01 rad/s, and reached values between 0.16 and 0.9 rad/s (depending on the condition) at the end of VT. Vertical angular velocity (dγ/dt) was between 5 and 20 times greater than dθ/dt. It was always >0.1 rad/s and varied steeply, reaching values between 1.4 and 5 rad/s at the end of VT. The lowest vertical angular velocity (dγ/dt) that is known to be detected by the visual system is about 0.0003 rad/s, and the discrimination of differences in dγ/dt is very precise (Weber fraction is about 5%) over a range between 0.03 and 1.2 rad/s (de Bruyn and Orban 1988; Orban et al. 1984). Outside the optimal range, the Weber fraction increases steeply. Thus our targets at lower speeds should have been within the optimal range of discrimination, but faster targets may have been outside this range.
Clearly, a visual preference for slower speed cannot explain why subjects were much more accurate at intercepting the fast 1g targets than the slower 0g targets in the punching task. Moreover, the interception performance of our subjects with slow 0g targets was dramatically different in the 2 tasks: to the degree that subjects were accurate in the virtual interception, they were inaccurate to the same degree when punching. Also this discrepancy cannot be explained by visual psychophysics.
Let us now consider the issue of visual acceleration. Although humans cannot accurately discriminate accelerations, they are able to detect them by comparing velocities in a 2-stage process (Werkhoven et al. 1992). The first stage involves a low-pass-filter stage (time window of 100–140 ms) and is followed by a discrimination stage sensitive to velocity changes (Δv/v) ≥ 17% (Brouwer et al. 2002; Werkhoven et al. 1992). Because VT was longer than 140 ms (see above) and Δv/v was >17% (it ranged between 26 and 99%), subjects should have been able to detect the presence or absence of acceleration. Here we have another conundrum for hypotheses attempting to explain the results based on visual psychophysics. If subjects could detect (if not measure) visual acceleration, why did they not cope with it for most 1g targets in the virtual interception task, whereas they did so quite accurately in the punching task?
When our subjects clicked a mouse button to intercept the descending target, response timing was consistent with the assumption of uniform motion in the absence of forces. Thus the responses to most 0g targets were on time, whereas the responses to most 1g targets were late. Motor timing was well reproduced by a first-order model (τ-model) that always assumes uniform motion, irrespective of the law of target motion. The model, however, did not reproduce accurately the responses to higher-velocity targets.
These results with virtual interception are in general agreement with several previous results obtained with different virtual environments, regardless of the motor modalities involved in the interception paradigms. In previous studies, a virtual target moved in different directions (including the vertical) on a 2D-screen with constant velocity or accelerating profiles and subjects were required either to track it with the finger (Engel and Soechting 2000; Viviani et al. 1987) or to hit it with the hand (van Donkelaar et al. 1992) or with a handheld rod (Brouwer et al. 2000) or to intercept it with a 2D-manipulandum (Lee et al. 1997; Port et al. 1997) or with a joystick controlling the screen cursor (Merchant et al. 2003). In one study, subjects had to squeeze a stationary ball attached to their hand when they thought that a virtual 3D ball projected in depth on a head-mounted display would hit them (Rushton and Wann 1999). Irrespective of the specific kind of movement involved, the timing of all motor responses was related to target position and velocity, but poorly related (if at all) to target acceleration. In particular, Engel and Soechting (2000) were able to account for all responses with a model including direction and speed aimed at keeping τ constant. Port et al. (1997) reported a dual strategy: the responses to slower targets were well described by the τ-model, whereas the responses to fast targets were better described by the distance model. First-order strategies that ignore on-line estimates of acceleration also were used in relative judgment tasks, as when subjects were asked to choose when accelerating or decelerating objects would collide with a stationary target (Bootsma and Oudejans 1993).
On the whole, it appears that uniform motion is attributed as the default motion of virtual, massless objects, and expected dynamics agrees with first-order visual information about TTC. However a different dynamic context (for instance, accelerative) can be attributed to virtual targets under different experimental conditions, as when they are masked for prolonged temporal intervals (Tresilian 1995). Thus if an observer views a virtual moving object that suddenly disappears, memory of the object's final position is shifted forward in the direction of motion, in accord with a representational momentum (Freyd 1987). Descending motion leads to larger forward displacement than does ascending motion, and the memory distortion for horizontally moving objects is also displaced downward below the path of motion, consistent with the hypothesis that the effects of gravity are internalized (Hubbard 1995).
When punching a hidden real ball arriving in synchrony with the visual target, our subjects systematically timed their responses consistent with the assumption of the effects of gravity on an object's mass. Thus the responses to 1g targets were correctly timed, whereas the responses to 0g targets were too early. Motor timing was well reproduced by the 1g model.
The present results for intercepting 1g targets agree with previous results on catching free-falling balls, where the anticipatory muscle responses were time-locked to contact over a wide range of drop heights (Bennett et al. 1994; Lacquaniti and Maioli 1989; Lacquaniti et al. 1993a,b; Lang and Bastian 1999). Present results also can be compared with those of 2 studies in which subjects punched balls dropped vertically toward the head (Lee et al. 1983; Michaels et al. 2001). In those studies (but not here), before punching there was a slow preparatory phase of elbow flexion. In all studies, the ball was punched by a ballistic limb extension timed independently of drop duration, as predicted by the 1g model. TTC was invariant across a 0.35- to 0.6-s range of drop durations here, across a 0.8- to 1.2-s range in Lee et al. (1983), and across a 0.9- to 1.3-s range, independent of ball size and viewing conditions (monocular/binocular) in Michaels et al. (2001).
The present results for intercepting 0g targets agree with the previous observation that astronauts continue to anticipate the effects of gravity to time catching movements in microgravity (McIntyre et al. 2001). A ball launcher attached to the “ceiling” of the Spacelab module projected a 0.4-kg ball downward at constant speed from a height of 1.6 m above the hand position, with one of 3 different, randomly assorted speeds. The peak of anticipatory biceps EMG occurred earlier (relative to impact) in microgravity compared with ground (when the ball was accelerated by gravity), and the time shifts varied nonlinearly with v0 as predicted by the 1g model.
Adaptation of 1g model
Here we showed that, with training, the gravity model was not switched off, but adapted to nonaccelerating targets by shifting the time for motor activation. We used 3 different training protocols and the results were independent of the order with which 1g and 0g trials were practiced.
In theory, with practice subjects might have switched off the 1g model, relying entirely on visual information to estimate TTC. Motion at constant speed should be accurately measured by the visual system and used to predict TTC, as it happened in virtual interception. For punching, this hypothesis is refuted by 2 observations: the responses to 0g targets always remained premature, and the unexpected occurrence of 1g catch trials during immersive 0g training caused significant aftereffects (Shadmehr and Mussa-Ivaldi 1994). Here an aftereffect is the response that results when the subject is expecting a given law of motion but a different law of motion unexpectedly occurs. The response to the unexpected 1g catch trial was delayed in time in the opposite direction to that of the predictable 0g test trials (Fig. 7).
The data also refute the alternative hypothesis that subjects develop a new model of target acceleration, intermediate between 1g and 0g. This hypothesis predicts that the TTC values of 1g catch trials should be much shorter than those found experimentally, and they should be inversely proportional to v0 (the opposite of the experimental trend). Moreover, it is known that catch trials cause errors that interfere with a new internal model and affect future movements (Thoroughman and Shadmehr 2000). If the changes of TTC with 0g repetition were achieved through learning a new internal model, one would expect that the performance error in the 0g test trial after a 1g catch trial should be greater than in the previous corresponding 0g test trial, indicating partial unlearning of the newly acquired internal model (Thoroughman and Shadmehr 2000). This prediction was not borne out by the results. The data contradict another outcome predicted by the hypothesis, that when 0g and 1g accelerations are equiprobable in the same session as in the Random protocol, the TTC values of 1g targets should shift progressively later with training, in parallel with the TTC shifts of 0g targets.
Nature of internal models for interception
Response timing was best explained by the τ-model in virtual interception and by the 1g model in punching. By definition τ is the ratio between target distance and velocity at the threshold time to generate the motor response within the CNS. Although τ could be an optic variable related to first-order visual mechanisms (e.g., the inverse of the rate of dilation of the retinal image or the inverse of the rate of change of relative disparity; Lee and Reddish 1981; Rushton and Wann 1999), it could also be an internalized control variable related to a 0g dynamic model of target motion. Context-dependent switching between the τ-model and the 1g model, as probably occurs when part of the target trajectory is masked (see above), favors the view that the τ-model is an internal dynamic model based on prior expectations.
The 1g model is related to target distance and velocity at threshold time and incorporates an estimate of gravitational acceleration (Lacquaniti and Maioli 1989; Lacquaniti et al. 1993a; McIntyre et al. 2001). This estimate could be an internalized control variable related to a 1g dynamic model. Although presumably it is fed by on-line sensory information (such as that from vestibular and other graviceptors), it is also independent of that. This hypothesis is supported by the following arguments. First, the 1g model was applied here for punching but not for virtual interception, despite graviceptive information that was obviously invariant. Second, the 1g model has been shown to be applied for catching a dropped ball in the Spacelab (McIntyre et al. 2001). In orbital flight, the spacecraft and its occupants are in free fall so there are no contact forces of support on an astronaut's body opposing the action of gravity. This means that there are no gravity-determined tactile cues related to body orientation and no gravity-determined otolith cues about head orientation (Lackner and DiZio 2000). One can speculate that a 1g model can also be applied to the perception of self-motion, in addition to that of the objects' motion. A dramatic example of the importance of cognitive context in switching on and off the application of a 1g model to self-motion perception again is provided by microgravity observations. Lackner and DiZio (2000) report that sensations of falling are normally absent in astronauts, presumably because of the stable visual field, but that they can be elicited by closing the eyes and imagining that one is jumping off a cliff.
In this report we considered threshold-based models for timing interception. These models assume that the response is triggered within the CNS when a given visual or internalized variable has reached a critical threshold value, and then the response proceeds without further corrections until the end. This assumption is legitimate for ballistic interceptions with a stereotyped waveform, such as punching or button-press responses or other fast movements (Fitch and Turvey 1978; Lee et al. 1983; Michaels et al. 2001; Tyldesley and Whiting 1975; Wollstein and Abernethy 1988). Here we showed that the waveform of the positive acceleration of the hand in punching was highly stereotyped, independent of the law of target motion (Fig. 4). The models should incorporate continuous control to be applied to slower responses developing under on-line sensory feedback until the end (Bootsma et al. 1997; Brenner et al. 1998; Engel and Soechting 2000; Viviani et al. 1987).
In virtual interception, the threshold time τ was about 160 ms and did not change appreciably with training. In punching, the threshold time λ was about 200 ms before adaptation and dropped rapidly to about 130 ms with training with 0g trials. The lower λ-value is close to the estimates of the shortest visuomanual delay times (Carlton 1981; Lacquaniti and Maioli 1989; Lee et al. 1983; Port et al. 1997). This may explain why the adaptation process saturated early on, and the responses to 0g trials remained premature through the rest of training.
Our models of target dynamics are meant as perceptuomotor models and do not specify the implementation. The primate cortical networks involved in visuomanual coordination have been extensively investigated for reaching to stationary targets, but much less for interception of moving targets. Neural correlates of predictive interception have been recently described in motor cortex and area 7a of monkeys (Merchant et al. 2004). Neural activity in these 2 areas was associated with different parameters of target motion (stimulus angle or τ) depending on the visual context (real or apparent motion, respectively). The parietoinsular vestibular cortex (PIVC; Grusser et al. 1990), instead, could be one substrate of a gravity model. This region receives vestibular information from the vestibular nuclei by a thalamic relay (VPLo and VPI nuclei), visual information from the accessory optic system and the pulvinar, and somatosensory information from areas 2v and 3a in primary somatosensory cortex (S1). PIVC is reciprocally connected with the posterior parietal cortex (area 7) and cingulate cortex. Many neurons in PIVC respond to head accelerations, visual motion, and somatosensory stimuli from neck and trunk (Grusser et al. 1990). Multisensory fusion may then result in a gravicentric code. A gravity model might be encoded in a distributed network also involving subcortical structures (e.g., the vestibulocerebellum) to be used for different forms of sensorimotor coordination (e.g., eye movements evoked by head tilt and rotation).
In conclusion, in Helmholtz's view our percepts are the best guess as to what is in the world, given both sensory data and prior experience (Helmholtz 1866). We presented evidence that interceptions of moving targets are based on visually measured kinematics and an a priori hypothesis about the causes of target motion. In the language of estimation theory, the posterior probability (the probability of a target dynamics given the visual measurements of target kinematics) would be computed from the likelihood of target kinematics and from the prior using Bayes's rule (Weiss et al. 2002). The prior for virtual, massless objects is that they are moving freely outside a force field. By contrast, the prior for a real object is that it moves in the earth's gravitational field. We surmise that, although the prior for virtual motion should be susceptible to modification by inserting an artificial but convincing force field in the scene (people can learn complicated videogames with arbitrary laws of motion), the prior of earth's gravity for real objects should be highly resistant to changes.
Here subjects did not develop a new model appropriate for nonaccelerating targets to improve their chances of success in punching. Instead they stretched the parameters of the preexisting 1g model. This contrasts sharply with the results of most other learning paradigms. Practice with a novel environment normally leads to formation of the appropriate internal model, and multiple internal models specific for individual objects can be stored in motor memory (Flanagan and Wing 1997; Gordon et al. 1993; Hore et al. 1999; Shadmehr and Mussa-Ivaldi 1994; Wolpert and Kawato 1998). However, although we normally experience many different types of objects, on earth we experience only one gravity level. The gravity model might be construed as one of the perceptual-cognitive universals that according to Shepard (1994) exist in our brain as reflections of world invariants (such as the period of the terrestrial circadian cycle). In a different perspective, Gibson (1966) also emphasized the ecological importance that organisms' perception becomes tuned to environment invariants (such as gravity) by continuous exposure over thousands of years. Survival in the forest as on the sports field may take advantage of a single gravity model with fast adaptation of its parameters. Adaptation would be the most parsimonious solution to deal with variable drag effects. Vertical motion of objects is accelerated by gravity and decelerated to a variable extent by air (or other fluid) friction depending on the object's mass, size, and fluid viscosity, but falling objects seldom reach constant velocity. The required adaptation then is generally more limited than that involved by the present punching experiments, and should be much more successful. However, a falling feather remains difficult to catch for everyone!
We thank B. Dicker from G. W. Linsk Co. for generous assistance with the solenoid and Drs. Andrea d'Avella and Daniel Wolpert for a critical reading of an earlier version of the manuscript.
The financial support of Italian University Ministry, Italian Health Ministry, and Italian Space Agency is gratefully acknowledged.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2004 by the American Physiological Society