| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
1Center for Visual Science, University of Rochester, Rochester, New York 14627; and 2Department of Psychology, University of Minnesota, Minneapolis-St. Paul, Minnesota 55455
Submitted 26 February 2003; accepted in final form 15 October 2003
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
; Goodale and Milner 1992; Haffenden and Goodale 1998; Milner and Goodale 1995). Although several authors have called into question the interpretation of some of these results (Bruno 2001; Franz et al. 2001), the full body of research raise questions about whether one can unquestioningly use results from perceptual studies to make inferences about visuomotor transformations. In the current paper, we use a natural, visually guided object-placement task to measure humans' abilities to transform visual information about a surface's orientation in three-dimensional space to motor behavior. We further compare subjects' performance on matched perceptual and motor tasks to parcel their visuomotor error into independent perceptual and motor components.
To address the main goals of the experiment, we applied discriminant analysis to the analysis of movement kinematics to accurately measure the visuomotor system's sensitivity to the visual information used to control a simple goal-directed hand movement in three-dimensional space. Such methods have been previously applied to study the time evolution of hand grip formation for grasping complex objects (Santello and Soechting 1998). We applied the analysis to motion trajectories in an object placement task to derive "d-prime" measures (d's) of subjects' visuomotor sensitivity to visual information about the three-dimensional orientation of flat surfaces. This allowed us to address two main questions. First, we measured visuomotor sensitivity to different sources of information about a surface's orientation in three-dimensional space. We measured sensitivity to binocular and texture information in isolation and looked for evidence that visuomotor sensitivity improves significantly when texture information is added to binocular information about orientation in depth. Second, by comparing perceptual and motor performance over a range of memory delays, we tested the hypothesis that visuomotor and perceptual performance derive from a common visual representation of surface slant. In particular, we tested the hypothesis that errors in visuomotor performance could be decomposed into independent perceptual and motor noise sources.
Background
The visual scene contains many cues to the three-dimensional layout of objects within it. Of these cues, binocular information (provided by retinal disparities and vergence angle) is often considered dominant within a person's immediate workspace—the space relevant to normal object manipulation movements. Consistent with this assumption, a number of researchers have found that reaching performance degrades in monocular viewing conditions—movement times increase, the proportion of time spent in the deceleration phase increases, the number of re-accelerations in hand movements increases and the number of secondary re-openings of finger grip increase (Kruyer et al. 1996; Moll and Kuypers 1980; Servos 2000; Watt and Bradshaw 2003). Marotta et al. (1997) have also argued for a special role of binocular information based on a functional dissociation between monocular and binocular performance in an apperceptive agnosic (patient DF). In some of these studies, however, monocular cues were kept sparse (for example an illuminated ball in the dark). In such conditions, it is not surprising to find degradations in motor performance in monocular conditions. Perhaps more importantly, the tasks studied required reaching to targets at different depths, for which accurate information about absolute egocentric depth is critical. Monocular cues generally provide poor information about absolute depth, so that it makes sense for the visuomotor system (or, for that matter, the perceptual system) to rely more heavily on binocular information to estimate absolute depth.1
To place binocular and monocular cues on more equal footing, we studied a task the performance of which requires visual information about planar surface orientation, a geometric property determined by relative depth rather than absolute depth. Cue-integration studies have shown that at low slants (angles away from the fronto-parallel), stereo information dominates perceptual judgments, while at larger slants monocular cues like texture can dominate (Knill and Saunders 2003). In the two experiments reported here, we studied a simple object-placement task in which subjects were required to place a cylindrical object flush onto a flat surface oriented at different slants away from the viewer. We fixed the axis of rotation (tilt) of the surface in space to be horizontal and the target location for cylinder placement to be at the center of that axis. This effectively removed uncertainty about the location at which subjects had to place the cylinder. The visual information relevant to accurately performing the task, therefore, was principally that specifying the orientation of the surface.
The first experiment measured subjects' visuomotor sensitivity to binocular and texture cues to three-dimensional surface orientation. By comparing sensitivity measures in single cue stimulus conditions with those found when both cues were available to subjects, we looked for evidence that subjects' performance improves when monocular cues like texture are added to binocular cues. Experiment 2 was designed to separate the contributions of perceptual and motor "noise" to variability in subjects' visuomotor performance. We did this by comparing subjects' perceptual discrimination thresholds and visuomotor discrimination "thresholds" for the same stimuli at a range of memory delays. This allowed us to test the additivity prediction of the hypothesis that the two sources of noise are independent. It also allowed us to more accurately test the cue additivity hypothesis because the hypothesis makes predictions specifically about perceptual uncertainty.
|
EXPERIMENT 1: VISUOMOTOR SENSITIVITY |
|---|
|
TOPABSTRACTINTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
|
|
Three undergraduates at the University of Minnesota served as subjects in the experiment. All three subjects had corrected to normal vision and were paid for their participation. The subjects were naive to the purpose of the experiment.
Apparatus and stimuli
Figure 1 illustrates the apparatus used in the experiment. Figure 2 shows the physical dimensions of the experimental setup. A robot arm (PUMA 260) positioned a flat surface 45 cm from the subject's eyes (30.5 cm in front of the subject and 32.5 cm below the level of their eyes). The arm was used to rotate the surface by different angles around a horizontal axis through a fixed point in space. The angle of the rotation axis in the horizontal plane was chosen to be horizontal from the viewpoint of the observer. Textures were printed on 8.5 x 11-in paper and slipped into slots on the edges of the target surface. Different textures could be displayed by changing the textured paper placed on the target surface. The target position for cylinder placement was generated using a laser pointer illuminating a spot at the center of the target surface. The horizontal starting platform for the cylinder was positioned to the right of and above the test surface. In a coordinate frame centered on the target location for placement, with the y axis taken parallel to gravity (positive up), the x axis parallel to the rotation axis of the surface (positive to the right of the subject), and the z axis parallel to the cross-product between those two (positive toward the subject), the starting position for the cylinder was at (35 cm, 4 cm, 0 cm). The starting platform had lips on two sides into which subjects could slot the cylinder at the end of each trial, guaranteeing that the starting position was the same on each trial.
|
Two different types of texture patterns could be mounted on the target surface, a white-noise texture and a texture composed of a regular array of dots (see Fig. 3). Subjects viewed these either monocularly or binocularly, making four stimulus conditions for the experiment. Seven different surface orientations (target slants) were tested, ranging from 70 to 100°, where 90° was level (perpendicular to gravity) and 47° would have been fronto-parallel to the subject.
Subjects, with heads fixed in a chin rest, viewed the target surfaces through translucent cylinders mounted in front of each eye in such a way that each eye's view formed a circle centered on the target location for cylinder placement. Subjects' field of view on the surface subtended a visual angle of 13.2°, which, in all conditions, was within the bounds of the target surface. The left eye was occluded in the monocular conditions.
Subjects wore headphones through which auditory signals were given to begin a movement, to return to the starting position and to close eyes between trials.
Procedure
Subjects ran in eight sessions on separate days. Each session was further subdivided into four blocks, one for each stimulus condition (monocular vs. binocular viewing crossed with regular vs. noise textures). Stimulus blocks were randomized across sessions. Data from the first session were discarded as practice. Each block consisted of ten trials per test slant, making 70 trials per block. After every 10 trials, there was a brief break of 
20 s during which the experimenter changed the texture pattern mounted on the target surface. Within a block the different textures represented different samples of the same type of texture (different white-noise patterns, large or small dot patterns). Subjects were given a break of several minutes after each block of trials. Subjects finished each session in 40 min. The order of blocks was randomized between sessions and counter-balanced so that the different cue conditions appeared in each temporal position within a session twice.
Each trial began with the subject holding the cylinder stationary in the starting position. A trial was initiated by a "go" signal given over the headphones. One second later, a "stop" signal was given over the headphones. Subjects were instructed to place the cylinder flush onto the surface before hearing the stop signal. Movements lasted for between 300 and 600 ms, reflecting the fact that the 1-s window given to complete the movement was well within the natural limits imposed by the task. One second after the stop signal, a "return" signal was given instructing the subject to return the cylinder to the start position. After the cylinder was stably placed at the start position, a close signal was given instructing subjects to close their eyes, during which interval the robot arm rotated the surface to a new test orientation. This was followed by an "open" signal, a 1-s delay and the beginning of a new trial with the go signal.
We recorded 2 s (200 frames) of data from the Optotrak on each trial, beginning at the time of the go signal.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
DETECTING MOVEMENT START AND STOP TIMES. For purposes of analyzing visuomotor performance, we defined the trajectory of the cylinder to extend from the time a movement started to the time of initial contact with the surface. We defined the start time to be the time at which the cylinder's orientation first deviated from the starting orientation (vertical) by >0.5°. We used the cylinder's acceleration profile to determine the initial contact time. Figure 4 shows an example acceleration profile for the point at the center of mass of the three markers on the cylinder (calculated using discrete differences). As clearly evident in Fig. 4, contact with the target surface was marked by a sharp negative peak in acceleration. Often, a second peak was also evident, reflecting final contact with the surface following a secondary rotation to bring the cylinder flush with the surface. To mark the initial contact time, we first found the two local minima in acceleration with the largest negative values in the movement. When these occurred in brief succession (within 50 ms of each other), the first was selected as the contact time; otherwise the minimum with the largest negative value was selected.
|
TIME NORMALIZATION. Subjects' movements varied in duration from trial to trial with a standard deviation of 10% of the mean. We used a cubic spline interpolator to normalize the trajectories for analysis. The marker positions on the side of the cylinder were used to calculate the orientation (slant and tilt) and position (at the center) of the cylinder at each sample point in time. For each trial, the sampled orientations and positions of the cylinder were interpolated to give 100 uniformly spaced measures between the detected start and contact times. The result was a set of 100-dimensional vectors characterizing the slant, tilt, and three-dimensional positions of the cylinder as it moved from the starting surface to the target surface.
OUTLIER REJECTION. We defined outlier trials to be thrown out of the analysis as those which matched one of several criteria. We discarded trials with durations <250 ms, durations >1 s or a cylinder orientation at the time of the go signal >0.5° away from the vertical. Histograms of final contact slants showed that occasionally, though rarely, subjects were very far off from the mean settings. These contact slants were far enough away from the mean that they showed up as isolated points well away from the mass of the histogram. To account for these outliers, we removed trials on which the final contact slant was more than three standard deviations away from the mean of the contact slants found for a given stimulus and target slant condition. Based on the criteria listed here, we rejected on average 10 trials of 70 for each stimulus/slant condition. On average, less than one of those trials was rejected on the basis of being more than three standard deviations away from the mean.
Kinematic analysis
To assess the rotation kinematics of subjects' movements, we decomposed the orientation of the cylinder into three components: its slant (angle in the z-y plane as defined in METHODS), its tilt (angle in the x-y plane defined in METHODS) and its spin (angle around its central axis). Note that we have defined slant in a gravitational frame of reference so that 90° is upright relative to gravity. To obtain the slant of the target surface in the subjects' frame of reference when the cylinder was flush on the surface, one would subtract 47° from the measured slant of the cylinder. Tilt was the same in both frames of reference. We will focus our analysis on the rotational motion of the vertical axis of the cylinder (its slant and tilt). The spin of the cylinder was irrelevant to the task. Analyses showed that the spin kinematics did not vary significantly between stimulus conditions.
Figure 5 shows average slant and tilt trajectories (expressed as functions of normalized time) for the seven different target slants in the full cue condition. The figures show data from the best stimulus condition (the binocular, good texture condition). The qualitative shapes of the paths and the trajectories were the same for all three "informative" stimulus conditions (excepting the monocular noise texture condition). Not surprisingly, because only the slant of the target surface changed between stimulus conditions, most of the between condition variation in the rotation kinematics appears in the slant kinematics of the cylinder.
|
A standard way to analyze the sensitivity of the visuomotor system to the visual information in our task would be to measure constant and variable errors in the end-point slants of the cylinder as a function of target slant. Figure 6 plots the average slant of the cylinder at initial contact with the target surface as a function of the target slant for each of the four stimulus conditions. For the three information-rich stimulus conditions, there is little constant error except at the highest slant. Although the contact slants in the monocular/white-noise stimulus condition show a regression toward a constant, intermediate slant, it clearly shows that some information for target slant is available in that condition. Figure 6 also shows the standard deviations of cylinder slants at initial contact for the four stimulus conditions. Note that the standard deviations increase dramatically for the monocular/white-noise condition.
|
One way to do this would be to derive from the sample data an optimal, unbiased estimator of target slant from cylinder trajectory data. The variance of this estimator would reflect the visuomotor sensitivity of the system to target slant information. Doing so would require making assumptions about the global form of the estimator (e.g. linear) that might bias the results. Because the data from the experiment were derived from a small set of discrete target slants, we took a somewhat different tack, treating each target slant as a discrete stimulus category. We applied linear discriminant analysis (Duda and Hart 1973) to derive d' measures specifying the average discriminability of trajectories generated for target slants that differ by 5° (see APPENDIX A for details). This approach fits a different linear discriminant function to local pairs of slants (e.g. 75 vs. 80° and 85 vs. 90°), allowing for local deviations from global linearity. We refer to the resulting d' measures as the visuomotor d's for slant estimation. A d' of 1, for example, would indicate that an optimal linear model could correctly discriminate randomly drawn trajectories to surfaces differing in slant by 5° 76% of the time. The d' measure provides a bias-free measure of the reliability of visuomotor slant "estimates" that uses all of the information provided by the output of the visuomotor system, the motion kinematics.
In applying discriminant analysis to our problem, we are faced with the problem of selecting the appropriate representation to use. Given the constraints on our data (65 trajectories per target slant per stimulus condition) and the total number of samples we have for each trajectory (35–65 per trajectory), we cannot use every sample point in the trajectory for the analysis (the number of free parameters in the discriminant function would approach or exceed the number of samples available for the analysis). Our first simplification was to analyze only the slant trajectories of the cylinder; that is, the temporal trajectories of the angle of the cylinder in the same plane in which the target surface rotated. Including other kinematic parameters (e.g. the 2 other rotation angles and the transport parameters) did not increase the d' discriminability indices, justifying our choice. We then applied the discriminant analysis to three different representations of the slant trajectories. Two were derived by subsampling the trajectories, and a third was derived from a principal components analysis of the trajectories.
Ten-point trajectories—These were 10-dimensional vectors specifying the slant of the cylinder at 10 equally spaced times between movement start and initial contact with the target surface.
Contact slants—These were scalar values specifying the slant of the cylinder at the time of initial contact with the target surface.
Principal components—For each stimulus condition, we computed the principal components of the entire set of cylinder trajectories. Ten-dimensional vector representations of the trajectories were derived by projecting them into the space of the top 10 principal components (which accounted for 99.9% of the variance in the trajectories).
d' values were computed for pairs of neighboring slants in each stimulus condition using covariance matrices and mean vectors estimated from the experimental data. Because d' values are non-negative, such a direct estimation method is inherently biased. We used a parametric bootstrap procedure to estimate both the bias in the d' estimates and the standard errors in the estimates (Efron and Tibshinari 1993).3 The d' values reported here have been corrected for the estimated bias.
The d' measures did not show any consistent variation as a function of slant, so we only report the average d' measures here. Figure 7 shows the d' values computed for each stimulus condition averaged across target slants, using each of the three estimation methods described in the preceding text. As can be clearly seen, the different trajectory representations used lead to the same estimates of d' (there were no interactions between the trajectory representation used for the analysis and target slant). Most notable is the fact that the contact slant of the cylinder captures all of the discriminable information from the trajectories. We explored the question of whether kinematic features derived from non-linear functions of the trajectories (e.g. maximum angular acceleration, time of maximum angular acceleration) add to the discriminability of the trajectories by combining them with the contact slant in the trajectory representation used to compute d' values. This gave no significant improvement, leaving us confident that the contact slant of the cylinder captures all of the information in the trajectories that reflects the different target slants used in the experiment.
|
) = 6.4, P < 0.01; subject MDY, F(2,) = 7.0, P < 0.01; subject MEL, F(2,) = 2.6, P < 0.1). Planned post hoc comparisons revealed a significant difference between the monocular texture condition and the stereo noise condition for two subjects (subject LES, Z = 2.5, P < 0.05, subject MEL, Z = 2.15, P < 0.05) and a significant difference between stereo-noise and stereo texture conditions for only one subject (subject MDY, Z = 4.6, P < 0.001). Discussion
The most significant feature of the results is how well subjects are able to use visual information about surface slant to control their orienting movements. In the full cue condition, subjects had visuomotor d's ranging from 1.5 to 2.5 for 5° differences in target surface slant. These correspond to slant discrimination thresholds in a two-alternative forced-choice task of 1.9 to 3.3° (at 76% correct). By comparison, in a perceptual slant discrimination experiment using computer rendered displays, we have found thresholds ranging from, on average, 4–10° for the range of slants used in the current experiment (Saunders and Knill 2003). This may reflect a higher efficiency for visuomotor processing or simply a difference in visual conditions (virtual vs. real surfaces). Experiment 2 was designed in part to resolve this question.
Two of the three subjects showed somewhat better performance in the stimulus condition containing only binocular information than in the condition containing only monocular, texture information. Only one of the three subjects showed a significant improvement in performance when both stereo and texture information were available. Two things are notable about the results. First, subjects' performance with binocular information is little better, on average, than it is with monocular texture information. This is consistent with earlier results using virtual stimuli, in which perceptual discrimination thresholds for surfaces slanted away from the viewer at 30° are, on average, equivalent for stimuli containing only stereo information and stimuli containing only texture information (Saunders and Knill 2003). Second, only one of the three subjects shows evidence for efficient cue integration (subject MDY). The lack of improvement in the multiple cue condition (binocular views of regular textures) in the other two subjects may, however, simply reflect high levels of motor noise relative to noise in perceptual estimates of slant. Were this the case, the high levels of motor noise would overwhelm the small improvements in d' predicted by efficient cue integration. Experiment 2 helps to resolve this issue.
|
EXPERIMENT 2: COMPARISON WITH PERCEPTUAL SENSITIVITY MEASURES |
|---|
|
TOPABSTRACTINTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
) found slant discrimination thresholds no better than 10° for surfaces at 30° slant, which is within the range tested here. Even for stereo views of textured surfaces, they found thresholds averaged 7° for test surfaces at 30° slant. This runs counter to expectation, as visuomotor sensitivity is limited by the cumulative effects of sensory/perceptual uncertainty and motor noise. One would expect that visuomotor thresholds would be higher than visual discrimination thresholds. Of course, the results are not strictly comparable, as the current results were obtained with real surfaces while the others were obtained with virtual stimuli. One explanation for the high levels of visuomotor sensitivity measured here is that visuomotor performance is driven by specialized transformations that are more efficient than those subserving perceptual judgments. This is the position argued by Goodale and Milner in their "two visual systems" hypothesis. Experiment 2 was designed to test whether this explanation accounts for the results of experiment 1 or if a simpler hypothesis, that both perceptual judgments and visuomotor performance derive from a common representation of slant, could account for the data.
The logic of the experiment follows from the claim that visuomotor transformations have a short memory. Goodale et al. (1994) have argued that the visuomotor system's effective memory is <2 s and that for memory delays greater than that, the system reverts to stored perceptual representations of object properties to program grasping movements. Previous studies have also shown that some perceptual illusions only begin to reflect themselves in visuomotor behavior after delays 
2 s between stimulus presentation of the initiation of movement. The result has been interpreted as reflecting a shift in relying on special-purpose visuomotor mechanisms to a reliance on biased perceptual representations at long delays (Bridgeman et al. 2000), although this interpretation has been called into question.
Were subjects' visuomotor performance based on the same representation as perceptual judgments regardless of delay, one would expect similar changes in performance as a function of the delay between stimulus presentation and movement, on the one hand, or judgment, on the other. Previous studies of perceptual discrimination performance as a function of memory delay are consistent with a random walk model on the stored variable. This leads to a linear change in squared discrimination thresholds as a function of delay. The common representation hypothesis, therefore, predicts that visuomotor performance will decay in a similar manner. The "two systems" hypothesis, on the other hand, predicts a non-linear change in performance at the point where the system switches from using special purpose visuomotor transformations to relying on stored perceptual representations of stimuli.
In experiment 2, we measured visuomotor slant discrimination "thresholds" as a function of the delay between stimulus presentation and the signal to initiate the object pacement movement used in experiment 1. For the same subjects, we measured perceptual discrimination thresholds using the same stimuli used in the visuomotor task as a function of the delay between stimuli that subjects were required to compare to perform the task.
Motor task
The methods used to measure visuomotor sensitivity were similar to those used in experiment 1. Differences in the specifics are outlined in the following text.
STIMULI. The viewing geometry for viewing stimuli duplicated that used in the first experiment. Stimuli consisted of field-limited, binocular views of real surfaces oriented at different slants around the horizontal by the robot arm. Target surfaces were planar white-noise patterns mounted on the end of the robot arm. The depth of the target surface away from the viewer (measured at the center of the subjects' field of view) was randomized within a range from 44 to 46 cm. This was done so that in the discrimination experiment (see following text), subjects could not rely on absolute depth judgments at any individual point on a surface to make their judgments of slant (e.g. that the stimulus in a test pair that appeared further away at the top of subjects' field of view was the most slanted).4 The robot positioned the target surface randomly within the plane of the surface so that different noise patterns appeared in the subject's field of view both within a trial and from trial to trial.
Subjects viewed stimuli at slants ranging between 13 and 37° in steps of 3° (specified in the subjects' frame of reference, where a slant of 0° represents a fronto-parallel surface). Nine slants in all were tested. The spacing between the target slants allowed us to calculate psychometric functions at different slants and hence to calculate equivalent visuomotor discrimination thresholds for the test slants used in the discrimination experiment. Subjects viewed stimuli for 2 s before the shutter glasses closed.
PROCEDURE. Four delay conditions were tested, corresponding to delays of 0, 1, 2, and 3 s between the extinction of target surface and the signal to initiate the movement to place the cylinder on the surface. Subjects ran in eight sessions on separate days. Each session was further subdivided into four blocks, one for each delay condition. Stimulus blocks were randomized across sessions. Each block consisted of 10 trials per test slant, making 90 trials per block. Subjects were given a 20- to 30-s break every 20 trials to avoid fatigue. They also had approximately a 2-min break between blocks within a session. The order of blocks was randomized between sessions and counter-balanced so that the different delay conditions appeared in each temporal position within a session twice. Experimental sessions took <1 hour to complete. The first session was discarded as practice.
Figure 8A illustrates the time course of an experimental trial. The shutter glasses were initially closed. At a preset time, the shutters opened on the stimulus and remained open for 2 s, after which they closed again. Three delay conditions were tested, in which a 1-, 2-, or 3-s delay was imposed between shutter closing and the start signal to begin the motion. A fourth, no delay condition was tested in which the start signal was given 2 s after the shutter glasses first opened, and the glasses remained open until the subject began moving the cylinder at which point the shutters were closed. Thus in the no delay condition, subjects had vision of the target up to the point of movement initiation, but not during the movement. Subjects were asked to place the cylinder on a target projected onto the center of the target surface by a laser pointer.
|
Perceptual discrimination experiment
In the discrimination experiment, subjects judged which of two sequentially presented test surfaces was more slanted away from them. We measured discrimination thresholds around target slants of 22, 25, and 28° (measured from the fronto-parallel as seen by the subject) as a function of the delay between presentations of the first and second stimulus in a trial. We varied the delay from 1 to 3 s.
STIMULI. Stimuli were equivalent to those used in the motor task. Subjects wore a pair of opaque liquid-crystal shutter glasses (PLATO shutter glasses) that allowed us to automatically limit the duration over which they viewed the target surfaces. As in experiment 1, they viewed surfaces through translucent tubes placed in front of each eye to limit their field of view on the surface. The shutter glasses were closed between trials and between stimulus presentations within a trial.
PROCEDURE. We used a temporal, two-alternative forced choice task to measure discrimination thresholds. Figure 8B illustrates the sequence of events in a trial. A subject viewed the target surface at one slant for 2 s. The view was then occluded while the robot repositioned the surface at a different slant (and depth), and the subject viewed the second surface for another 2 s. Four delay conditions were tested, corresponding to periods of 1, 1.5, 2, and 3 s between the extinction of the first surface and the display of the second surface. Because of the time it took the robot to move the surface from one slant to another and for the surface to stabilize after the transient surface motion (wobble) created by stopping the robot, 1 s was the minimal delay we could impose between views of the two surfaces being discriminated. Subjects used the computer mouse to indicate which of the two surfaces appeared more slanted (closer to a ground plane). Subjects ran in eight sessions on separate days. Each session was further subdivided into four blocks, one for each delay condition. There was a small break of 2 min between each block within a session. Stimulus blocks were randomized across sessions and counterbalanced so that each delay condition appeared in each of the four temporal positions within a session twice. Experimental sessions took <1 h to complete. The first session was discarded as practice.
We used a method of constant stimuli to estimate discrimination thresholds. Test stimuli differed by 2, 4, or 6° around each of the target slants for which we estimated discrimination thresholds (22, 25, and 28°). Thus for a target slant of 25°, test surface pairs were shown at 24 and 26°, 23 and 27°, and 22 and 28°, corresponding to slant differences of 2, 4, and 6°, respectively. Test slants thus varied between 19 and 31° with an equal number of presentations at each slant. The range of slants corresponded to angles ranging from 15 to 27° away from the horizontal (relative to gravity).
Subjects were given feedback in the form of a tone when their judgment was incorrect.
|
SUBJECTS |
|---|
|
TOPABSTRACTINTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
Results
VISUOMOTOR PERFORMANCE. Experiment 1 showed that contact slant carried all of the discriminative power of the motion trajectories. We therefore measured equivalent visuomotor discrimination thresholds by simulating an observer that made discrimination judgments based on the contact slants derived from a pair of cylinder orientation trajectories derived from randomly drawn trials in the visuomotor experiment. For each target slant, we simulated a discrimination experiment by randomly drawing trials from the target slant condition and each of the slant conditions within ±9° of the target slant condition. The average probability correct for each pair of slants was obtained by calculating the proportion of all combinations of trials on which the contact slant of the cylinder for the surface with the greater slant was greater than the contact slant for the surface with the smaller slant.5 Figure 9 plots the visuomotor thresholds calculated from these psychometric functions for each of the three subjects.
|
In the experiment, the 25° slant condition was the only one that truly isolated memory effects. Thresholds for the other two slant conditions are contaminated by the fact that some stimulus slants in these conditions were either always the smallest within a pair (21, 20, and 19°) or were always the largest within a pair (29, 30, and 31°). When one of these stimuli was presented as the first stimulus in a pair, subjects could, in theory, have made their judgment accurately based only on the first stimulus and learned absolute thresholds. We will, therefore, focus our analysis and discussion on the data for the 25° target slant.
Figure 10 shows plots of the perceptual discrimination thresholds for the 25° target slant conditions at the 1- to 3-s delay conditions at which we also measured visuomotor thresholds. Discrimination thresholds for subjects SS and JJ increase significantly with increasing delay. SS showed a 60% increase in threshold when the delay increased from 1 to 2 s (Z = 2.4, P < 0.01), whereas subject JJ showed a 76% increase (Z = 3.9, P < 0.01). Subject MEL only showed a 5% increase, which was not significant (Z = 0.3, P > 0.35). These values clearly show a significant memory cost for two of the three subjects.
|
, and the thresholds at zero delay. The correction takes the form
 | (1) |
m() is the corrected visuomotor threshold for stimuli delayed by seconds, Tm() is the visuomotor threshold computed for the same stimuli as described in the preceding text and Tm(0) is the visuomotor threshold calculated as in the preceding text for the no delay condition. DISCUSSION. Visuomotor thresholds for subjects SS and JJ closely follow the increase as a function of memory delay that would be predicted by a constant temporal decay in perceptual uncertainty. Threshold functions for MEL flatten at large delays. To gain more quantitative insight into the results, we fitted a simple version of the "common representation" model to the data. According to this model, performance on both the perceptual discrimination task and the visuomotor task is determined by the same uncertainty in visually derived slant, which is expected to vary with memory delay. The two tasks, however, are affected by different constant noise sources: decision noise for the perceptual task and motor noise for the motor task.
According to the model, perceptual and visuomotor thresholds should be related by
 | (2) |
Figure 11 shows the value of K derived for each of the three common delay conditions in the experiment. The common representation hypothesis predicts that the estimated values of K should remain fixed as a function of delay. The data show no significant difference between the estimates of K, consistent with the model. Note that this is true for subject MEL as well as subjects SS and JJ, despite the flattening of her visuomotor threshold function at high delays. This is because her perceptual thresholds follow a similar pattern as a function of delay.
|
). Discrimination thresholds for a number of simple, two-dimensional geometric properties of visual stimuli (e.g. contour curvature) show a linear shift as a function of inter-stimulus delay (Hole 1996; Sakai 2003). The small change in subjects SS and JJ's visuomotor thresholds between the zero delay and 1-s delay conditions is consistent with this model.
The data from experiment 2 are notable in two more regards. First, subjects are remarkably good at making the slant discrimination required in the experiment—much better than has been reported in psychophysical experiments using virtual stimuli (Saunders and Knill 2003). It may derive from the fact that virtual stimuli contain cues that remain fixed, regardless of the simulated surface geometry, causing cue conflicts (e.g. accommodation and blur), while those same cues covary appropriately with surface geometry in real stimuli. It may also reflect the greater reliability of stereo information when cues to absolute surface depth, needed to calibrate the interpretation of disparity, are stronger in the stimulus, as they are for real surfaces. Second, assuming that the difference between perceptual and visuomotor performance is largely due to motor noise, the data suggest highly efficient transformations between visual estimates of slant and motor output. The variance in subjects' motor performance induced by motor noise had a slant discrimination equivalence of between 1 and 2.5° for the three subjects.
|
GENERAL DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
The results of experiment 2 give no support for the hypothesis that orienting movements of the hand to match the target orientation of a surface during a natural object-placement movement rely on a visuomotor channel that is independent of a perceptual processing channel. Rather, they are consistent with the hypothesis that visuomotor variability is a result of independent perceptual and motor noise sources. To test this more fully, however, conditions must be created in which the perceptual uncertainty varies over a larger range than followed from the memory delay manipulation used in this experiment. Nevertheless, the results interpreted according to the common representation hypothesis show very small levels of internal noise, both at the perceptual input and the motor output. Because visual feedback from the moving hand (or the cylinder) was not available for the task, the low motor noise suggests efficient transformations from the perceptual representation of slant to the motor output (equivalent motor noise on the order of 1–2°, standard deviation). That end-point slant carries all of the information needed to discriminate trajectories toward one target slant from trajectories toward another further suggests that this efficiency derives both from motor planning and the efficient use of on-line control based either on internal feedback loops (using efferent copy information) and/or proprioceptive feedback.
Role of binocular and monocular cues for guiding orienting movements
Experiment 1 showed that subjects could accurately guide hand orienting movements to place a cylindrical object on a flat surface based on either binocular and texture information about surface orientation alone. The sensitivity analysis left open the question of whether or not performance improved significantly when both cues were available. One subject showed a statistically significant improvement, one showed a small but not quite significant improvement and the other showed no improvement. The results of the second experiment may well explain the small amount of improvement obtained in the multiple cue condition. Because different subjects ran in the two experiments, we cannot compare results directly; however, we can do a simple back-ofthe-envelope calculation to show why the motor noise that corrupts subjects' performance compresses any effect obtained to a very small one.
Experiment 2 showed that subjects' motor noise remains constant across memory delays. If we assume that it is independent of perceptual noise and hence equivalent across cue conditions, we can express the visuomotor d-primes measured in experiment 1 as a non-linear sum of perceptual and motor d-primes
 | (3) |
 | (4) |
. After some algebraic manipulation, this predicts that a subjects' visuomotor d-prime for the combined cue condition cue condition in experiment 1, d'TS, would be related to the visuomotor d-primes in the individual cue conditions, d', by
 | (5) |
Visuomotor control issues
Although we have focused on visuomotor sensitivity in this paper, we also noticed patterns in the kinematic data charactering the transport of the cylinder that suggest an interesting synergy between hand rotation and hand transport. Examination of the transport kinematics revealed that the transport kinematics of the cylinder appeared to be invariant over target surface orientation when represented by the mid-point of the bottom of the cylinder (Fig. 12). This result suggests that the position of the bottom of the cylinder is one of the parameters, along with orientation, that is controlled by the motor system to perform the object placement. Such a strategy makes sense given the task demands, which were to center the cylinder on the target location. Because subjects grasped the cylinder at its midpoint, controlling the path of the bottom of the cylinder so as to maintain invariance over cylinder orientation required coordinated control of the transport of the wrist through space and of the movements used to orient the cylinder— rotation of the wrist and coordinated movements of the fingers to "roll" the cylinder between them. Although a number of physiological and behavioral results support the hypothesis that proximal (e.g., hand transport) and distal components (e.g., grip formation) of prehension movements are controlled through independent visuomotor "channels" (Jeannerod 1981; Jeannerod et al. 1995; Paulignon and Jeannerod 1996), other results strongly suggest some interaction between the two, particular as regards transport and hand orientation (Haggard 1991; Gentilluci et al. 1996; Mamamssian 1997; Soechting and Flanders 1993). The current results are consistent with the concept of a flexible control system that coordinates distal and proximal components of hand movements to successfully match the demands of specific motor tasks (Todorov and Jordan 2002).
|
|
SUMMARY |
|---|
|
TOPABSTRACTINTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
|
APPENDIX A: DISCRIMINANT ANALYSIS |
|---|
|
TOPABSTRACTINTRODUCTIONEXPERIMENT 1: VISUOMOTOR...RESULTSEXPERIMENT 2: COMPARISON WITH...SUBJECTSGENERAL DISCUSSIONSUMMARYAPPENDIX A: DISCRIMINANT...APPENDIX BACKNOWLEDGMENTSREFERENCES |
|---|
, that supports maximum discriminability of the two sets of trajectories. This is given by the equation
 | (A1) |
and 
are the covariance matrices and 
and 
are the means of the trajectories measured for each target slant. Given a sample trajectory, an ideal classifier would project the N-dimensional vector representation of the trajectory onto the discriminant vector and compare the result to a scalar decision criterion to decide which target slant had been presented to the subject. The discrimination performance of such an observer is characterized using a single d' value computed as
 | (A2) |
is the inner product between 
and 
. The numerator is the distance between the projections of the two mean trajectories onto the discriminant vector, and the denominator is the square root of the average variance of each set of trajectories when projected onto the discriminant vector. One way to interpret the discriminant vector is that it is the vector that maximizes the d' value computed from Eq. A2; that is, it specifies the projection of the trajectory vectors that maximizes their discriminability. Figure A1 illustrates the analysis in two dimensions. d' also specifies the probability correct of the ideal classifier, with an unbiased observer having a probability correct given by the value of the cumulative normal distribution at the value specified by d'; thus for example, d' = 1 corresponds to 76% correct classification performance.
, the back-projection of the round viewing apertures onto the target surface when the surface was level (perpendicular to gravity). – – – in the bottom figure represents the axis around which the surface was rotated to create different target surface slants.
