Abstract
Integration of proprioceptive and visual positioninformation: an experimentally supported model. To localize one’s hand, i.e., to find out its position with respect to the body, humans may use proprioceptive information or visual information or both. It is still not known how the CNS combines simultaneous proprioceptive and visual information. In this study, we investigate in what position in a horizontal plane a hand is localized on the basis of simultaneous proprioceptive and visual information and compare this to the positions in which it is localized on the basis of proprioception only and vision only. Seated at a table, subjects matched target positions on the table top with their unseen left hand under the table. The experiment consisted of three series. In each of these series, the target positions were presented in three conditions: by vision only, by proprioception only, or by both vision and proprioception. In one of the three series, the visual information was veridical. In the other two, it was modified by prisms that displaced the visual field to the left and to the right, respectively. The results show that the mean of the positions indicated in the condition with both vision and proprioception generally lies off the straight line through the means of the other two conditions. In most cases the mean lies on the side predicted by a model describing the integration of multisensory information. According to this model, the visual information and the proprioceptive information are weighted with directiondependent weights, the weights being related to the directiondependent precision of the information in such a way that the available information is used very efficiently. Because the proposed model also can explain the unexpectedly small sizes of the variable errors in the localization of a seen hand that were reported earlier, there is strong evidence to support this model. The results imply that the CNS has knowledge about the directiondependent precision of the proprioceptive and visual information.
INTRODUCTION
Humans can use different sources of sensory information for localizing their hand, i.e., for finding out the hand’s position with respect to the body. When the hand is seen, proprioception and vision simultaneously provide useful information, and the CNS uses both sources to plan movements (Rossetti et al. 1995). An analysis of variable errors in tasks where a nonmoving hand had to be localized showed that the available proprioceptive and visual information are combined in a very efficient way (van Beers et al. 1996). In the present study, we analyze constant errors in localization tasks to gain more insight into how the multisensory information is integrated.
Various psychophysical studies have been performed to investigate in what position a hand is localized on the basis of simultaneous proprioceptive and visual information. In most of these studies (e.g.,Hay et al. 1965; Pick et al. 1969;Warren 1980; Warren and Pick 1970), wedge prisms were placed before the subject’s eyes to introduce a conflict between the hand position encoded by vision and the one encoded by proprioception. If the conflict is not too large, subjects perceive their seen hand in one single location, which is somewhere between the two positions where it is perceived on the basis of vision only and proprioception only, respectively (Warren and Cleaves 1971). It usually is localized closer to the visually perceived position than to the position perceived proprioceptively (e.g.,Hay et al. 1965; Pick et al. 1969;Warren 1980; Warren and Pick 1970). This is consistent with the results of other conflict studies in which the proprioceptive information was modified by muscle vibration, whereas the visual information was veridical (DiZio et al. 1993;Lackner and Levine 1978; van Beekum 1980). There thus seems to be a weighting of the visual and the proprioceptive information, the greater weight usually being given to the visual information. This does not mean to say that we fully understand how this weighting takes place. There are at least two different ideas about how the weights given to each modality are determined. According to one idea, the weights are determined by the precision of the information in each modality (Pick et al. 1969; Welch et al. 1979); according to another idea, they are related to the attention that is directed to each modality (Canon 1970, 1971; Kelso et al. 1975; Uhlarik and Canon 1971; Warren and Schmitt 1978).
The conflict studies thus have exposed a problem that still is unresolved: how are the weights given to vision and to proprioception determined when both sources simultaneously provide information about the hand’s position? In this study, we follow a new approach to test whether the weights are related to the precision of the information in each modality. Unlike the abovementioned conflict studies, which all involved localization in one dimension, we will study localization in two dimensions. This enables us to distinguish between different mechanisms the differences of which vanish in onedimensional space. Crucial here is the recent finding that the precision of localization in a horizontal plane on waist level is spatially nonuniform for both vision and proprioception (van Beers et al. 1998). Proprioceptive localization is generally more precise in the radial direction with respect to the ipsilateral shoulder than in the azimuthal direction. This can be understood from the geometry of the arm. Visual localization, on the other hand, is more precise in the azimuthal direction with regard to the cyclopean eye than in the radial direction. This is at least partly a result from the subjects looking down on the horizontal plane. The question then is how the visual information and the proprioceptive information are weighted in twodimensional space. One possibility, in line with the directedattention hypothesis, is that weights are given to the two modalities irrespective of the directiondependent precision. This would predict that the seen hand is localized somewhere on the straight line through the two positions where it is localized on the basis of vision only and proprioception only, respectively. The exact position at which it is localized on this line depends on the weights given to each modality. Although this may seem a reasonable idea, it is not likely to be correct because the actual variable errors in localization of a seen hand were found to be smaller than the variable errors predicted by such a model (van Beers et al. 1996). This suggests that the CNS uses the available information more efficiently. A second possibility, in which the available information is indeed used more efficiently, is that the CNS takes the directiondependent precision of proprioceptive and visual localization into account and uses different weights in different directions related to this precision. We will show that such a model predicts that the seen hand generally will be localized off the straight line through the two positions where it is localized on the basis of vision only and proprioception only. We tested this prediction in the following experiment.
Subjects were asked to match the position of a target in a horizontal plane with their unseen left hand. We used three conditions that differed in the modality in which the target was presented: proprioception only (condition P, eyes closed, right hand is target), vision only (condition V, visual target), or both proprioception and vision (condition PV, seen right hand is target). We will refer to conditions P and V asunimodal conditions because the target position was available in only one modality; condition PV is referred to as the bimodal condition. We take advantage of the fact that different constant errors generally are found in the two unimodal conditions. This enables us to study how vision and proprioception are combined without there being any need to introduce an unnatural conflict between the two modalities. However, the subjects also performed the tasks wearing prism goggles. In this way, the difference between the positions indicated in the two unimodal conditions could be varied, which allowed a more effective test of the “directiondependent” model. The results of this study provide evidence to support the hypothesis that the CNS takes the directiondependent precision of visual and proprioceptive localization into account when integrating these two types of information. This implies that the CNS has knowledge about the directiondependent precision of the visual and proprioceptive information.
METHODS
Model
We will propose a model that describes how the localization of one’s hand on the basis of simultaneous visual and proprioceptive information is related to proprioceptive and visual localization in isolation. The outcome of the localization of a hand can be described by two quantities: the mean position at which the hand is localized and the precision of this mean position. We will use variances to describe this precision because the reciprocal of the variance in a signal is a useful measure of the information in that signal (Fisher 1966). For localization in the horizontal plane, one single variance is not sufficient to describe the precision because the variance depends on direction for both vision and proprioception (van Beers et al. 1998). The mean position and the directiondependent variance, which together describe the outcome of the localization, can be described concisely by twodimensional normal distributions. In general, an ndimensional normal distribution can be written as (e.g., Winer et al. 1991)
Figure 1 shows an example of confidence ellipses of such distributions for visual localization [P _{V}(x)] and for proprioceptive localization [P _{P}(x)] of the right hand in the left hemifield of a horizontal plane (we used target positions in this area, see Fig. 3). The confidence ellipse of a twodimensional spatial probability distribution consists of spatial locations of equal probability. The narrower an ellipse is in a certain direction, the more precise the hand is localized in this direction. The relative locations of the two ellipses in the figure reflect what generally is found in experiments like the present one: when the unseen right hand is the target that has to be matched with the unseen left hand, subjects usually point too far to the right (Crowe et al. 1987; Slinger and Horsley 1906) and when a visual target has to be matched with the unseen left hand, an overreach in the forward direction often is found (de Graaf et al. 1995; Foley and Held 1972). The shapes, the relative sizes, and the orientations of the two ellipses in Fig. 1 are chosen to comply with what was found by van Beers et al. (1998).
The distribution for bimodal localization can be found by multiplying the two distributions for unimodal localization. It can be shown that the product of two normal distributions is again a normal distribution (after normalization). The parameters describing the bimodal distribution P
_{PV}(x) are related to those of the two unimodal distributions as follows:
The bimodal ellipse derived from the two unimodal ellipses shown in Fig. 1 is indicated in the same figure. It is striking that the center of the bimodal ellipse lies off the straight line through the centers of the two unimodal ellipses. To understand this, one needs to consider the direction of the minor axis of the proprioceptive ellipse. In this direction, proprioceptive localization is more precise than visual localization. Accordingly, in this direction more weight will be attached to proprioception than to vision. Similarly, a larger weight will be attached to vision in the direction of the minor axis of the visual ellipse. As a result, the predicted center of the bimodal ellipse lies off the straight line through the centers of the unimodal ellipses as sketched in Fig. 1. How far the predicted center of the bimodal ellipse lies off the straight line depends on the characteristics of the unimodal ellipses. In general, the more the major axes are perpendicular to each other and the more eccentric the ellipses, the larger the deviation will be. However, when the center of one unimodal ellipse lies in the vicinity of the extended part of the major axis of the other unimodal ellipse, the deviation is small. The relation between the relative locations of the two unimodal ellipses and the predicted position of the bimodal ellipse is illustrated in Fig. 2. The figure shows that the predicted center of the bimodal ellipse generally lies on the same side of the straight line through the centers of the two unimodal ellipses as the intersection of (the extended parts of) the major axes of these two ellipses. It can be shown that this applies to all possible relative locations of the two unimodal ellipses, except for a very small number of relative locations where the center of one unimodal ellipse is on, or very close to, (the extended part of) the major axis of the other unimodal ellipse.
The way in which the information is combined can be described as weighting of the visual and proprioceptive information, the weights depending on the spatial direction. This can be seen explicitly by considering the variances in any arbitrary direction in the horizontal plane. It can be shown that in each direction, the reciprocal of the bimodal variance is equal to the sum of the reciprocals of the two unimodal variances in that direction. For onedimensional normal distributions, the available information is used with an optimal efficiency in such a weighting (Ghahramani et al. 1997). This shows that the model describes the situation in which two twodimensional normal distributions are combined such that the available information is used optimally. It therefore is conceivable that the model predicts bimodal ellipses that are generally smaller than those predicted by the much coarser method with fixed, i.e., directionindependent, weights (the model tested in van Beers et al. 1996). The present model therefore may explain the small variances reported in van Beers et al. (1996) that remained unexplained by the model tested in that study.
To test the proposed model, we verify one of its striking predictions, namely the prediction for the mean positions. To derive reliable predictions for these means, the characteristics of the unimodal distributions have to be known with a certain degree of accuracy for each subject. However, these characteristics cannot be determined with sufficient accuracy for individual subjects as an estimation reveals that this would require each subject to participate for >100 h. We therefore will base the analysis on the results of our previous study (van Beers et al. 1998). For proprioception, we found that the variance in the radial direction with respect to the shoulder was smaller than the variance in the azimuthal direction. For visual localization, the variance was smaller in the azimuthal direction with respect to the cyclopean eye than in the radial direction. We used these origins (the shoulder and the cyclopean eye) because they make sense physiologically. However, we could not demonstrate that the ellipses are indeed oriented toward these origins. The use of slightly different origins, such as the projection of the body midline on a horizontal plane, would describe the results equally well. In the analysis of the present study, we will assume that the major axes of visual ellipses point toward the cyclopean eye and that the major axes of proprioceptive ellipses are perpendicular to the imaginary straight line to the shoulder. Because we have no strong support for this assumption, we also will analyze the data using the body midline projection as the origin for both vision and proprioception. From the assumed orientations and from the positions at which each subject indicated in the unimodal conditions, we will derive where the model predicts the mean of the bimodal condition to lie. This prediction, then, will be compared with where the subject actually pointed.
Subjects
Ten subjects (1 woman, 9 men, aged 19–50 yr), who previously had given their informed consent, participated in this experiment. All subjects were naive as to the purpose of the experiment. The research was performed in accordance with the regulations of the Commissie Mensproeven i.o. of Delft University of Technology. All subjects had normal or correctedtonormal vision and no one had any history of neuromuscular disorders.
Setup
Subjects were seated on a chair at a table the top of which was 92 cm above the floor. The chair and the table could not be moved during the experiment. The table top was covered with a bright yellow sheet of cardboard. The positions of the targets were indicated by small pieces of black cardboard on this sheet. Because the pieces of cardboard could be felt and seen by the subjects, they could be used to define target positions both proprioceptively (by placing the right hand on them) and visually. We used three target positions which were all in the subjects’ left hemifield and relatively far in front of them (see Fig. 3). Assuming the cyclopean eye and the right shoulder as origins for the ellipse orientations, the major axes of the visual and the proprioceptive ellipses were expected to be approximately perpendicular in this area (with the body midline as origin, this relation is exact). The predicted deviations from the straight line were therefore close to maximal for these target positions.
We used an OPTOTRAK 2010 system to record the indicated positions. This system measures the threedimensional coordinates of an infrared emitting diode (IRED) with a precision of 0.5 mm in all directions. An IRED was placed on the nail of the left index finger.
Procedure
At the beginning of each trial, subjects had to put both hands in starting positions. The left hand was placed just in front of the subject, slightly to the left of the subject’s midline, on a support 10 cm below the table (see Fig. 3). The starting position for the right hand was on the table top, slightly to the right of the subject’s midline. In all conditions, subjects had to match the target position with their unseen left hand touching the underside of the table. The conditions differed with regard to the information about the target position.
CONDITION P.
The task was to match the position of the unseen right hand on the table top (a proprioceptive target) with the unseen left hand on the underside of the table. After the experimenter had presented the target number, subjects closed their eyes and moved the tip of their right index finger to the appropriate piece of cardboard so that this fingertip defined the target position. Thereupon the subjects moved their left hand to match the position of the right hand. Subjects did not open their eyes until the indicated position had been recorded. One could argue that this condition is not a purely proprioceptive matching task because subjects saw the target position at the beginning of each trial and thus also could use visual memory to position their left hand. However, we have evidence that the subjects did not rely on visual memory to position their hands in this condition: they quickly found a target with their right hand after having closed their eyes, but in many cases a wrong target. Only after moving their right hand over the cardboard to find the other targets (which they did spontaneously) did they rest their right hand on the, now correct, target. This indicates that visual memory did not provide useful information about the target position.
CONDITION V.
Subjects used their unseen left hand on the underside of the table to match the position of a visual target. They moved their left hand after the experimenter had presented the target number. The right hand was not used. It remained at its starting position.
CONDITION PV.
Subjects matched the position of the tip of the index finger of the seen right hand on the table top (target position available by both proprioception and vision) with the unseen left hand on the underside of the table. After the experimenter had presented the target number, subjects moved their right hand to the appropriate piece of cardboard. Thereupon the subjects moved their left hand to match the position of the seen right hand.
In all conditions the task was to match the target position as accurately as possible. Usually, subjects first made a relatively fast movement to bring the left hand close to the target position, whereupon the eventual matching was achieved by some slower movements. The total adjustment usually took a few seconds. When satisfied that the left index finger matched the target position, the subject told the experimenter so. The subject then did not move the finger until its stationary position had been recorded. The subjects were asked not to move their trunk with respect to the chair during the entire experiment. Small movements, however, could not be prevented. Head movements were allowed; subjects always fixated the target when it was presented visually. We did not immobilize any body part for two reasons. First, in the analysis we made use of the results of a previous study (van Beers et al. 1998), and these were obtained in conditions where the subject’s body was not restrained. Second, restraining body parts can cause subjects to behave differently than in natural conditions (Steinman et al. 1990).
In each series, every target position was presented five times in each condition, providing a total of 45 trials per series. The order of conditions and targets was randomized, so subjects did not know the condition and the target of a trial until the experimenter had presented them.
Before the actual experiment started, subjects performed three practice trials in each condition to get accustomed to the tasks. In the experiment proper, each subject performed three series. In the first series, subjects had normal vision during trials of conditions V and PV. In the second series, they wore prism goggles with a strength of 11 diopter that displaced the visual field 6.3° to the left. In the third series, rightward displacing prisms of the same strength were worn. We had subjects perform two series wearing prism goggles to vary the positions indicated in conditions P and V relative to one another. This was interesting because the model predicts that the magnitude of the deviation from the straight line strongly depends on these relative positions (see Fig. 2). We did not want prism adaptation to take place during these series because that would increase the variability in the indicated positions. Therefore, we had subjects adapt to the prisms prior to each prism series: subjects looked through the prisms at their right hand and arm, which they moved between the starting position and the target positions for ∼10 s. Although small movements during prism exposure with simultaneous vision of starting and target locations do not produce adaptive aftereffects, aftereffects usually are found when the distance between starting and target locations is of a magnitude that necessitates eye movements between initial and target positions (see Redding and Wallace 1996). In our setup, the distance between starting and target positions spanned ∼40° of the visual field. Therefore we expected adaptation to occur. The results show that the positions indicated in the conditions Pand V during a prism series are shifted relative to those indicated in the series with no prisms (see results). This shows that prism adaptation indeed had occurred during the exposure period before the prism series. We assumed that the adaptation was as good as saturated at the end of this period because adaptation took place under circumstances with errorcorrective feedback and with concurrent vision of the hand. Adaptation has been shown to be completed quickly under these circumstances (von Helmholtz 1925; Welch 1978). The absence of drift in the indicated positions (which would indicate adaptation during the experimental series) showed that these series indeed were performed in a state of approximately unchanging adaptation. In this adapted state, both proprioceptive position sense of the right hand and the visual system may have been adapted. Although the left hand could not be seen at any time when the subject wore prism goggles, there even may have been transfer of adaptation to this hand (e.g., Cohen 1967; Hamilton 1964). The exact site of the adaptation, however, was not relevant as we were interested only in the differences in the positions indicated in the various conditions.
Each subject thus performed three series of 45 trials. Each series took ∼10 min. After these series, we recorded the real target positions. For each subject, we also recorded the positions of the cyclopean eye and the right shoulder. These positions were used to determine the predictions of the model.
Analysis
We tested the model by analyzing where subjects pointed incondition PV relative to where they pointed inconditions P and V. We performed the following analysis for each target in each series for each subject. We determined the mean of the five positions that were indicated in each of the three conditions. Next, we assumed that the precision of visual localization can be described by a confidence ellipse the major axis of which is directed toward the cyclopean eye. To visualize this, we drew a straight line with this orientation through the mean ofcondition V. Likewise, we assumed that the precision of proprioceptive localization of the right hand can be described by a confidence ellipse the major axis of which is perpendicular to the imaginary straight line through the position of the hand and the right shoulder. We drew such a line through the mean of condition P. To test the model, we analyzed whether the mean ofPV was on the same side of the straight line through the means of the two unimodal conditions as the intersection of the two drawn lines. The effect of the few relative locations for which this method is not correct (see the model description) is expected to be marginal because the predicted deviation from the straight line is small in comparison with the scatter in indicated positions for these relative locations.
Because we had 10 subjects who matched three target positions in three series, we compared the predicted side with the observed side for 90 cases. We used a onesided sign test to test the null hypothesis that the number of correct predictions is equal to or smaller than the number of incorrect predictions.
In addition to this “binary” analysis, we also determined the distances of the mean of PV from the straight line through the means of P and V for all 90 cases. We made a histogram of these distances and used a onesided ttest to check whether the observed distribution of distances could correspond to the mean of PV lying on the straight line through the means of P and V.
To test the proposed directiondependent model in a more direct way, we performed similar analyses of the location of the mean ofcondition PV with respect to the curvedline predicted by the model. This curved line can only be determined when the two unimodal distribution are known. As argued in themodel section, we could not derive these distributions for our individual subjects. Therefore we used the means of the results of the 10 subjects in van Beers et al. (1998) for the target that was closest to the targets in the present study. For proprioceptive localization, we assumed that the variance in the radial direction was 48 mm^{2}, whereas a variance of 64 mm^{2} was assumed in the azimuthal direction. For visual localization, these values were 10 and 23 mm^{2}, respectively. From these distributions and from the means ofconditions P and V, we determined the curved lines predicted by the model. We determined at which side of this curved line the mean of condition PV lay. This could be either “inside” the curved line, i.e., on the side of the straight line, or “outside” the curved line. A twosided sign test was used to test the nullhypothesis that the numbers of means ofPV inside and outside the curved line were equal. We also analyzed the distances of the mean of PV from the curved line similar to how we analyzed the distances from the straight line.
To investigate whether the choice of ellipse orientations was crucial for the interpretation of our results, we repeated the complete analysis using slightly different orientations. We now used the body midline as the origin for both modalities.
Note that the localization of the indicating, left hand is not used in the analysis. Because the indicator was the same in all conditions, we assume that all observed differences between conditions result from differences in target localization. The uncertainty of localization of the left hand adds only to the scatter in the indicated positions. Note also that different sensorytomotor transformations were required in different conditions. It might be possible that this influenced the results, but because we have no arguments suggesting that it would (and if it would it is not clear how), we did not take such possible influences into account.
RESULTS
Figure 4 A shows an example of the positions that one subject (SK) indicated in the series without prism goggles. Like most subjects, this subject overreached the targets in condition V, which is consistent with earlier observations (de Graaf et al. 1995; Foley and Held 1972). In condition P, this subject pointed slightly too far to the right and overreached the target. Most subjects pointed too far to the right in this condition, which is in agreement with the “overlap effect” reported previously in similar experiments (Crowe et al. 1987; Slinger and Horsley 1906).
The positions indicated in condition PV relative to those of conditions P and V are judged more easily in Fig. 4 B, which shows the means of the indicated positions shown in Fig. 4 A. This figure also shows the lines that indicate an assumed orientation of the major axes of the ellipses for visual localization (condition V, the lines point toward the cyclopean eye) and for proprioceptive localization (condition P, the lines are perpendicular to the imaginary straight line to the right shoulder) of the targets. For each target, the intersection of these two lines indicates on which side of the straight line through the means of P and Vthe model predicts that the mean of PV will lie. In this case, the intersections are to the left of this line for all three targets. The observed mean is indeed on this side for all targets.
When subjects wore prism goggles, they all noticed that the visual field was affected, but none of them understood in what way it was affected. Inquiries after the experiment revealed that no subject had used a cognitive strategy to compensate for the experienced distortion. Figure 4, C and D, show the results of the series in which leftward displacing prism goggles were worn. As expected, incondition V in this series all subjects pointed more to the left than in the first series. To a lesser extent, this also was observed for condition P for all subjects. This shift in condition P reflects the adaptation of proprioceptive position sense that had occurred during the exposure period in which the moving right hand was viewed through the prisms before the series. The shift was to the left because the adaptation diminished the conflict between the two senses. Because there also may have been some adaptation of proprioception about the position of the left hand, the shift actually reflects the difference between the adaptation for the right hand and the left hand. For the subject whose results are shown in Fig. 4, the predicted deviations from the straight line are larger in this series than in the series without prisms. For all target positions, we did indeed find a relatively large deviation on the predicted side of the straight line through the means ofP and V.
The results of the series with the rightward displacing prisms are shown in Fig. 4, E and F. In this series, we found a rightward shift of the positions indicated in conditions V and P for all subjects. The shift incondition V was always larger than the shift incondition P. The deviations from the straight line predicted by the model for this subject are small in comparison with those of the series with the leftward displacing prisms. The intersections are to the right of the straight line through the means of P and V for all targets. For one of the targets, the observed mean lies on the opposite side of this line. For the two other targets, the predicted side is correct.
For the subject of Fig. 4, the model prediction was correct in eight of nine cases. For all subjects, the prediction was correct in 61 of the 90 cases. The middle column of Table 1shows how the numbers are distributed over the various series. For all series, the number of correct predictions is larger than the number of incorrect predictions. We tested the null hypothesis that the number of correct predictions was equal to or smaller than the number of incorrect predictions. A sign test shows that this hypothesis has to be rejected (P < 0.001). The results therefore refute the hypothesis that the mean of condition PV lies on the straight line through the means of conditions P andV or on the side opposite to the one predicted by the model. The null hypothesis has to be rejected also on the basis of only the series with no prisms (P < 0.025). For the series with the leftward displacing prisms, the hypothesis has to be rejected as well (P < 0.01). No significant effect was found for the series with the rightward displacing prisms.
We also analyzed the distances of the mean of PVfrom the straight line through the means of P andV. Figure 5 A shows this distance for all 90 cases. In this figure, a distance was chosen to be positive when the mean of PV lay on the side predicted by the model. This figure shows that the distribution of distances is biased in the predicted direction (P < 0.001, twosided ttest); this confirms the results of the binary analysis.
Both types of analysis thus show that a directionindependent (or straight line) model has to be rejected. To examine whether the directiondependent (or curved line) model provides a better fit for the data, we also determined on which side of the curved lines the means of PV lay. For this purpose, we had to make assumptions about the visual and proprioceptive ellipses (seeAnalysis). The righthand column of Table 1 shows that the mean of PV was inside this line in 40 of the 90 cases. Sign tests show that the hypothesis that the mean of PV lies on the curved line cannot be rejected for the total of all 90 cases nor for any of the individual series. Figure 5 B shows the distances from the curved line, where a positive distance denotes a mean that lies outside the curved line. This figure shows a smaller bias than Fig. 5 A. Statistical analysis proved that the bias from the straight line was indeed larger than the bias from the curved line (P < 0.001, onesided paired ttest). The bias in the latter case is not significantly different from zero (P > 0.2, twosided ttest).
Because we have no evidence that the ellipses are oriented as we assumed in the aforementioned analysis, we repeated the analysis using different orientations. We now assumed that the major axis of the visual ellipse points toward the body midline and that the major axis of the proprioceptive ellipse is perpendicular to the imaginary straight line to the body midline. The results of this analysis are very similar to those of the previous analysis. The directionindependent model has to be rejected on the basis of both the binary analysis (P < 0.001, onesided sign test, see Table 2) and the analysis of the distances (P < 0.001, twosided ttest, see also Fig. 5 C). The binary analysis also shows that the directiondependent model cannot be rejected for the total of all 90 cases nor for any of the individual series (Table 2). The bias that was present in the distances from the straight line (Fig. 5 C) has again been reduced by taking the distances from the curved line (P < 0.001, onesided paired ttest), and the resulting bias does not differ significantly from zero (P > 0.05, twosided ttest, see also Fig.5 D).
DISCUSSION
In this study, we have investigated in what location a hand is localized on the basis of simultaneous visual and proprioceptive information relative to the positions at which it is localized on the basis of vision only and proprioception only. The results indicate that a seen hand is generally not localized on the straight line through the two positions where it is localized in the unimodal situations. Instead the location is predicted better by a model that describes this multisensory integration as a directiondependent weighting of the proprioceptive and the visual information, the weighting being related to the precision of the unimodal information.
Validity of the proposed model
Before discussing the implications of this study, we want to make sure that the results do not rely on an artifact caused by the subject’s posture. We are concerned specifically about whether the result that subjects indicated at different locations in different experimental conditions could have been caused by the subjects having had different postures in different conditions (this could have happened for instance because the right hand was at the starting position in condition V, whereas it was at the target position in conditions P and PV). If such differences were present, their effect on the position of the mean ofPV relative to the means of P and V is expected to be in the same direction in all three series. A comparison of Fig. 4, B, D, and F, reveals that this is not the case. In the series with the leftward displacing prisms the mean of PV is shifted further to the left from the straight line through the means of P and V than in the series with no prisms; in the series with the rightward displacing prisms, it is shifted to the right. Obviously, this is different from the constant effect expected from conditiondependent postures. We conclude that the significant effects we found do not rely on this artifact.
The hypothesis that the seen hand is localized somewhere on the straight line between the positions at which it is localized on the basis of vision only and proprioception only is refuted by the present data. This is direct support for the proposed model but an alternative explanation could be that the CNS does not represent locations in Cartesian coordinates but in another coordinate system, such as spherical or jointbased coordinates. Interpolating between two locations in such a system and then converting the locations to Cartesian coordinates would produce the interpolated location to lie off the straight line through the outer two locations. Although it may be possible to find a coordinate system that is in agreement with our results, it seems unlikely that such a system also makes sense physiologically. The two abovementioned systems that do make sense physiologically do not agree with the data as can be concluded from Fig. 4 using the same argument as in the previous paragraph.
The results show that the mean of condition PV was more often on the side predicted by the proposed curved line model than on the opposite side. This suggests that this model gives a better description of the results than a straight line model would do. This result cannot be due to using prism goggles because the effect was also significant for the series with no prisms. The fact that we found no significant effect for the series with the rightward displacing prisms is another indication of the validity of the model: the relative locations of the means of conditions P andV in this series were frequently comparable to situation E in Fig. 2, so that only very small deviations from the straight line were to be expected. In these situations, the predicted deviations were very small compared with the scatter in the indicated positions. As a result, relatively many incorrect predictions (i.e., observed means that lay opposite to the side predicted) were to be expected in this series, which is indeed what we found with rightward displacing prisms. In general, the relatively large number of incorrect predictions (29 of 90) does not disprove the model. Such a number is to be expected from the scatter in the indicated positions and from the uncertainty in the actual ellipse orientations (compare the width of the distribution of distances with its bias in Fig. 5 A).
More direct evidence in favor of the curved line model is provided by comparing the distances of the means of condition PVfrom the straight and curved lines, respectively. Whereas we found a significant bias relative to the straight line, this bias disappears when we analyze the data relative to the curved line. Because no significant bias resulted, we conclude that the curved line model gives a good quantitative prediction of the position in which a seen hand is localized.
Reanalyzing the data using slightly different orientations of the visual and proprioceptive ellipses produced basically the same results. This indicates that the exact ellipse orientations were not crucial and that it was legitimate to use estimated orientations.
The constant errors found in the present study are thus in good agreement with the predictions of the model. We cannot use the results of this study to test the variable errors predicted by the model because the precision of unimodal localization cannot be derived from these results (that would require an experiment as in van Beers et al. 1998). However, evidence that the model also gives a correct description of the variable errors can be found in van Beers et al. (1996). In that study, we found that the positions indicated in a condition with both proprioception and vision showed less scatter than was predicted by a directionindependent model. The difference between predicted and observed variance was found to vary among the target positions used. It varied qualitatively in a way that is predicted by the model proposed in the present paper. For instance, we found the largest difference for the target to the subject’s left (target 1 in that study). The model indeed predicts a large directiondependent effect in this area because here the visual and proprioceptive ellipses are expected to be relatively elongated and their major axes will be approximately perpendicular. In conclusion, both the constant errors found in the present study and the variable errors found in van Beers et al. (1996) provide evidence in favor of a model like the one proposed here.
Implications of the model
What conclusions can be drawn from this study about how the CNS processes simultaneous proprioceptive and visual information about hand positions? We found that a seen hand is generally not localized on the straight line through the two positions where it is localized on the basis of vision or proprioception only. This implies that the CNS uses directiondependent weights when combining information from the two modalities. We could predict on which side of the straight line the mean of the bimodal condition would lie on the basis of the directiondependent precision of visual and proprioceptive localization. This is strong evidence that the weights are related to the directiondependent precision of the information in each modality. The weights are chosen in such a way that the available information is used very efficiently, i.e., more efficiently than if the weighting were directionindependent. Our results do not support the hypothesis that the weights are determined by the attention directed to each modality: this hypothesis cannot explain the observed deviations from the straight lines. However, this does not rule out the possibility that attention can influence the weights used (for an example, seeWarren and Schmitt 1978).
The finding that the weights used are related to the precision of the information implies that the CNS has knowledge about this directiondependent precision. It would be interesting to know how the CNS obtains this knowledge. There seem to be at least two possible ways. First, the CNS may have learned the precision from experience, for instance, from the errors occurring in reaching movements, which also may be direction dependent. This would suggest that the knowledge is stored somehow. Second, the precision may be derived instantaneously from sensory signals. For example, the hand’s position in space might be derived directly from proprioceptive signals such as muscle spindle output. To do this, the CNS must be able to transform the muscle spindle output into spatial coordinates. When this transformation is made, noise in the muscle spindle output is translated directly into directiondependent uncertainty in the derived hand position. The results of this study do not distinguish between these two possibilities. Note that both mechanisms will have approximately the same result because the precision of localization is determined primarily by the geometry of the sensory system and the density of sensory organs (as argued in van Beers et al. 1998), irrespective of the way in which the knowledge about the directiondependent precision is obtained.
Another question is how the integration of multisensory information as described by the model actually is accomplished by the CNS. Although neural nets may implicitly carry out such computations, the CNS not necessarily performs an actual multiplication of two probability distributions. Another possibility is that the CNS processes the available information in a way that minimizes some error, for instance, the errors made in reaching movements. If this minimization acts near optimally, which may be achieved after sufficient experience, the system’s performance will be reasonably well described by our model. The model thus may describe the overall performance of the CNS rather than the computations it actually performs.
It is possible that the proposed model, or a generalized version that also minimizes variance, is also applicable in other situations, such as involving other modalities and in threedimensional space. This also is suggested by the results of Ghahramani et al. (1997), which provide evidence that a similar model would describe correctly the integration of visual and auditory position information in the azimuthal direction with respect to the head. A more generalized version of the model may even be applicable in many other situations, e.g., situations involving multisensory integration of information about orientation, velocity, shape, etcetera. It is interesting to note that the principle of minimizing of variance also may be on the basis of the planning of goaldirected eye and arm movements (Harris and Wolpert 1998).
Finally, it also would be interesting to know how the spatial information is processed and represented at the neural level. A possibility is that spatial probability distributions of spatial locations as we used to describe the outcome of localization are actually represented in the CNS. Neurophysiological studies (e.g.,Georgopoulos et al. 1984, 1986; Helms Tillery et al. 1996; Kalaska and Crammond 1992) have shown that quantities such as hand positions and movement directions are represented by the activity of large populations of cells. Because large numbers of cells are involved, not only a single value of the represented quantity can be derived but possibly also the reliability of this value. It therefore might be possible to interpret the activity of a population of cells as a spatial probability distribution. Theoretical studies (Anderson 1994; Sanger 1996; Zemel et al. 1998) have indicated that this may indeed be possible and that it may be a very efficient and useful way to represent spatial information. It therefore would be interesting to see whether spatial probability distributions indeed can be derived from neurophysiological data.
Acknowledgments
We thank J. Smeets and E. Brenner for helpful discussions about the experiment and for putting the prism goggles at our disposal. We thank D. Wolpert and Z. Ghahramani for suggesting an elegant mathematical representation, S. Gielen and an anonymous reviewer for useful suggestions on earlier versions of this paper, and S. McNab for improving the language and clarity of the paper.
Footnotes

Present address and address for reprint requests: R. J. van Beers, Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London WC1N 3AR, United Kingdom.

The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
 Copyright © 1999 The American Physiological Society
Appendix
A twodimensional normal distribution can be written as (e.g.,Winer et al. 1991)