To the Editor:
The paper by Tollin and colleagues in the March 2005 issue of the Journal of Neurophysiology describes the sound-localization ability of cats trained to orient their eyes to the source of a sound (Tollin et al. 2005). The main finding of this paper, which was the subject of an Editorial Focus (Sparks 2005), was that cats are extremely accurate in directing their eyes to the source of a sound when their heads are unrestrained. For example, Tollin and colleagues state that cats are able to localize a 15-ms noise burst in the horizontal plane with a mean signed error of 0.16° (±0.97° SD), an accuracy that they say “… is comparable, and perhaps even superior” to that of humans and the barn owl. If so, this would make cats the best sound localizers of any terrestrial mammal. However, before drawing this conclusion, it is important to consider the sound localization acuity of cats, that is, their ability to discriminate between the locus of two sounds (i.e., minimum audible angle). When tested on their ability to discriminate between two sound sources centered around their midline, where acuity is greatest, cats have an average 50% threshold of around 5° (Heffner and Heffner 1988). Although comparatively good, their acuity is exceeded by at least five other species, including domestic pigs (4.5°) and humans (1.2°) (Heffner and Heffner 2003). We suggest that the differences in the results of the two procedures, particularly in how they rank different species, lie in the calculation of accuracy in the orientation task.
Tollin and colleagues calculated localization accuracy by taking the mean signed error. This measure of average accuracy reveals only whether an animal had a systematic bias to overshoot or undershoot the target. Consider, for example, two animals, one with a minimum audible angle of 5°, the other with a minimum audible angle of 20°. As long as both animals orient equally to the left and right of the target, even if one has a spread of ±2.5° and the other a spread of ±10°, each will receive the same mean signed error score of 0°. Yet the first animal clearly has a more accurate perception of locus than the second. Moreover, consider a third animal that consistently orients 1° to the left of the target; although its individual orientations are more accurate than most of those of the first two animals, its mean signed error of 1° makes it appear worse than either. In short, mean signed error is not a good estimate of an animal’s ability to perceive locus and, although it is technically a definition of accuracy, it is never used as a measure of accuracy in contests of marksmanship. Instead, it is more meaningful to use either the absolute error or some measure of the range of the scores as an indication of sound-localization accuracy.
In addition to using the mean signed error as a measure of accuracy, Tollin and colleagues arrive at the value of 0.16° for azimuthal localization accuracy by averaging the results of the three cats they tested. However, as can be seen in their Fig. 4, one cat (cat 18) had a moderate tendency to undershoot the target, whereas the other two cats had smaller tendencies to overshoot (cats 17 and 21). Thus the undershooting of one cat was cancelled out by the overshooting of the other two. However, in reality, an individual cat is often required to locate the source of a sound based on a single occurrence of that sound and does not have the opportunity either to determine an average location based on multiple occurrences of the sound or to confer with other cats. Thus a horizontal signed error of 0.16° does not give a true picture of the ability of a cat to localize sound in the horizontal plane.
We agree that the orientation procedure can be a good way to study sound-localization ability and that Tollin and colleagues have significantly improved on the procedure by measuring eye position in animals whose heads are not restrained. However, it will probably not work with all animals because, as noted by Gordon Walls (1942), “… unless one spot of the retina is clearly superior to the rest in resolving power, there is no advantage in aiming any one part of the retina at the object of interest.” Indeed, the size of the region of the retina with the best resolving power appears to be functionally related to localization acuity. Animals with narrow fields of best vision, such as humans, localize more accurately than animals with wider fields, such as cats, who in turn localize more accurately than animals with visual streaks, such as horses and cattle (Heffner and Heffner 1992, 2003). Relying on the mean signed error as a measure of accuracy would conceal this relationship because virtually all animals will have a score of near 0° and it would become apparent only when some measure of the spread of the animals’ orientation responses was taken into account. Finally, although we agree that looking at the source of a sound is a natural response, it is not the only one. Animals often freeze at the occurrence of a sound, a response that can be easily used to assess sound-localization acuity (Heffner and Heffner 1995).
- Copyright © 2005 by the American Physiological Society
To the Editor: In their letter regarding our paper (Tollin et al. 2005), Heffner and Heffner dispute our claim that the sound localization accuracy of cats is “… comparable, and perhaps even superior, to that…” of humans and barn owls on the basis that our measures of accuracy are not consistent with prior measures of localization acuity in cats. Their argument stems from the fairly common misconception that the acuity of a sensory system (or any measurement device for that matter) is related to its accuracy. In this letter we show that precision of localization is more likely related to acuity. We should first make clear what is meant by these terms: accuracy describes the closeness of a measurement to the true value, whereas precision describes the consistency of the measurement or the degree to which several measurements provide similar answers.
Behavioral studies of sound localization generally use one of two psychophysical procedures. Relative procedures assess the acuity or spatial-resolving power of the localization system by measuring the minimum audible angle (MAA; Mills 1958), the smallest angle separating two sources that can be discriminated. Absolute procedures measure the actual ability to indicate the sound source location, which is quantified in terms of accuracy and precision. Although both methods purport to measure something about localization capabilities, it is, in fact, unknown how or even whether measures of acuity such as the MAA are related to measures of localization accuracy and precision. Despite this uncertainty, it is often believed that acuity can be informative of accuracy (e.g., small MAAs predict accurate localization; large MAAs predict poor accuracy).
To be fair, the misconception is a reasonable one because in daily usage the terms accuracy, precision, and—at least in psychophysics—acuity are often used interchangeably. However, two simple thought experiments show that acuity and accuracy need not be related at all, but rather that acuity is likely related to localization precision. First, suppose 100 darts are thrown at a dartboard, a process analogous to a cat localizing auditory targets by gaze shifts. Figure 1A shows four possible scenarios. 1) The bull’s-eye is hit virtually every time (top left). This means that the throws were both accurate [that is, the average location was close to the target (bull’s-eye)] and precise (that is, the location of the throws was consistent and reproducible regardless of accuracy). 2) The throws had the exact same consistency as before, but each throw misses to the left (top right). These throws were not accurate because, on average, the bull’s-eye was missed, but were still precise because the throws were highly reproducible. 3) Sometimes dart throws scored a direct hit within the bull’s-eye, but the rest were scattered evenly around the board (bottom left). These throws were accurate because the mean throw location was within the bull’s-eye but not precise. 4) The throws had the same consistency as that in the third example but miss to the left (bottom right). These throws were neither accurate nor precise. The point is that accuracy and precision capture two different aspects of darts and sound localization ability and they need not be related. Of course, different metrics can be used to quantify accuracy and precision; we used mean signed error and the SD of the responses about the mean, respectively. Other metrics, such as absolute error and range suggested by Heffner and Heffner, might also have been used, but this would not change the fundamental distinction between accuracy and precision. Note that the absolute error, or mean unsigned error, by itself will not convey whether the throws are undershooting or overshooting.
By attempting to find a meaningful interpretation of our estimate of cat sound localization accuracy and their estimate of acuity, Heffner and Heffner fall prey to the common misconception outline above. In fact, their own example of the three animals with different localization capabilities illustrates this. The first two animals have “spreads” (which we interpret here as a variability measure such as the SD) in their localization of ±2.5 and ±10° with a mean signed error of 0°, whereas a third animal consistently orients 1–2° to the left of the target. Heffner and Heffner claim that “… the first animal clearly has a more accurate perception of the locus than the second,” yet these two animals actually have identical accuracy, but different precision (e.g., Fig. 1A, left column). Furthermore, they say that the third animal is more accurate than the other two. In reality, the third animal was the least accurate of all, but the most precise (e.g., Fig. 1A, top right)!
The fundamental questions here are whether one can draw meaningful conclusions about localization accuracy and precision from measures of localization acuity, such as the MAA, and ultimately whether the same neural mechanisms are even used in these disparate tasks. Although we believe that acuity cannot inform us about localization accuracy, we think it can be informative about localization precision. To illustrate this, Fig. 1B shows another thought experiment where 100 darts were thrown at each of two different targets, in turn, that were separated in azimuth. The example on top shows throws that are accurate but not precise. With respect to sound localization, let’s suppose that the location of each dart throw was an exact copy of our perception of the target location on that trial (for simplicity, we ignore the contribution to accuracy and precision from the motor act of throwing). Thus the location of a throw directly represents the perceived location on a given trial. Because we threw many darts, we can estimate the effective distribution of the internal spatial representation of azimuth by forming a histogram of the response locations (shown beneath schematic). Suppose that we were then asked to discriminate the left target from the right target based only on these internal distributions. Clearly, good discrimination can occur only if the distributions are significantly separated and therefore it is the variability, or precision, of the internal representations of the two targets that limits the minimum angle that can be discriminated. If we were to require nearly perfect discrimination, then for the variability shown in Fig. 1B (top), the angle between the bull’s-eyes could correspond to one measure of the MAA. The bottom example in Fig. 1B shows throws that are not accurate, because the bull’s-eyes were never hit, but are very precise. Reflecting this precision, the internal distributions must have been narrow (Fig. 1B, bottom) and a similar degree of overlap between the distributions would allow a smaller angle separating the two targets to be discriminated and a smaller MAA. This example shows that excellent acuity, as assessed by the MAA, could theoretically occur even though the targets themselves are not accurately localized. Therefore acuity need not correlate with accuracy and is more likely proportional to precision.
Examples of the discrepancy between acuity and accuracy like that described above are found throughout the literature. For example, the classic measurements of MAAs for pure tones or narrow-band sounds can be as small as 1–2° for sources along the horizontal plane (Mills 1958) yet the average localization error for similar stimuli can be nearly an order of magnitude greater because these sounds are seldom localized accurately (e.g., Middlebrooks 1992; Stevens and Newman 1936). The concept that measures of localization precision and acuity are related is not a new one by any means, although acuity is still often confused with accuracy. The cats in our experiments were indeed accurate at localizing sounds in azimuth with an average signed error across cats of 0.16°. However, it is their precision that we should compare to measures of acuity. In short, the average precision of localization in azimuth as measured by SDs about the mean horizontal response was 3.91° (see Table 2 in Tollin et al. 2005) and this is comparable to the acuity of cats, as measured by the MAA, which generally ranges from about 3 to 6° (Heffner and Heffner 1988).