HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) All Versions of this Article: 94/2/1252    most recent 00949.2004v1 Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (7) Citing Articles via Google Scholar Google Scholar Articles by De Souza, W. C. Articles by Ono, T. Search for Related Content PubMed PubMed Citation Articles by De Souza, W. C. Articles by Ono, T.

Differential Characteristics of Face Neuron Responses Within the Anterior Superior Temporal Sulcus of Macaques

Department of Physiology, Faculty of Medicine, Toyama Medical and Pharmaceutical University, Toyama, Japan

Submitted 13 September 2004; accepted in final form 27 March 2005

	ABSTRACT

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
The anterior superior temporal sulcus (STS) of macaque monkeys is thought to be involved in the analysis of incoming perceptual information for face recognition or identification; face neurons in the anterior STS show tuning to facial views and/or gaze direction in the faces of others. Although it is well known that both the anatomical architecture and the connectivity differ between the rostral and caudal regions of the anterior STS, the functional heterogeneity of these regions is not well understood. We recorded the activity of face neurons in the anterior STS of macaque monkeys during the performance of a face identification task, and we compared the characteristics of face neuron responses in the caudal and rostral regions of the anterior STS. In the caudal region, facial views that elicited optimal responses were distributed among all views tested; the majority of face neurons responded symmetrically to right and left views. In contrast, the face neurons in the rostral region responded optimally to a single oblique view; right-left symmetry among the responses of these neurons was less evident. Modulation of the face neuron responses according to gaze direction was more evident in the rostral region. Some of the face neuron responses were specific to a certain combination of a particular facial view and a particular gaze direction, whereas others were associated with the relative spatial relationship between facial view and gaze direction. Taken together, these results indicated the existence of a functional heterogeneity within the anterior STS and suggested a plausible hierarchical organization of facial information processing.

	INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
In the process of identifying familiar faces, incoming perceptual information regarding facial stimuli must be compared with the long-term memory of faces or that of people in general. In a preceding paper (Eifuku et al. 2004), we reported that, in macaque monkeys, two anterior temporal cortical areas, i.e., the anterior superior temporal sulcus (STS) and the anterior inferior temporal gyrus (ITG), play different roles in the process of identifying familiar faces. The population of neurons in the anterior STS responded to faces with selectivity for viewing angle, and these neurons are thought to be closely associated with analyzing incoming perceptual information from faces, whereas the population of neurons in the anterior ITG represented facial identity and was essentially involved in the recognition of facial identity. These findings were consistent with previous findings concerning neuronal recordings in both anesthetized monkeys and in alert monkeys during the performance of passive viewing tasks (Perrett et al. 1982, 1985). Moreover, the results of our previous study were consistent with those of studies involving monkeys performing face discrimination tasks (Hasselmo et al. 1989; Young and Yamane 1992). Thus it appears that the functional roles of the anterior STS and the anterior ITG differ, although these roles might nonetheless be complementary. In this study, we focused exclusively on the functional organization of the anterior STS.

Neurons that are selectively responsive to the sight of faces, i.e., "face neurons," were originally identified in the anterior STS (Bruce et al. 1981; Perrett et al. 1982). To date, these neurons have been recorded in various regions of the monkey brain (Baylis et al. 1985; Desimone et al. 1984; Hasselmo et al. 1989; Nakamura et al. 1992; O'Scalaidhe et al. 1997; Perrett et al. 1985; Rolls 1984; Yamane et al. 1988). The responses of face neurons in the anterior STS of macaque monkeys are tuned to facial views and/or to gaze direction in the faces of other individuals (Perrett et al. 1985, 1992). The so-called "face space" composed by the population of face neurons in the anterior STS represented facial views (Eifuku et al. 2004). It has been reported that when recognizing familiar objects (or faces), humans usually use "view-specific representations" rather than "structural descriptions" (Bulthoff and Edelman 1992; Edelman and Bulthoff 1992; Logothetis et al. 1995; Tarr 1995). The selectivity for facial views among the face neurons in the anterior STS, already discussed in previous studies, is consistent with the notion that view-specific representations are preferentially employed for the recognition of objects.

Previous studies have shown that anatomical architecture and connectivity differ between the rostral and caudal regions of the anterior STS. The rostral region of the lower bank of the anterior STS receives input primarily from the anterior inferior temporal gyrus (ITG), which is known to be related to short-term and long-term visual memory (Naya et al. 2001). The caudal region of the anterior STS receives input from the posterior ITG, as well as from the intraparietal sulcus (IPS), posterior parahippocampal areas (areas TF and TH), and prestriate areas, including area V4 (Barnes and Pandya 1992; Saleem et al. 2000; Seltzer and Pandya 1978, 1984, 1994). The IPS is involved in the processing of three-dimensional structures (Sakata et al. 1997; Taira et al. 2000; Tsutsui et al. 2002).

In contrast to our current understanding of this anatomical heterogeneity, the functional differences between the rostral and caudal regions of the anterior STS are not well understood. The purpose of this study was to investigate the various characteristics of face neuron responses in the rostral and caudal regions of the anterior STS; a related aim of this study was to elucidate the functional heterogeneity of these responses within the anterior STS. Thus we analyzed the view-tuning curves of face neuron responses; here, we focused on optimal views for each neuron, as well as on right-left symmetry or asymmetry, and certain differences were observed between the rostral and caudal regions. In addition, we studied the modulation of face neuron responses according to the direction of gaze in the observed face, i.e., the modulation of neuronal responses to simulated eye contact directed toward the subject, or to gazes averted away from the subject. This approach was used because the direction of a gaze is an important determinant of the biological significance of an observed face. The results, taken together, might help achieve an understanding of the type of information processing specific to the anterior STS.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
Subjects

Three adult female macaque monkeys (Macaca fuscata) weighing 4.5–7.5 kg were used in this experiment. The neuronal activity detected in four hemispheres was recorded. All experimental protocols were approved by the Animal Care and Use Committee of Toyama Medical Pharmaceutical University, and all protocols conformed to the National Institutes of Health guidelines for the human care and use of laboratory animals.

Behavioral task

The monkeys were trained to perform a version of a sequential delayed matching-to-sample task that requires the identification of a face (I-DMS task; Fig. 1 A); this behavioral task was the same as that described in our preceding paper (Eifuku et al. 2004). In the I-DMS task, a sample (480 ms) stimulus was presented to the animal after fixation on a small point, and test (match or nonmatch, 480 ms) stimuli were subsequently presented to the subject after a period of interstimulus delay (992 ms). Eye position was monitored using the scleral search coil technique, and the size of the eye control window was ±2.0° (Judge et al. 1980). The sample face was always presented in the frontal view (0°). The match stimuli consisted of images of the faces of people with whom the monkeys were already familiar; these people were laboratory staff involved in the daily care of the subjects (Fig. 1B). Two types of facial stimuli (regular set) were used as the match stimuli in the initial experiments. In the first type (Fig. 1, Ba and Bc), the stimulus was one of seven faces viewed from one of seven different angles (from left to right profile: –90, –45, –22.5, 0, +22.5, +45, and +90°). Furthermore, the direction of the gaze shown on the viewed face matched that in which the face pointed (Face = Gaze). The second type of facial stimuli used for the regular set was employed for the investigation of the effects of the direction of the depicted gaze (Fig. 1, Bb and Bd). In this second type of stimulus, one of five views was used (from left to the right: –45, –22.5, 0, +22.5, and +45°), and the gaze was always projected toward the subject; thus the point of view presented to the subject gave the impression of eye contact with the subject (simulated eye contact). The frontal faces with simulated eye contact (stimuli 4 and 15) were common to the two types of facial stimuli. Each monkey was required to identify the same person who had been shown in the sample; if the test stimulus was a match, the monkey was to push a lever. A number of intervening (nonmatch) stimuli was presented until a match finally appeared (0–3 intervening stimuli); all correct trials were rewarded by the administration of juice (0.2 ml). Various geometric patterns served as the control stimuli (Fig. 1C). All visual stimuli were presented within the center of the receptive field of each recorded neuron, each of which had been mapped in advance of the experiment. In addition, the stimuli were typically centered on the fixation point, and the size of the images was 10–15 x 10–15°.

View larger version (51K): [in this window] [in a new window]   FIG. 1. A: delayed matching-to-sample task based on an identification (I-DMS) task. This task was a version of a sequential delayed matching-to-sample task; a sample (480 ms) was presented after each monkey fixated on a small point (FP; 0.2° diam) that appeared at the center of the display. Test (match or nonmatch; 480 ms) stimuli were presented to each subject after an interstimulus delay (992 ms). Intervening (nonmatch) stimuli were presented 0–3 times until a match finally appeared. Sample faces were always presented in the frontal view (0°), and the gaze shown in the sample faces always projected toward the subject, whereas a test face was chosen from images of various faces viewed from various angles; the gaze in some of the test faces was projected toward the subject and that in other test faces was pointed away from the subject (see also B). Animals were required to identify the same person shown in the sample, and if the test stimulus was a match, the monkey was trained to push a lever to obtain juice. Eye position was monitored using a scleral search coil during the I-DMS task, and the size of the eye control window was ±2.0°. B: facial stimuli (regular set). Faces of 2 persons (identities) were used as match test stimuli; facial stimuli consisted of 22 faces that differed with respect to facial view, facial identity, or gaze direction, as shown in examples depicted in Ba–Bd. In stimuli sets shown in Ba and Bc, direction of the gaze was identical to direction in which the face was pointed (Face = Gaze); here, the faces were viewed from 7 different angles (from the left to right profile: –90, –45, –22.5, 0, +22.5, +45, and +90°). In stimuli sets shown in Bb and Bd, the gaze was always projected toward the subject (i.e., simulated eye contact); these faces were viewed from 5 different angles (from the left to right profile: –45, –22.5, 0, +22.5, and +45°). The frontal face was the same facial stimulus as that shown in Ba and Bc and in Bb and Bd. Facial images represented laboratory staff members who were familiar to each monkey and who were involved in the daily care of the subjects. Visual stimuli were presented as follows: 256 gray-scale, 10–15 x 10–15° in size, and at the center of the display with FP. All of the stimuli were thereby within the receptive fields that were mapped before the experiments. C: geometric patterns. Neutral geometric patterns were used as control stimuli. Visual stimuli were also presented as follows: 256 gray-scale, 10–15 x 10–15° in size, and at the center of the display with FP.

Electrophysiological procedures

The procedures used for electrophysiological recording and data analysis have been described in detail in our preceding paper (Eifuku et al. 2004). In brief, these procedures were carried out as follows.

During the experiment, a grid was placed within the recording cylinders (Crist et al. 1988) to facilitate the insertion of stainless steel guide tubes through the dura to a depth of 10 mm above the STS. At the beginning of each recording session, the guide tube stylet was removed and an epoxy-coated tungsten microelectrode (FHC, 1.0–1.5 M at 1 kHz) was inserted. The electrode was advanced using a stepping microdrive, while neuronal activity was monitored to establish the relative depth of the landmarks, including the layers of gray and white matter, and to determine the properties of the neuronal responses. For all monkeys, we used 3D-MRI rendering to place an electrode into the anterior STS (Asahi et al. 2003). The positions of the anterior STS and of the recording sites were checked by MRI during the experiment, and these MR images included a marker (tungsten, 500 µm diam); we verified the calculated recording sites with reference to the coordinate of the marker. It was confirmed by 3D-MRI rendering that all of the tracks of the inserted microelectrodes were almost vertical to the orbito-meatal (OM) plane for each monkey.

While conducting the I-DMS task, we recorded neuronal activity from the anterior STS. In this study, we analyzed the neuronal activity of single neurons during a period of 64–496 ms after the onset of a match (the lag time of 64 ms was based on the minimum response latency of the neurons). Control firing was measured in the 208-ms period before when the sample was presented to each monkey. Because an overtrained monkey could perform the I-DMS task correctly when test stimuli were presented for <240 ms, it appeared that the entire period of 480 ms might not have been required for solving the task. However, to reduce variation in neuronal activity across different trials, we presented the monkeys with test stimuli for a duration of 480 ms and analyzed the neuronal activity that occurred during the 64- to 496-ms period after the onset of a match.

Histological procedures

After the final recording session, several small marking lesions were created in the anterior STS by passing a 20- to 30-µA anodal current for 40 s through a tungsten microelectrode; this procedure was monitored by MR imaging. Each animal was deeply anesthetized with an overdose of pentobarbital sodium (50 mg/kg, im) and perfused transcardially with heparinized 0.9% saline followed by 10% buffered formalin. The brains were removed and cut into 50-µm coronal sections through the target areas with a freezing microtome. Sections were stained with cresyl violet, and all sites marked by an electrical lesion were carefully verified microscopically. The location of each recording site was calculated by comparing the stereotaxic coordinates of the recording sites with those of the lesions. MR images obtained during the experiment were compared with those showing the marking electrodes to verify the calculated recording sites. The reconstruction of the recording sites based on histological investigation and MRI confirmed that all of the responses of the face neurons used for this analysis were recorded from the anterior STS in the range of 10–24 mm anterior to the interaural line; most of these face neurons were located in the lower bank and fundus of the anterior STS.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
General

The behavioral performance of the subjects on the I-DMS task was better than 95% correct during the neuronal recording experiments; the analysis of neuronal activity included only the data obtained from correct trials. In our preceding paper (Eifuku et al. 2004), we behaviorally characterized this I-DMS task. Consistent with the results of that study, the behavioral reaction times during the correct trials in this study ranged between 350 and 680 ms. In addition, behavioral reaction times tended to be proportional to the difference between the sample and match stimuli with respect to the angle of the facial view, as was also observed in our previous report.

A total of 255 neurons exhibited significantly increased activity in response to the presentation of match stimuli during the I-DMS task (visually responsive neurons; paired t-test, P < 0.05); the match stimuli were either faces or geometric patterns, as those shown in Fig. 1, B and C. Neurons that exhibited both significant responses during the match period, as well as significantly larger responses to faces than to geometric patterns, were defined as face neurons in this study; neuronal responses to an optimal face were statistically compared with those to an optimal geometric pattern (t-test, P < 0.05). According to this definition, 152 (59.6%) of 255 visually responsive neurons were classified as face neurons. The selectivity of neuronal responses to faces was further analyzed by two-way ANOVA (factors: facial view and facial identity, i.e., the facial view included 7 views in the 1st set of regular faces, whereas the facial identity included 2 different identities; significance level: P < 0.05). Of these 152 face neurons, 131 (86.2%) exhibited responses to facial view (as opposed to facial identity). Thirty-two (21.1%) and 49 (32.2%) of the 152 face neurons showed significant responses to facial identity (as opposed to facial view), as well as showing significant interactions between facial views and facial identity, respectively. The 131 face neurons showing a significant effect in response to facial views were further analyzed.

In this paper, we focused on neuronal responses to match stimuli. However, the majority of responses of face neurons to the presentation of nonmatch faces did not significantly differ from those to match faces [126 (82.9%), t-test, P > 0.05] when identical faces were used as nonmatch or match stimuli. Only 15 (9.9%) and 11 (7.2%) neurons showed an enhancement (nonmatch > match) or a suppression (nonmatch < match) of response, respectively (t-test, P < 0.05). These results were in contrast to our earlier findings regarding the anterior ITG face neurons (Eifuku et al. 2004), and they were also in contrast to the results of other previous studies (Miller et al. 1991, 1993, 1996; Riches et al. 1991) of the anterior ITG.

Optimal facial views for face neuron responses

Figure 2 provides an example of the responses detected in a single face neuron, the activity of which was recorded from a relatively caudal portion of the anterior STS; the rostro-caudal coordinate of this face neuron was 12 mm anterior to the interaural line. Neuronal responses to match faces, when the facial view and the direction of the gaze in the presented face were equivalent, are shown in Fig. 2A. Figure 2B summarizes the facial-view tuning of responses of this particular face neuron. Moreover, in the case of this particular face neuron, the effect of facial views was significant [2-way ANOVA factors: facial view and facial identity; F(6,144) = 30.252, P < 0.0001], whereas neither facial identity [F(1,144) = 1.027, P = 0.3126] nor the interaction between facial view and facial identity [F(6,144) = 1.203, P = 0.3079] was found to be significant. This neuron responded best to profiles within a range of –90 to +90° (Newman-Keuls test, P < 0.05), and the responses showed right-left symmetry across the midline (0°), as if the facial-view tuning curve were the mirror image. There was no difference between neuronal responses to profiles (–90 or 90° view) of different facial identities (Fig. 2C; t-test, P > 0.05). It should be noted that the activity of the neuron described in Fig. 2 offers a representative example of a neuron located in the relatively caudal portion of the anterior STS.

View larger version (25K): [in this window] [in a new window]   FIG. 2. An example of a face neuron in the caudal region of the anterior superior temporal sulcus (STS). A: neuronal responses to a match face, when the direction in which the face pointed was identical to the direction of the gaze. Responses to 7 different views (from left to right, –90 to +90°) are displayed in rasters and by spike density functions (SPDs: SD = 10 ms) that are aligned to the onset of match (time 0). Different colors in the rasters indicate 2 different identities. Solid lines in the graphs indicate the mean firing rates during the control period (i.e., 208-ms period before the sample faces were presented to the monkeys) ± 2 SD. B: tuning to view angles of faces. In the line graph, neuronal firing during the 64- to 496-ms period after onset of a match (means ± SE) was compared across view angles. Blue line indicates condition under which the gaze direction and face direction were the same. Red line indicates the condition with simulated eye contact. C: selectivity to facial identity. In the columnar graph, neuronal firing during a 64- to 496-ms period after onset of a match (means ± SE) is compared across face identities. Solid and dashed lines in B and C indicate mean firing rates in the control period ± SE. Neuron selectively responded to facial profiles and produced responses that were symmetrical to the right and the left profiles.

View larger version (36K): [in this window] [in a new window]   FIG. 3. Optimal facial views. A: location of face neurons in anterior STS and their optimal views for tuning to particular facial views. Location of each face neuron was identified by histological investigation, and MRI pictures were taken during the experiment. Coronal sections including the recording sites are displayed in the rostro-caudal order, from 22 mm (or more) anterior (A 22, left top) to the interaural line (0), to 12 mm (or less) anterior (A 12, right bottom) to that line. B: percentages calculated for optimal facial view for rostral and caudal regions of anterior STS are compared. C: optimal facial view and selectivity index (SI) of each face neuron in the sample was plotted; each dot represents an individual neuron. D: percentages of optimal facial view for rostral and caudal regions of anterior STS are compared; data shown here represent only face neurons for which SI exceeded 0.33 (n = 47 for rostral region and n = 38 for caudal region).

We then retrospectively compared the percentages of the optimal facial views for particular face neuron responses in the rostral group and the caudal group. The optimal facial view was the view that elicited the largest magnitude of responses to different facial views. It should be noted that in this analysis, the data obtained for two different identities were averaged. In the pie charts shown in Fig. 3B, it can be seen that the proportion of optimal facial views differed significantly between the rostral and the caudal groups (² test, P < 0.01). In the caudal group, the face neurons showed selectivity to all views, including profiles, and there were no significant differences in the proportion of cells tuned to different views (² test, P > 0.05). However, in the rostral group, a significant difference was observed in the proportion of cells tuned to different views (² test, P < 0.05). The percentage of neurons tuned to oblique views of the face was largest compared with the percentages obtained in response to other views.

All of the face neurons that showed significant effects in response to facial views were used for the analysis summarized in Fig. 3B. Thus face neurons that were broadly tuned to facial views were also included in the sample. We analyzed the relationship between the optimal facial views and selectivity in terms of facial-view tuning. To quantify the selectivity in terms of tuning, we used a selectivity index (SI) defined by the following equation: SI = (R_max –R_min)/(R_max + R_min), where R_max and R_min are the maximum and minimum responses in the facial-view tuning curve, respectively. For both values, the control firing value was subtracted from the neuronal firing value obtained in response to a match face. The SI assumes values in the range of 0–1, where 1 represents the highest selectivity to facial views and 0 the lowest. The scatter plot in Fig. 3C shows the relationship between the optimal facial views and the SIs of all face neurons tested, indicating that the distribution of SIs was independent of the optimal facial view. There were no significant differences in the mean SI between the rostral and caudal populations (t-test; P > 0.05). The mean of the SIs for the rostal population (n = 70) was 0.533 ± 0.029 (SE), whereas that for the caudal population (n = 60) was 0.499 ± 0.031. In Fig. 3D, we provide another comparison of the percentages of the optimal facial view between the rostral group and the caudal group of the anterior STS; only face neurons for which the SI exceeded 0.33 (i.e., R_max > 2R_min) are included in this figure. Thus this figure exclusively depicts the behavior of face neurons with a relatively higher level of selectivity in terms of facial views. Again, in the caudal group, the face neurons showed selectivity for all views, including profiles, and no differences were observed with respect to neuronal responses to different views of faces (² test, P > 0.05). However, in the rostral group, a significant difference was observed in neuronal responses to different views of faces (² test, P < 0.05). The proportion of optimal facial views differed significantly between the rostral and the caudal groups (² test, P < 0.01). In addition, as regards optimal facial views, the difference between the rostral and caudal groups of face neurons in the anterior STS was consistently observed in all of the subjects.

Right-left symmetry/asymmetry of face neuron responses

We frequently recorded responses of face neurons, the facial-view tuning of which was right-left symmetrical across the midline (0°; see the example shown in Fig. 2). In Fig. 4, the frequency of right-left symmetry is indicated, together with the frequency of asymmetry among face neuron responses along the rostro-caudal axis in the anterior STS. In this analysis, face neurons that appeared to preferably respond to oblique views or profiles were included (n = 53, rostral; n = 43, caudal), and the magnitude of responses to the first or second type of facial stimuli in the original set were compared. For the quantification of right-left symmetry among face neuron responses, we used a tuning symmetry index (TSI), which was defined as follows: TSI = (R_max –R_opp)/(R_max + R_opp), where R_max is the maximum response in the facial-view tuning curve, i.e., the response to the optimal facial view, and R_opp is the response to the facial view shown from a perspective opposing that of the optimal view. For both values, the control firing value was subtracted from the neuronal firing value obtained in response to the match face. The TSI accepts values in the range of 0–1; when the TSI approached 1, a greater degree of asymmetrical shaping in the facial-view tuning curves was observed, and when the TSI approached 0, a greater degree of symmetrical shaping of the facial-view tuning curves was observed. In the scatter plot shown in Fig. 4A, the relationship between the rostro-caudal coordinates and the TSIs of face neurons in the sample is given; each dot represents an individual neuron in this plot. The TSI and the rostro-caudal coordinate of the face neurons revealed a significant positive correlation (r = 0.319, P < 0.005), indicating that the facial-view tuning curves of the caudal population of face neurons tended to show right-left symmetry, whereas those of the rostral population tended to show right-left asymmetry.

View larger version (23K): [in this window] [in a new window]   FIG. 4. Tuning symmetry/asymmetry. A: rostro-caudal coordinate (reference: interaural line) and tuning symmetry index (TSI) of each face neuron in the sample was plotted; each dot represents an individual neuron. B: percentages of symmetry or asymmetry in the facial-view tuning curves were compared between the rostral and caudal regions of the anterior STS.

Modulation of face neuron response according to gaze direction

Figure 5, A–C, provides an example of face neurons belonging to the rostral group; the rostro-caudal coordinate of this face neuron was 22 mm anterior to the interaural line. Neuronal responses to all of the 22 match faces in the original set are displayed in Fig. 5A. For this particular face neuron, the effect of facial views was significant [2-way ANOVA, factors: facial view and facial identity, F(6,109) = 8.111, P < 0.0001], but neither the effect of facial identity [F(1,109) = 0.366, P = 0.5465] nor the interaction between facial view and facial identity [F(6,109) = 0.362, P = 0.9013] was significant. Figure 5B summarizes the facial-view tuning of the face neuron depicted in Fig. 5A. The face neuron shown here responded best to a –45° view of the face (Newman-Keuls test, P < 0.05). Two-way ANOVA (factors: facial view and eye contact, the facial view included ±45 and ±22.5° views, whereas the eye contact included two subordinate conditions, i.e., Face = Gaze and simulated eye contact; significance level: P < 0.05) revealed that the effect of facial view [F(3,119) = 11.701, P < 0.0001] and the effect of eye contact [F(1,119) = 8.003, P < 0.01], were significant, but the interaction between facial view and eye contact [F(3,119) = 1.611, P = 0.1903] was not significant for the particular neuron shown in Fig. 5, A–C. It should be noted in this context that the face neuron response to the optimal facial view (–45°) was significantly facilitated by simulated eye contact (t-test, P < 0.01); we therefore referred to the type of neuron responding to such facilitation as an excitation type of neuron. The best response of this particular face neuron was observed in association with the left oblique view simulating eye contact. There were no significant differences between neuronal responses to the –45° views of two different identities (Fig. 5C; t-test, P > 0.05). The activity of the neuron described in Fig. 5, A–C offers a representative example of that recorded in the relatively rostral part of the anterior STS. Other examples of face neurons in the rostral group that were influenced by simulated eye contact are shown in Fig. 5, D and E. The activity of the neuron in Fig. 5D responded best to a –45° view of the face. For this particular neuron, two-way ANOVA (factors: facial view and eye contact) revealed that the effect of facial view [F(3,125) = 3.315, P < 0.05], the effect of eye contact [F(1,125) = 7.868, P < 0.01], and the interaction between facial view and eye contact [F(3,125) = 5.426, P < 0.05] were all significant. Responses to the optimal facial view were significantly suppressed by simulated eye contact (t-test, P < 0.005); we referred to this type of neuron responding to such suppression as an inhibition type of neuron. The neuron in Fig. 5E showed an asymmetrical modulation of responses under the simulated eye-contact condition; significant facilitation of responses occurred in association with right views of the face (t-test, P < 0.05), whereas significant suppression of responses occurred in association with left views (t-test, P < 0.05). We thus referred to this type of neuron showing such facilitation and suppression as an asymmetrical-effect neuron. Moreover, in the case of this particular face neuron, the effect of facial views [2-way ANOVA factors: facial view and eye contact; F(3,121) = 27.819, P < 0.0001] and the interaction between facial view and eye contact [F(3,121) = 5.426, P < 0.005] were significant, but the effect of eye contact was not significant [F(1,121) = 0.126, P = 0.7228].

View larger version (33K): [in this window] [in a new window]   FIG. 5. Examples of a face neuron in the rostral region of the anterior STS. A: example of face neurons in the rostral anterior STS. Neuronal responses to all of the 22 match faces are displayed in rasters and by spike density functions (SPD: SD = 10 ms) that are aligned to the onset of a match (time 0) stimulus. Configuration of this figure is the same as that shown in Fig. 2A, except that neuronal responses to the match face in the condition (simulated eye contact) are displayed in the bottom panels; responses to 5 different views (from left to right, –45 to +45°) are displayed in the bottom panels. Different colors shown in the rasters indicate 2 different identities. Solid lines in the graphs indicate mean firing rates during the control period (i.e., the 208-ms period before sample faces were presented to the monkeys) ± 2 SD. B: tuning to facial view angles of the face neuron depicted in A. In the line graph, neuronal firing during the 64- to 496-ms period after onset of a match (means ± SE) was compared across view angles. Blue line indicates condition under which direction of the gaze and direction in which the face pointed were the same. Red line indicates condition with simulated eye contact. This neuron responded best to a –45° view of faces; neuronal response was facilitated by simulated eye contact (Excitation Type). C: selectivity for facial identity of the face neuron depicted in A. Neuronal firing during the 64- to 496-ms period after onset of a match (means ± SE) was compared across face identities. Neuron exhibited no selectivity with respect to facial identity. D: another example of a face neuron in the rostral anterior STS. Neuronal firing during the 64- to 496-ms period after onset of a match (means ± SE) was compared across view angles. Blue lines indicate condition under which gaze direction and face directions were the same. Red lines indicate condition with simulated eye contact. Neuron responded best to a –45° view of the faces. Responses decreased with exposure to simulated eye contact (Inhibition Type). E: another example of a face neuron in the rostral anterior STS. This neuron exhibited an asymmetrical modulation of responses under simulated eye-contact condition; excitation occurred in association with the right views, whereas inhibition occurred in association with the left views (Asymmetrical Effect).

View larger version (43K): [in this window] [in a new window]   FIG. 6. Gaze modulation effects on face neuron responses. A: location of face neurons exhibiting or lacking the 3 types of gaze effects in the anterior STS. Coronal sections including recording sites are displayed in the rostro-caudal order, from 22 mm (or more) anterior (A 22, left top) to the interaural line (0), to 12 mm (or less) anterior (A 12, right bottom) to that line. B: respective percentages of the 3 types of gaze effects are shown within the rostral (A > 16) and caudal (A 16) regions of the anterior STS.

Influence of averted gaze

In the examinations carried out up to this point, gaze modulation was studied only in cases involving oblique faces such as those depicted in Figs. 5 and 6. However, the presence of gaze modulation still needed to be examined in terms of faces presented from views other than the oblique view. To this end, we carried out additional experiments using an additional set of facial stimuli and additional samples of face neurons in the anterior STS.

In this series of experiments, the faces of two persons (identities) were used as match test stimuli; the facial stimuli consisted of images of 18 faces that differed with respect to facial view or gaze direction, such as those depicted in Fig. 7, A–F (hyper set). As shown in Fig. 7, A and D, the stimuli set of gazes was pointed in the same direction as that in which the face was pointed (Face-Gaze); the faces were viewed from five different angles (from left to right profile: –90, –45, 0, +45, and +90°). This stimuli set was similar to that depicted in Fig. 1, Ba and Bc. As shown in Fig. 7, B and E, the stimuli set of faces depicted the gaze as consistently projected toward the subject (simulated eye contact); these faces were viewed from three different angles (from left to right: –45, 0, and +45°). This stimuli set was similar to that depicted in Fig. 1, Bb and Bd. In Fig. 7, C and F, the stimuli set of faces was always pointed toward the subject; under this condition, the direction of the gaze was pointed in three different angles (from left to right: –45, 0, and +45°). We referred to this latter facial stimulus set as the averted gaze condition. The frontal faces with simulated eye contact (stimuli 3 and 12) were common to all three types of facial stimuli.

View larger version (62K): [in this window] [in a new window]   FIG. 7. Facial stimuli (hyper set). Facial views of 2 persons (identities) were used as match test stimuli; facial stimuli consisted of 18 images that differed with respect to facial views or gaze direction, as shown in the examples given in A–F. In A and D, stimuli set shows direction of the gaze as identical to direction in which the face was pointed; these faces were viewed from 5 different angles (from left to right profile: –90, –45, 0, +45, and +90°). In B and E, stimuli set shows gaze always projected toward the subject (i.e., simulated eye contact); these faces were viewed from 3 different angles (from left to right: –45, 0, and +45°). In stimuli set shown in C and F, the face was always pointed toward the subject; direction of gaze pointed in 5 different angles (from left to right: –45, 0, and +45°; averted gaze condition). The frontal face was the same facial stimulus (3 and 12) as that shown in A and D, B and E, and C and F.

Figure 8Aa summarizes the facial view/gaze direction-tuning of a face neuron in the caudal group (14 mm anterior to the interaural line). The data for two different identities were averaged, because there were no differences between the neuronal responses to these two identities. The effect of simulated eye contact as well as the effect of averted gaze on face neuron responses was further analyzed by two-way ANOVA (factors: view condition and face/gaze condition; the view condition included ±45° views, whereas the face/gaze condition included 3 subordinate conditions, i.e., Face = Gaze, simulated eye contact, and averted gaze; significance level: P < 0.05). In the case of the particular neuron depicted in Fig. 8A, the effect of view [F(1,92) = 58.256, P < 0.0001], the effect of face/gaze [F(2,92) = 155.666, P < 0.0001], and the interaction between the view condition and the face/gaze condition [F(2,92) = 6.168, P < 0.005] were all found to be significant. This neuron responded to the frontally presented faces, regardless of whether they simulated eye contact or showed an averted gaze; moreover, the presentation of averted gazes did not diminish the intensity of responses of this neuron. Responses to the frontal views showing simulated eye contact were compared with those showing an averted gaze in the columnar graph provided in Fig. 8Ab, and the results did not indicate any significant effects associated with gaze (t-test: P > 0.05). However, oblique views of faces with simulated eye contact significantly diminished the magnitude of response (t-test: P < 0.005).

View larger version (26K): [in this window] [in a new window]   FIG. 8. Influence of averted gaze. Aa: tuning to facial views of a face neuron in the caudal anterior STS. Neuronal firing during the 64- to 496-ms period after onset of a match (means ± SE) was compared across view angles. Open squares indicate condition under which the gaze direction and face direction were the same (Face = Gaze). Crosses indicate condition with "simulated eye contact." Filled circles indicate condition showing "averted gaze." Abscissa of line graph shows facial view or gaze direction. Solid and dashed lines indicate the mean firing rates in the control period ± SE. Ab: summary of gaze modulation of response of the face neuron in Aa. Average responses to the following are compared: frontal views with simulated eye contact (0°), oblique views with simulated eye contact with averted gaze (±45° face), and frontal views with averted gaze (±45° gaze). Control firing value was subtracted from neuronal firing value obtained in response to a match face. This neuron responded to frontally presented faces, whether or not they involved simulated eye contact or an averted gaze; averted gaze was not found to diminish the responses of this neuron, whereas faces presented in an oblique manner but with simulated eye contact significantly diminished the responses of this neuron. Ba: tuning to facial views of a face neuron in the rostral anterior STS. Bb: summary of gaze modulation of the response of the face neuron in Ba. This neuron responded to any view showing simulated eye contact, whether or not the view was frontal or oblique; however, the representation of an averted gaze significantly diminished the responses of this neuron. C: tuning to facial views of another face neuron in the rostral anterior STS. This neuron responded to 2 particular combinations of facial views and gaze directions, i.e., the frontal view (0°) with the +45° averted gaze, and the –45° facial view with simulated eye contact. Note that relative spatial configuration between facial views and gaze directions are the same between the 2 face-gaze configurations.

In the caudal region, only 5 of 12 neurons tested (41.7%) significantly changed their responses to the averted gaze series of facial stimuli, whereas in the rostral region, 9 of 11 neurons tested (81.8%) changed their responses to this series. The percentage of gaze modulation was larger in the rostral region than in the caudal region (² test, P < 0.05).

Figure 8C summarizes the facial view/gaze direction-tuning of yet another face neuron in the rostral group (22 mm anterior to the interaural line). Two-way ANOVA (factors: view and face/gaze) revealed that the effect of the face/gaze condition [F(2,90) = 59.876, P < 0.0001] and the interaction between the view condition and the face/gaze condition [F(2,90) = 179.643, P < 0.0001] were significant, but the effect of view [F(1,90) = 3.035, P = 0.0849] alone was not significant for this particular neuron. This neuron responded to two combinations of facial views and gaze directions; the first was a combination of the frontal view (0°) and the +45° averted gaze, and the other was a combination of the –45° facial view and simulated eye contact. It should be noted that the relative spatial configuration between the facial views and the gaze directions were the same between the two face-gaze configurations. The results suggested that the relative spatial configuration was encoded in the responses of this face neuron. Four of 14 neurons tested (1 of 5 in the caudal region and 3 of 9 in the rostral region) responded to different facial view/gaze direction combinations, the relative spatial configurations of which were the same with regard to the respective facial views and the gaze directions.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
In this study, 56% of the visually responsive neurons were classified as face neurons. This percentage was larger than those obtained in previous studies of face neurons (Hasselmo et al. 1989; Perrett et al. 1982). However, our sample might have been somewhat biased, for a number of reasons. First, we analyzed neurons that showed only significant visual responses to match faces in the I-DMS task. The previous studies used a passive viewing task or a simple visual discrimination task. Thus differences in behavioral tasks between the present and previous studies might be reflected by differences in neuronal activities. Second, all of the recording sites in this study were localized exclusively in the lower bank and fundus of the anterior STS, whereas in the previous studies, activities were recorded from the anterior STS including the upper bank. Third, fewer nonface stimuli, which served as a control in this study, were used here than in previous studies of face neurons (Hasselmo et al. 1989; Perrett et al. 1982). In this context, we would like to point out that our definition of face neurons might have been broader than those used in the previous studies. At the same time, we would also emphasize that the face neurons analyzed here exhibited a significant effect in response to facial views (Figs. 2 and 5). Thus the pattern of the view-tuning of face neuron responses was analyzed in depth in this study.

Neuronal responses in the rostral region of the anterior STS

Oblique facial-view neurons (Fig. 5) were typically found among the rostral population of neurons located relatively anterior to the neurons discussed in previous reports (Hasselmo et al. 1989; Perrett et al. 1982, 1985). A major question concerning the processing of facial representations focuses on determining which view, if any, is used as the "canonical view" for faces (Bruce 1988; Perrett et al. 1991; Valentin et al. 1997). The psychological data suggest that the ±45° (3/4) view is advantageous (Bruce et al. 1987; Fagan 1979; Krouse 1981; Logie et al. 1987), whereas the physiological data thus far suggest that higher numbers of face neurons are tuned to frontal or profile views than to the 3/4 view (Perrett et al. 1985, 1991, 1992). A previous report suggested that the advantage of the 3/4 view, according to physiological results, derived from the partial activation of face neurons tuned to various views. To confirm this explanation, a scheme using a distributed representation was proposed, i.e., the simultaneous partial activation of frontal and profile neurons might mediate the advantage of the 3/4 view (Valentin et al. 1997). Some reports have suggested the presence of neurons selectively responsive to a particular view across the entire 360° range of views (Perrett et al. 1992). However, our results indicated simply the dominance of the 3/4 view as the optimal view for face neurons in the rostral region of the anterior STS; in this region, the responses clearly showed the best direction for the identification of a face, i.e., these neurons preferred the 3/4 view. Because the 3/4 view reveals a relatively large number of features, it may prove to be more useful for recognition tasks than either frontal or profile views. Furthermore, provided that the 3/4 view maximizes the number of encoded features, it is more likely to promote an enhanced recognition of faces presented from this perspective. The neuronal preference for 3/4 facial views and the lack of right-left symmetry in the view-tuning curves among the face neuron responses in the rostral region both imply that this region represents either the +3/4 face (right oblique view) or the –3/4 face (left oblique view), and the information stored might be used at later stages for face recognition or identification. Because the proportion of optimal facial views did not differ significantly between the +3/4 (right) and the –3/4 (left) views, it is likely that both facial views were equally represented in the population of face neurons in the rostral regions.

Gaze direction and orientation of the face toward or away from the viewer are important social and communicative signals in both humans and monkeys (Argyle and Cook 1976). In contrast to the corresponding responses in the caudal region, in the rostral region, the neuronal responses to faces appeared to be frequently modulated according to the direction of the gaze shown in the viewed face. Moreover, simulated eye contact with the subject easily changed the tuning of the face neurons. These results were observed not only in cases involving faces presented in an oblique manner (Fig. 5), but similar results were also obtained when frontal views of the face were used (Fig. 8). Direction of gaze is a determinant of the biological significance of faces as communicative signals. In other words, the biological significance of a face either making eye contact with a subject or avoiding such contact appears to be differentially reflected by the effective modulation of the response of face neurons in the rostral, but not caudal, region. These findings suggest a plausible functional hierarchy in information processing based on the biological significance of faces.

Gaze modulation in some face neurons, whether it is a case of facilitation, as in the examples in Fig. 5, A–C, or a case of inhibition, as in the example in Fig. 5D, implies that facial view signals and gaze direction signals are not independent of each other in the rostral region. This line of reasoning suggests that face neuron responses are selective for a particular combination of a particular facial view and a particular direction of the gaze of another individual. However, in the case of other neurons, the curves for facial-view tuning differed from those for gaze direction tuning, as seen in the example in Fig. 5E (asymmetrical effects); facial view signals and gaze direction signals therefore appear to be independent of each other in the case of these neurons. Previous studies of evoked potential using humans have suggested an early separation of face and gaze signals in the processing streams used for face perception (Allison et al. 1999; McCarthy et al. 1999; Puce et al. 1999; Shibata et al. 2002).

Neuronal responses to a particular facial view/gaze direction combination

Some face neurons, as those depicted in Fig. 5, A–C, showed responses to a particular combination of a particular facial view and a particular gaze direction shown the faces presented to the subjects. The responses of the majority of these face neurons did not reflect the relative spatial configuration between the facial views and the gaze directions (Fig. 8B), because the observed neuronal responses differed significantly between the two face-gaze configurations that were in the same relative spatial configuration with respect to the facial views and the gaze directions. However, it was also found that a small number of face neurons (Fig. 8C) were able to encode the relative spatial configuration between the facial views and the gaze directions shown in these faces, because the neuronal responses to the two face-gaze configurations that were in the same relative spatial configuration with respect to face and gaze were similar to each other. Despite the finding that face neurons with sensitivity to the relative spatial configuration were few in number, the results suggested that an "object-centered reference frame" was used in the case of some face neurons in the anterior STS. Similar neuronal responses to three-dimensional objects in the object-centered reference frame have been reported in the IPS (Sakata et al. 1997), although the afferent connectivity from the IPS to the rostral region of the anterior STS is relatively weak. It is of note that the rostral region of the lower bank and fundus of the anterior STS have a strong reciprocal connection with the area TEav (Barnes and Pandya 1992; Saleem et al. 2000; Seltzer and Pandya 1978, 1984, 1994); face neurons in the area TEav have been shown to represent facial identity, which is closely related to the eventual decision regarding the determination of facial identity (Eifuku et al. 2004). Thus these results imply that the observed combinatorial neuronal responses to facial views and gaze directions might frequently serve as major bottom-up signals in the process of face identification. However, the involvement of the facial area in the anterior STS in determining the identity of observed faces remains under debate (Heywood and Cowey 1992; see also Eifuku et al. 2004).

Neuronal responses in the caudal region of the anterior STS

Although previous studies have suggested that many, but not all, face neurons in the anterior STS prefer frontal views and profiles, other reports have indicated that an optimal view might be identified across the entire 360° range of views (Perrett et al. 1985, 1991, 1992). These results were quite similar to those obtained with our sample of caudal neurons (Fig. 2). In addition, in these regions, the neuronal responses of many face neurons were found to be right-left symmetrical. Given that right and left views of faces share many common facial components, and given that the viewing of the components depends on the angle of the view to the midline, these results imply that these responses tended to be related to the spatial layout of facial components that cannot simply be ascribed to the mere visibility or invisibility of particular components (Eifuku et al. 2003).

In our previous report (Eifuku et al. 2003), we investigated brain representations of familiar and unfamiliar faces using reaction time (RT) measurements in humans; in that study, we used a pair association paradigm using facial stimuli [pair association task based on an identification (I-PA) task]. In the I-PA task, subjects learned four paired associates in advance of performing the test (prelearning): the pair consisted of a neutral geometric pattern and the face of a person viewed from –45 or +45°. In the test, a stimuli pair was presented to the subject, and the subject was required to judge whether or not the pair was correct. The correct pair was the pattern and the face of the person who was associated with the pattern during prelearning, which could have been one of six images of the person viewed from one of six different angles. While conducting that series, we found that RTs were influenced by prelearning in the case of both unfamiliar and familiar faces, but the shaping of the RT curves differed markedly between cases involving familiar and unfamiliar faces; the RT curve for unfamiliar faces had two volleys, which were right-left symmetrical to the midline (0°), whereas the RT curve for familiar faces had only a single volley, i.e., those showing right-left asymmetry. The results revealed a significant difference between the mental representations of familiar and unfamiliar faces. The results of our previous report are in agreement with those obtained in this study in terms of right-left symmetry and/or asymmetry. The right-left symmetry reflected in the RT curves for unfamiliar faces might have been associated with the behavior of face neuron responses frequently observed in the caudal region of the anterior STS, where the majority of face neuron responses had two peaks that were right-left symmetrical to the midline (0°).

Functional heterogeneity and hierarchical organization within the anterior STS

Previous anatomical studies have indicated that the rostral region of the lower bank and the fundus of the anterior STS have a strong reciprocal connection with area TEav (Barnes and Pandya 1992; Saleem et al. 2000; Seltzer and Pandya 1978, 1984, 1994), which also has a strong reciprocal connection with medial limbic structures such as the perirhinal cortex. It is already known that there are only weak connections between the parietal and occipital areas and the rostral region of the anterior STS. On the other hand, the caudal region of the lower bank and fundus of the anterior STS has abundant afferent connections stemming from the intraparietal (area POa), preoccipital (area V4), and posterior parahippocampal areas (areas TH and TF) (Barnes and Pandya 1992; Saleem et al. 2000; Seltzer and Pandya 1978, 1984, 1994). Moreover, it was recently reported that the caudal region of the anterior STS has a strong reciprocal connection with the dorsal portion of the anterior inferior temporal gyrus (area TEad), which also has a strong reciprocal connection with the posterior parahippocampal regions (areas TF, TH) (Saleem et al. 2000). It is thus likely that the anatomical connectivity differs in the rostral and caudal regions of the anterior STS.

We separated two groups of face neurons in the anterior STS based on their rostro-caudal coordinates: the rostral group consisted of neurons that were located in the rostral region of the anterior STS, and the caudal group consisted of neurons that were located in the relatively caudal region of the anterior STS. Three types of functional difference between the rostral group and the caudal group of face neurons were identified in this study. In the caudal group, optimal facial views for the face neuron responses were distributed among all types of views as is shown in Fig. 3. The majority of face neurons in the caudal group responded symmetrically to right and left views, as those seen in mirror images (Fig. 4). In contrast, face neurons in the rostral group responded best to a single oblique view; moreover, right-left symmetry among the responses was less evident in the rostral group, as summarized in Figs. 3 and 4. These results imply the possibility that face neurons in the rostral region represent certain canonical faces, whereas no such specialization was apparent in the caudal region. The modulation of face neuron response by gaze direction was more evident in the rostral group than in the caudal group (Fig. 6). These findings suggest that biological significance is a more important factor for determining the face neuron responses in the rostral region than in the caudal region.

It is well known that in the domain of visual information processing related to "object vision," the inferior temporal cortical areas of primates are hierarchically organized along the rostro-caudal (or antero-posterior) axis. In particular, single neuronal recording studies using monkeys have thus far shown the presence of a functional hierarchy in the inferior temporal gyrus along the rostro-caudal (or antero-posterior) axis. Moreover, recent reports on neural organization in the anterior superior temporal gyrus of rhesus monkeys have indicated a hierarchical organization in the domain of auditory information processing; the rostral region of the left anterior superior temporal gyrus was shown to be selectively activated by species-specific vocalizations (Poremba et al. 2003, 2004).

Our results revealed differences between the rostral and caudal regions of the anterior STS in terms of the response characteristics of face neurons, and these results are consistent with previous findings of anatomical connectivity. In addition, these findings were suggestive of a plausible functional hierarchy in the anterior STS, a notion that is also in agreement with the findings of previous reports regarding the organization of the anterior superior/inferior temporal gyri.

	GRANTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
This study was supported by a Grant-in-Aid for Scientific Research from the Japanese Ministry of Education, Culture, Sports, Science and Technology (14017037, 14780618, and 16500260)

	ACKNOWLEDGMENTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION GRANTS ACKNOWLEDGMENTS REFERENCES

 
We thank Drs. Richard J. Krauzlis (The Salk Institute, CA), and Michele A. Basso (University of Wisconsin, Madison, WI) for comments on early versions of this manuscript.