|
|
||||||||
1Wellcome Department of Imaging Neuroscience and 2Institute of Cognitive Neuroscience, University College London, London WC1N 3BG, United Kingdom
Submitted 17 February 2004; accepted in final form 23 April 2004
| ABSTRACT |
|---|
|
|
|---|
| INTRODUCTION |
|---|
|
|
|---|
Functional imaging studies comparing familiar versus unfamiliar faces also support a role for fusiform cortex in representing facial identity (George et al. 1999
; Henson et al. 2000
, 2003
). However, differences between familiar and unfamiliar faces, such as increased attention to familiar faces (Wojciulik et al. 1998
), may confound these findings. Support for dissociable roles of fusiform and STS comes from studies that directed attention to different aspects of face stimuli (Hoffman and Haxby 2000
; Narumoto et al. 2001
; Sergent et al. 1992
, 1994
; Winston et al. 2003a
). Nevertheless, it is difficult to draw definitive conclusions about the nature of the representations in these regions using such task manipulations, because differences in activation might reflect processing involved in directing visual attention to the specific aspects of the face required by each task rather than representations of those face components themselves.
Functional imaging studies of emotional facial expression have reported data that seem inconsistent with the distributed model, showing enhanced fusiform activity to emotional compared with neutral faces (Breiter et al. 1996
; Dolan et al. 1996
; Morris et al. 1998
; Pessoa et al. 2002
; Surguladze et al. 2003
; Vuilleumier et al. 2001
; Winston et al. 2003a, b
). If fusiform cortex is specialized for identity, it is unclear why it should show this enhanced response to emotional faces. This effect has been attributed to modulatory effects from amygdala, reflecting enhanced attentional processing associated with emotive stimuli (Dolan 2002
), but direct evidence for this proposal is sparse (for exceptions, see Morris et al. 1998
and Pessoa et al. 2002
).
Functional MRI-adaptation (fMRI-A) is a technique used to infer regional specialization with greater specificity than the subtractive methodology used in the preceding imaging studies. The logic of fMRI-A, outlined previously (Grill-Spector and Malach 2001
; Henson 2003
; Naccache and Dehaene 2001
), can be summarized as follows: if a region contains subpopulations of neurons excited by distinct aspects of stimuli, when two stimuli are shown sequentially in which one of these aspects is repeated, the firing neurons will habituate, and decreased fMRI signal from that region will be seen compared with when that aspect is not repeated. We report here the use of an fMRI adaptation paradigm to test the hypothesis, derived from the distributed model (Haxby et al. 2000
), that fusiform cortex would show adaptation when identity was repeated relative to when it changed, and STS would show adaptation when an emotional expression was repeated relative to when it changed. Given our factorial design (Fig. 1B), the prediction was that fusiform cortex should show a significant main effect of identity repetition and STS would show a significant main effect of expression repetitions, but that neither area would show a significant interaction, because the interaction in this experimental design looks for areas showing a co-dependence between identity and expression repetitions.
|
| METHODS |
|---|
|
|
|---|
Stimuli
The stimuli were a selection of five male faces from the KDEF database (Lundqvist and Litton 1998; The Karolinska Directed Emotional Faces, Department of Clinical Neuroscience, Psychology Section, Karolinska Institute). A major advantage of using this database is that it contains two exemplars of each expression for each identity. This allowed the use of different images for the first and second faces in the adaptation session for all trial types, even SISE. The five specific identities (males 13, 14, 16, 23, and 24) were selected on the basis of successful emotion recognition by nine subjects who took part in a pilot study judging the emotions expressed across the entire database. All face stimuli showed head and eye-gaze direction that were forward-facing toward the viewer. Five emotions were used in the study: anger, disgust, fear, happiness, and sadness. One exemplar of each expression (series "A") was nominated as the prime face and the second (series "B") as the second face for the adaptation phase. Stimuli were converted to grayscale and equated for mean luminance in Matlab (The Mathworks, Natick, MA) and cropped to a standardized outline in Photoshop (Adobe, San Jose, CA). Faces for the filler trials were neutral male faces from a variety of sources and were prepared similarly to emotional faces, as were chairs and female faces (the targets in the localizer and adaptation phases, respectively). One hundred scrambled faces for the localizer session were derived from the 50 emotional faces and 50 neutral male faces used in that session by permuting the phase of each spatial frequency in the image while maintaining a constant power density spectrum and cropping to the same outline.
Subjects
Sixteen right-handed healthy subjects gave informed consent to take part in the study, which was approved by the local ethics committee. Data were rejected from two of these subjects: one because of gross head movement during fMRI scanning and the second due to an incidental structural abnormality that made normalization of scans difficult. Age range of included subjects was 1829 yr (mean, 23 yr), and there were seven males. Subjects had normal or corrected-to-normal vision.
fMRI experiment
There were three sessions to the fMRI component of the study. The first was a face localizer session, designed to familiarize participants with face stimuli and provide a generic map of face- and expression-responsive brain regions. The second and third sessions were adaptation sessions and were split only for subject comfort. In the first ("localizer") session, subjects' task was chair detectionthey responded on an MRI-compatible button box when they saw a chair. The stimuli seen in this session were the 50 emotional faces from critical trials of the second and third ("adaptation") sessions, 50 neutral male faces, 100 scrambled faces, and 20 chairs. In the adaptation phase, the task was to press the button when a female face appeared. In this way, trials of interest in all sessions were uncontaminated by motor response, and repetition of faces was incidental to the subject's task. A number of different trial types occurred in the adaptation phase. The four trial types of interest consisted of a pair of emotional faces that could exhibit the same or different identities, crossed with the same or different emotional expression (Fig. 1B). In addition, between any pair of trials of interest, one or two "filler trials" occurred to reduce the predictability of repetition. Three types of filler trials were used: "repeated fillers," in which the face pair consisted of two neutral faces of the same identity; "different fillers," consisting of a pair of neutral male faces of different identities; and "target trials," containing a neutral male and a neutral female face (which could be either the 1st or 2nd face of the pair). In total, 80 repeated filler trials, 60 different filler trials, and 20 target trials occurred in each of the two sessions. Twenty-five trials of interest occurred for each of the four trial types in each session. Because there is only one way of combining the faces to produce SISE trials, but multiple ways of combining to produce the other trial types, the face pairs used for such trials were counterbalanced across subjects, with the constraint that each first face occurred once in each condition and each second face once in each condition within each session.
fMRI scanning
A Siemens 1.5T Sonata system (Siemens, Erlangen, Germany) was used to acquire blood oxygenation level dependent (BOLD) contrast-weighted echoplanar images (EPIs) for functional scans. Volumes, which consisted of 24 horizontal slices of 2 mm thickness with a 1 mm gap, were acquired continuously every 2.16 s. This sequence was sufficient to obtain coverage from above the corpus callosum to below the inferior temporal lobes, thus including all regions of interest in this study: fusiform, amygdala, STS, inferior frontal cortex, and orbitofrontal cortex (Fig. 2). In-plane resolution was 3 x 3 mm. The first six volumes were discarded to allow for T1 equilibriation effects. Subsequent to functional scans, a T1-weighted structural image (1 x 1 x 1 mm resolution) was acquired for co-registration and display of the functional data. Because of the difficulty of normalizing limited field-of-view EPIs, we additionally acquired whole brain EPIs in each subject for improved normalization.
|
fMRI data were spatially preprocessed using SPM2 (Wellcome Department of Imaging Neuroscience, London; http://www.fil.ion.ucl.ac.uk/spm). The volumes were co-registered (Friston et al. 1995a
) and normalized to an EPI template corresponding to the MNI reference brain in Talairach space. The normalization parameters for each subject's EPIs were obtained by normalizing the whole brain EPIs acquired after the experimental session. The limited field of view EPIs were co-registered with the raw whole brain images (Collignon et al. 1995
) and normalized by applying the parameters calculated for normalization of the whole brain images. Normalized images were smoothed using an 8-mm Gaussian kernel to account for residual intersubject differences and to allow statistical inference using Gaussian random field theory.
Data analysis
Data analysis used SPM2, applying a mass univariate general linear model (GLM) (Friston et al. 1995b
). First, delta functions were constructed corresponding to the onset of each event type and for button presses in the adaptation phase to accommodate false alarms. These delta functions were convolved with a synthetic hemodynamic response function to create regressors for the subsequent GLM. Also included in the model were six movement parameters estimated by the realignment stage, regressors representing session effects, and for the adaptation sessions, three regressors of no interest corresponding to the potential confound of similarity between face pairs (see Visual similarity between face pairs). Serial autocorrelations were modeled using an AR(1) process, and the data were high-pass filtered at 1/128 Hz. Linear contrasts pertaining to the main effects and interaction of the factorial design were calculated. Consistent effects across subjects were tested using the resultant contrast images in a one-sample t-test (conforming to a "random effects" model). The model for the localizer session included separate regressors for the distinct facial expressions. Because no significant (P < 0.05, uncorrected) interactions with emotion type were detected in regions of interest (see also Winston et al. 2003a
), we modeled the adaptation sessions without reference to different emotion types. Statistical threshold was set at P < 0.05, corrected for multiple comparisons across a small volume of interest, using a mask derived from the localizer session (see Mask-defining regions of interest).
Mask-defining regions of interest
Regions of interest were defined using two statistical results from the localizer session. First, we tested for effects of faces > scrambled faces and thresholded the result at P < 0.001 (uncorrected). We next tested for emotional faces > neutral faces and thresholded this result at P < 0.001 (uncorrected). The two resulting masks were combined using a logical OR function, yielding the combined mask (Fig. 2) that was subsequently used for small volume correction (SVC) (Worsley et al. 1996
).
Visual similarity between face pairs
It could be argued that changes in identity and expression across a pair of faces do not represent only categorical changes, but also a variation in a continuous spectrum of visual similarity. Thus, for example, a pair of faces with the same identity and expression are likely to be more visually similar than a pair where identity is the same but the expression different. To avoid this potential confound affecting fMRI data analysis, three measures of visual similarity were collected and included in fMRI data analysis as covariates of no interest. The first two were image-based metrics, derived from mathematical analysis of image pairs. These were derived from normalized least squares measures of differences between face pairs. Briefly, faces were normalized for luminance, and the second face was subtracted from the first. The root mean square of the value at each pixel of this difference image was the difference score for a given image pair. In a refinement of the technique that accounted for minor differences in co-registration of salient features, faces were allowed to move over one another by
25 pixels in either plane, and the minimum resulting value was taken as the difference score (Vogels et al. 2001
). The third measure adopted was derived from an independent group of subjects (see Control data on explicit tasks for identity/expression detection) who each saw 200 face pairs (50 of each trial type) presented with the same parameters as the imaging component of the study and rated each pair for visual similarity. The three ratings were in good agreement (Table 1). All three were included in the main statistical model as regressors of no interest by generating a design matrix in SPM whereby all events of interest were modeled as one trial type that was modulated parametrically by expansions to model the three similarity confounds. The columns pertaining to similarity confounds were extracted and used in the model described above.
|
fMRI differences in regions such fusiform cortex could be attributable to variations in visual attention with trial type (Wojciulik et al. 1998
), and differences in emotionally responsive regions such as amygdala would be similarly attributable to variations in arousal (Critchley et al. 2002
). To explore whether such differences might exist, we used on-line eye-tracking. Data were acquired during scanning for a majority of subjects using an ASL504LRO eye-tracker (Applied Science Laboratories, Bedford, MA). Specifically, accurate pupillometry was achieved in nine subjects during the scanning session, and accurate eye-gaze tracking was achieved in eight. Pupillometry data were analyzed by defining a window of 1.2 s after the second face and measuring the minimum, maximum, and mean pupil diameter during averaged traces (low-pass filtered at 7.5 Hz and baselined for the onset of the 2nd face) from this window for each subject. These three measures were entered into separate 2 x 2 ANOVAs. Eye-gaze direction was also assessed using a summary statistic approach. For each of the four critical trial types, spatial maps of eye-gaze density were constructed. Each of these maps was compared with the mean map, and difference images constructed. The root mean squares of the density difference values for these latter maps were entered into a 2 x 2 ANOVA.
Control data on explicit tasks for identity/expression detection
An important consideration is the possibility that subjects might not have noticed repetition of identity or expression of faces within each pair. Furthermore, if a change in one dimension (e.g., the expression of the faces) affected subjects' ability to detect repetition of the other dimension (e.g., the identity of the faces), interactions between the two dimensions detected by fMRI would be difficult to interpret (in that a decrease in the fusiform response for SIDE trials relative to SISE trials, for example, could simply reflect a reduction in the number of trials in which subjects realized it was the same identity). Control behavioral experiments were conducted to test these possibilities. An independent group of 16 subjects (age range, 2236 yr; mean age, 28.5 yr; 11 males; 2 left-handers) completed three behavioral tasks, using identical procedural parameters to those in the imaging study. In the first task, they rated pairs of faces presented for visual similarity using a computer-based visual analogue scale (providing the subjective measures of similarity mentioned above). In the second and third tasks, they classified face pairs as exhibiting either the same or different identity or the same or different emotional expression, with the order of identity/expression task and the buttons used for same/different responses counterbalanced over subjects. In all, each subject performed 50 trials of each type for each task. A short (25 trial) practice session preceded each task.
| RESULTS |
|---|
|
|
|---|
Subjects detected 99 ± 2% (SD) of targets (chairs) in the localizer (false alarm rate, 0.2 ± 0.4%) and 88 ± 7% of targets (female faces) in the adaptation phase (false alarm rate, 7 ± 6%).
The two measures derived from the eye-tracking data from the fMRI scanning sessions showed no significant differences between the four trial types of interest (for gaze direction, all F(1,7) < 0.125, all P > 0.7; for pupil diameter, all F(1,8) < 2.1, all P > 0.18, except a marginal trend for an interaction between identity and expression repetitions in the minimum pupil constriction: F(1,8) = 4.0, P = 0.08). These nonsignificant results suggest that there were no detectable differences in visual attention (indexed by eye-gaze direction) or arousal (indexed by pupil diameter changes) between the different experimental conditions.
Localizer phase
The results of the two t-tests performed on the data from the localizer phase, faces > scrambled faces and emotional > neutral faces, are shown in Fig. 2. As expected, the activated regions included bilateral fusiform and more posterior occipital areas, as well as STS and amygdala. The two contrasts were combined to create a mask of regions that responded to faces and/or facial expression. This mask defined a search region for the subsequent comparisons in the adaptation phase, allowing a principled means for correcting for multiple comparisons over voxels.
Adaptation phase: main effect of repeated identity
As predicted, a significant main effect of repeated identity [reduced response when the 2nd face exhibited the same identity as the 1st; (DISE + DIDE) > (SISE + SIDE)] was seen in right fusiform cortex (x,y,z = 39,60,15; Z = 3.76; P < 0.05, 1-tailed, SVC for the localizer mask; Fig. 3). In addition, adaptation for repeated identity was seen in right posterior STS (STSp; x,y,z = 63,51,15; Z = 3.73; P < 0.05 SVC; Fig. 4). Because an interaction or main effect of repeated expression would influence interpretation of these results, we examined for such effects at reduced threshold. In the peak right STSp voxel, a marginally significant main effect of repetition of expression was evident (Z = 1.74; P < 0.05, 1-tailed, uncorrected), whereas in fusiform, no such effect was evident (Z = 0.99, P > 0.1, 1-tailed, uncorrected). There was no evidence for an interaction in the peak fusiform voxel (Z = 1.26, P > 0.2, 2-tailed, uncorrected) or in the right STSp (Z = 0.50, P > 0.2, 2-tailed). Although simple effects are not conventionally inspected in the absence of a significant interaction, we checked whether changing expression modulated fusiform responses either in the context of identity remaining constant or changing. In neither case was there a significant effect (simple effect of DE relative to SE with a change in identity, P = 0.93; with identity constant, P = 0.14). A more posterior region of right occipital cortex, possibly corresponding to a face-responsive occipital region (FROR), showed uncorrected repetition effects but failed to withstand correction for multiple comparisons across the volume of the mask (x,y,z = 42,75,18; Z = 3.59; P = 0.071 SVC; see Fig. 3A).
|
|
A region of right STS anterior to that described above was shown to be less active when the second face exhibited the same expression as the first face [Fig. 5; (SIDE + DIDE) > (SISE + DISE)]. This activation corrected for multiple comparisons across the volume of our mask (x,y,z = 57,18,12; Z = 3.80; P < 0.05, 1-tailed, SVC). Despite the apparent trend toward an interaction in Fig. 5B, this was not significant when tested at a lenient statistical threshold (Z =1.06, P > 0.2, 2-tailed, uncorrected). Similar to the fusiform region, we checked for significant simple effects (DI relative to SI) opposite to the detected main effect and found no significant differences in the context of expression changing (P = 0.14) or being held constant (P = 0.97).
|
To determine whether the differences in detectable main effects between the mid-STS region and fusiform were significant, we undertook a region-by-condition interaction using a 3-mm sphere centered on each peak. A significant three-way interaction was obtained (P < 0.05) in the direction predicted by the distributed model (adaptation to identity in fusiform and to expression in STS).
Adaptation phase: interaction
No areas within the mask defining our regions of interest showed an interaction between identity and expression.
Control experiments
One potential confound in our design is the presence of differences in the visual similarity between face pairs of different trial types. To account for this confound, we obtained mean subjective and objective similarity measures for the four different trial types (Table 1). The subjective data were obtained from an additional behavioral experiment (see METHODS). All three measures showed significant differences between the four trial types (all F(1,13) > 190, all P < 0.001). SISE pairs were more similar than the other three types, despite our use of different images in this condition. Unsurprisingly, and consistent with the concept of identity as an invariant feature of the face, trials with same identity had greater similarity than trials with different identity. To account for these differences, we included all three measures as covariates of no interest in the analysis of the fMRI data (see METHODS), which removed any linear contribution of similarity to the above fMRI findings. A random effects analysis of the contribution of these regressors to the model (using an F-contrast spanning the 3 regressors in an ANOVA model) suggested that they were explaining effects in visual regions, although not within the mask used for SVC (e.g., peaks at x,y,z = 51,60,6, Z = 3.76; x,y,z = 9,78,9; Z = 3.46; x,y,z = 54,45,15, Z = 3.39; x,y,z = 30,60,15, Z = 3.38; all P < 0.001 uncorrected).
An additional concern, noted above, is that subjects might not notice repetitions of identity or expression or that the presence of repetition in one dimension would affect behavior to the other dimension. Data from a behavioral experiments on a separate subject cohort (see METHODS) showed that the mean accuracy in an identity discrimination task was 88% for trials when expression was held constant and 83% for trials when expression changed (paired t-test: Z = 3.78, P < 0.001). This was paralleled by slower reaction times (RTs) when judging identity in the context of expression changes (806 vs. 773 ms, Z = 3.33, P < 0.001). Mean accuracy for the emotion discrimination task was 87% when identity was unchanged across the face pair and 80% when identity changed (Z = 3.53, P < 0.001). Again, RTs were slower on trials where the task-irrelevant dimension (identity) changed compared with being held constant (888 vs. 848 ms, Z = 4.41, P < 0.001). These data show that people's ability to detect repetition of identity or expression with these stimuli was generally high. The data also suggest that changes in one dimension do affect sensitivity to repetition of the other dimension. This behavioral interaction does not, however, confound our findings of two orthogonal main effects in the fusiform/STSp and mid-STS regions.
| DISCUSSION |
|---|
|
|
|---|
In the distributed model of face processing (Haxby et al. 2000
), a dissociation is posited between processing of invariant and changeable aspects of faces. Specifically, it is suggested that invariant features are coded in ventral occipital and temporal cortex in the lateral fusiform region (also known as the "face area"; Kanwisher et al. 1997
), whereas changeable aspects are coded by right STS. Our data broadly support this model. Within the framework of fMRI-A, our demonstration of a main effect of repeated identity in right fusiform cortex indicates this region represents identity, an invariant aspect of human faces. Although previous studies have shown repetition decreases to faces in fusiform cortex (Gauthier et al. 2000
; George et al. 1999
; Henson et al. 2000
, 2003
), to our knowledge, this study is the first to show repetition effects in fusiform cortex across dramatically different views of the same identity (i.e., with different expressions). In our view, this finding is important because it suggests that face representations in this region encode not just a specific visual image but a more abstract representation of facial identity (see also Eger et al. 2004
; Vuilleumier et al. 2003a
).
A consistent finding in neuroimaging studies of emotional face perception is activation of fusiform cortex in perception of emotional relative to neutral faces (Breiter et al. 1996
; Dolan et al. 1996
; Morris et al. 1998
; Pessoa et al. 2002
; Surguladze et al. 2003
; Vuilleumier et al. 2001
; Winston et al. 2003a,b
). This has been interpreted as relating to enhanced attentional processing associated with arousing emotional faces relative to nonarousing neutral faces (Dolan 2002
). However, an alternative explanation is that this region encodes the emotionality of the face, resulting in enhanced activation when expressive faces are presented. The use of an adaptation paradigm in this study enables us to potentially dissociate between these possibilities. If this region coded for specific expressions, it should have shown adaptation for expression, akin to that for identity. The lack of evidence for adaptation for repeated expressions is consistent with the former interpretation that fusiform modulation is mediated by an amygdala-associated effect (although we note that this inference is based on a null result). At the very least, it seems that any bottom-up effects of expression in right fusiform cortex are of less importance than those of identity, i.e., it exhibits relative preference for identity processing from faces.
In contrast to right fusiform, a focus in right mid-STS showed a main effect for repetition of emotional expression, with repeated expressions associated with reduced activation relative to differing expressions. This agrees with a role for this region in coding the specific emotion expressed in a face. We were surprised by the anterior locus of this activation, which fell at 18 on the anterior-posterior axis. Previous studies concerning facial expression have reported activation in right STS in a more posterior locus (around 35 to 60 mm) (Critchley et al. 2000
; Iidaka et al. 2001
; Narumoto et al. 2001
; Winston et al. 2003a
). This more anterior locus is, however, within the portion of STS reported as activated in studies of social cues (Allison et al. 2000
; Martin and Weisberg 2003
; Ojemann et al. 1992
; Saxe and Kanwisher 2003
). We have additionally checked our previous data for activation in this area and found it was activated in an explicit emotional judgment task relative to a gender judgment task (x,y,z = 52,16,18; Z = 3.96; see Fig. 5A in Winston et al. 2003a
). Note also that this region fell within our face localizer mask, and by definition, is responsive to faces or facial expression.
Posterior STS, like the fusiform, showed adaptation to repeated identity. This is contrary to a previous study that failed to observe repetition effects in this region (Henson et al. 2003
), although that study used much longer repetition lags. A role for posterior STS in processing personal identity is, however, consistent with a recent human lesion study describing a patient with an infarct in the vicinity of left STS who described novel faces as familiar (Vuilleumier et al. 2003b
). Unlike fusiform, however, posterior STS showed a trend toward an additive main effect for repeated emotion, implying that its role in face processing may be multifaceted. Intriguingly, in a recent re-analysis of single neuron data from monkeys, Tiberghien et al. (2003)
suggested that all facial features contribute to distinguishing identity, whereas only a subset determine facial expression. They hypothesize that, as a consequence, inferior temporal regions in monkeys may contain identity-selective neuronal populations, while STS might contain populations sensitive to identity and expression. Such a view fits with our demonstration of identity repetition in posterior STS and sensitivity to expression in posterior and mid-STS. However, with regard to the human lesion literature, the majority of reported prosopagnostic patients described have inferior occipitotemporal rather than lateral temporal lesions (see e.g., Damasio et al. 1990
; Wada and Yamamoto 2001
), presumably corresponding to fusiform rather than STS (but see Fig. 1A in Tranel et al. 1997
; see also Rossion et al. 2003
). In addition, monkeys with STS lesions appear to have only minor identity discrimination deficits (Heywood and Cowey 1992
), and recent evidence from multidimensional scaling analysis of single neuron data from monkey STS and inferior temporal (IT) cortex also suggests that STS is more concerned with analysis of facial view and the code in IT is more concerned by facial identity (Eifuku et al. 2004
). This apparent discrepancy with our result of posterior STS responsivity to identity might be explained in a number of ways. First, it is possible that activation in this region is epiphenomenal and of no functional consequence for identity recognition. However, it is also possible that our stimuli tax identity processing across different views of a face, and the role of STS in processing different views of face stimuli is well known (Eifuku et al. 2004
; Perrett et al. 1985
, 1991
). In addition, it has been shown that STS neurons in the macaque monkey process identity, at least in the form of a population code (Baylis et al. 1985
), and there is evidence that single neurons in STS code for the same identity across different face views and other STS cells code conjunctions of identity and view (Perrett et al. 1991
). It may be the case that the aspect(s) of identity processing that occur in STS do not commonly lead to complaints of prosopagnosia or that tests designed to probe prosopagnosia are relatively insensitive to these aspect(s) of identity processing. As an unpredicted, although significant activation, we would like to see this effect of adaptation for repetitions of identity across different views replicated before drawing strong conclusions.
Previous work has implicated other brain regions in processing facial expressions, most notably the amygdala (Breiter et al. 1996
; Morris et al. 1996
; Pessoa et al. 2002
; Vuilleumier et al. 2001
; Whalen et al. 1998
; Winston et al. 2003a
). There are a number of potential reasons why we did not detect adaptation in this region. One possibility is that amygdala responses are emotion-specific, with greatest responses to fearful faces (Calder et al. 2001
), and thus our collapsing across different expression subtypes may have obscured emotion-specific responses. Alternatively, the amygdala might code for facial expression in a different manner from cortical regions such as STS, with a nonspecific code, whereby responses are dependent on the arousal engendered by the emotion (H. D. Critchley, P. Rothstein, and R. J. Dolan, unpublished observations). Another possibility is that an expression-specific amygdala response may be insensitive to adaptation, although this seems unlikely, given positive findings concerning the amygdala and stimulus repetition (Ono and Nishijo 2000
; Rotshtein et al. 2001
).
Although identity and emotion may be processed by partially dissociable neural pathways, the two pathways are likely to interact in production of behavioral responses. This would appear to be the case for our explicit identity and emotion detection tasks, in which a change in one dimension (identity or emotion) impaired ability to detect changes in the other dimension. In behavioral studies, other authors have also found evidence for the nonindependence of identity and emotion processing (Ganel and Goshen-Gottstein 2004; Schweinberger and Soukup 1998
; Schweinberger et al. 1999
). A further brain region may be responsible for the integration of distinct aspects of the face that we failed to detect in this study. A more explicit behavioral task during fMRI may help to clarify this issue in future studies.
One issue that deserves consideration is the meaning of BOLD changes in adaptation paradigms such as this. It has been shown that local field potentials (LFPs) correlate with the BOLD signal better than multi- or single-unit activity in the macaque monkey (Logothetis et al. 2001
). Thus a region showing fMRI-A may not be transmitting fewer spikes but may either be showing a reduced afferent input or reduced local processing. This highlights one possible dissociation between fMRI-A and response suppression as recorded in single unit work in monkeys (Desimone 1996
). We do not think that fMRI experiments based on adaptation are uniquely problematic in this regard, but this is a more general interpretational issue for unifying electrophysiological and fMRI work (see Henson and Rugg 2003
for a more extensive discussion of hemodynamic decreases and response suppression).
In conclusion, we have shown that fusiform cortex shows fMRI-A when the identity of a face is repeated, and a region of STS shows adaptation when the emotional expression of a face is repeated. The response profiles of these two regions were significantly different in the directions predicted by the distributed model of face processing (Haxby et al. 2000
), and we suggest that our findings are generally consistent with this model. However, an adaptation response in posterior STS to repeated identity suggests that STS may also manifest a degree of functional segregation in face perception.
| GRANTS |
|---|
|
|
|---|
| ACKNOWLEDGMENTS |
|---|
|
|
|---|
| FOOTNOTES |
|---|
Address for reprint requests and other correspondence: J. Winston, Wellcome Department of Imaging Neuroscience, 12 Queen Square, London WC1N 3BG, UK (E-mail: j.winston{at}fil.ion.ucl.ac.uk).
| REFERENCES |
|---|
|
|
|---|
Allison T, Puce A, and McCarthy G. Social perception from visual cues: role of the STS region. Trends Cogn Sci 4: 267278, 2000.[CrossRef][Web of Science][Medline]
Baylis GC, Rolls ET, and Leonard CM. Selectivity between faces in the responses of a population of neurons in the cortex in the superior temporal sulcus of the monkey. Brain Res 342: 91102, 1985.[CrossRef][Web of Science][Medline]
Breiter HC, Etcoff NL, Whalen PJ, Kennedy WA, Rauch SL, Buckner RL, Strauss MM, Hyman SE, and Rosen BR. Response and habituation of the human amygdala during visual processing of facial expression. Neuron 17: 875887, 1996.[CrossRef][Web of Science][Medline]
Bruce V and Young A. Understanding face recognition. Br J Psychol 77: 305327, 1986.
Calder AJ, Lawrence AD, and Young AW. Neuropsychology of fear and loathing. Nat Rev Neurosci 2: 352363, 2001.[CrossRef][Web of Science][Medline]
Collignon A, Maes F, Delaere D, Vandermeulen D, Suetens P, and Marchal G. Automated multi-modality image registration based on information theory. In: Proceedings of Information Processing in Medical Imaging Conference 1995, edited by Bizais Y, Barillot C, and Di Paola R. Dordrecht, The Netherlands: Kluwer Academic Publishers, 1995, p. 263274.
Critchley HD, Daly E, Phillips M, Brammer M, Bullmore E, Williams S, Van Amelsvoort T, Robertson D, David A, and Murphy D. Explicit and implicit neural mechanisms for processing of social information from facial expressions: a functional magnetic resonance imaging study. Hum Brain Mapp 9: 93105, 2000.[CrossRef][Web of Science][Medline]
Critchley HD, Melmed RN, Featherstone E, Mathias CJ, and Dolan RJ. Volitional control of autonomic arousal: a functional magnetic resonance study. Neuroimage 16: 909919, 2002.[CrossRef][Web of Science][Medline]
Damasio AR, Tranel D, and Damasio H. Face agnosia and the neural substrates of memory. Annu Rev Neurosci 13: 89109, 1990.[CrossRef][Web of Science][Medline]
Desimone R. Neural mechanisms for visual memory and their role in attention. Proc Natl Acad Sci USA 93: 1349413499, 1996.
Dolan RJ. Emotion, cognition, and behavior. Science 298: 11911194, 2002.
Dolan RJ, Fletcher P, Morris J, Kapur N, Deakin JF, and Frith CD. Neural activation during covert processing of positive emotional facial expressions. Neuroimage 4: 194200, 1996.[CrossRef][Web of Science][Medline]
Eger E, Schyns PG, and Kleinschmidt A. Scale invariant adaptation in fusiform face-responsive regions. Neuroimage 22: 232242, 2004.[CrossRef][Web of Science][Medline]
Eifuku S, De Souza WC, Tamura R, Nishijo H, and Ono T. Neuronal correlates of face identification in the monkey anterior temporal cortical areas. J Neurophysiol 91: 358371, 2004.
Epstein R, Graham KS, and Downing PE. Viewpoint-specific scene representations in human parahippocampal cortex. Neuron 37: 865876, 2003.[CrossRef][Web of Science][Medline]
Friston K, Ashburner J, Frith CD, Poline J-B, Heather JD, and Frackowiak RSJ. Spatial registration and normalization of images. Hum Brain Map 2: 165189, 1995a.
Friston K, Holmes AP, Worsley K, Poline J-B, Frith C, and Frackowiak RSJ. Statistical parametric maps in functional imaging: a general linear approach. Hum Brain Map 2: 189210, 1995b.[CrossRef]
Ganel T and Goshen-Gottstein Y. Effects of familiarity on the perceptual integrality of the identity and expression of faces: the parallel-route hypothesis revisited. J Exp Psychol Hum Percept Perform 30: 583597, 2004.[CrossRef][Web of Science][Medline]
Gauthier I, Tarr MJ, Moylan J, Skudlarski P, Gore JC, and Anderson AW. The fusiform "face area" is part of a network that processes faces at the individual level. J Cogn Neurosci 12: 495504, 2000.[CrossRef][Web of Science][Medline]
George N, Dolan RJ, Fink GR, Baylis GC, Russell C, and Driver J. Contrast polarity and face recognition in the human fusiform gyrus. Nat Neurosci 2: 574580, 1999.[CrossRef][Web of Science][Medline]
Grill-Spector K and Malach R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol (Amst) 107: 293321, 2001.
Hasselmo ME, Rolls ET, and Baylis GC. The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey. Behav Brain Res 32: 203218, 1989.[Web of Science][Medline]
Haxby JV, Hoffman EA, and Gobbini MI. The distributed human neural system for face perception. Trends Cogn Sci 4: 223233, 2000.[CrossRef][Web of Science][Medline]
Henson RNA. Neuroimaging studies of priming. Prog Neurobiol 70: 5381, 2003.[CrossRef][Web of Science][Medline]
Henson RNA, Goshen-Gottstein Y, Ganel T, Otten LJ, Quayle A, and Rugg MD. Electrophysiological and haemodynamic correlates of face perception, recognition and priming. Cereb Cortex 13: 793805, 2003.
Henson RNA and Rugg MD. Neural response suppression, haemodynamic repetition effects, and behavioural priming. Neuropsychologia 41: 263270, 2003.[CrossRef][Web of Science][Medline]
Henson RNA, Shallice T, and Dolan R. Neuroimaging evidence for dissociable forms of repetition priming. Science 287: 12691272, 2000.
Heywood CA and Cowey A. The role of the face-cell area in the discrimination and recognition of faces by monkeys. Philos Trans R Soc Lond B Biol Sci 335: 3137, 1992.
Hoffman EA and Haxby JV. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nat Neurosci 3: 8084, 2000.[CrossRef][Web of Science][Medline]
Iidaka T, Omori M, Murata T, Kosaka H, Yonekura Y, Okada T, and Sadato N. Neural interaction of the amygdala with the prefrontal and temporal cortices in the processing of facial expressions as revealed by fMRI. J Cogn Neurosci 13: 10351047, 2001.[CrossRef][Web of Science][Medline]
Kanwisher N, McDermott J, and Chun MM. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci 17: 43024311, 1997.
Kourtzi Z and Kanwisher N. Representation of perceived object shape by the human lateral occipital complex. Science 293: 15061509, 2001.
Logothetis NK, Pauls J, Augath M, Trinath T, and Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412: 150157, 2001.[CrossRef][Medline]
Lundqvist D and Litton JE. The Averaged Karolinska Directed Emotional FacesAKDEF, 1998.
Martin A and Weisberg J. Neural foundations for understanding social and mechanical concepts. Cognit Neuropsychol 20: 575587, 2003.[CrossRef]
McCarthy G, Puce A, Belger A, and Allison T. Electrophysiological studies of human face perception. II: Response properties of face-specific potentials generated in occipitotemporal cortex. Cereb Cortex 9: 431444, 1999.
Morris JS, Friston KJ, Buchel C, Frith CD, Young AW, Calder AJ, and Dolan RJ. A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain 121:4757, 1998.
Morris JS, Frith CD, Perrett DI, Rowland D, Young AW, Calder AJ, and Dolan RJ. A differential neural response in the human amygdala to fearful and happy facial expressions. Nature 383: 812815, 1996.[CrossRef][Medline]
Naccache L and Dehaene S. The priming method: imaging unconscious repetition priming reveals an abstract representation of number in the parietal lobes. Cereb Cortex 11: 966974, 2001.
Narumoto J, Okada T, Sadato N, Fukui K, and Yonekura Y. Attention to emotion modulates fMRI activity in human right superior temporal sulcus. Brain Res Cogn Brain Res 12: 225231, 2001.[CrossRef][Medline]
Ojemann JG, Ojemann GA, and Lettich E. Neuronal activity related to faces and matching in human right nondominant temporal cortex. Brain 115: 113, 1992.
Ono T and Nishijo H. Neurophysiological basis of emotion in primates: neuronal responses in the monkey amygdala and anterior cingulate cortex. In: The New Cognitive Neurosciences, edited by Gazzaniga MS. Cambridge, MA: MIT Press, 2000, p. 10991114.
Perrett DI, Oram MW, Harries MH, Bevan R, Hietanen JK, Benson PJ, and Thomas S. Viewer-centred and object-centred coding of heads in the macaque temporal cortex. Exp Brain Res 86: 159173, 1991.[Web of Science][Medline]
Perrett DI, Smith PA, Potter DD, Mistlin AJ, Head AS, Milner AD, and Jeeves MA. Visual cells in the temporal cortex sensitive to face view and gaze direction. Proc R Soc Lond B Biol Sci 223: 293317, 1985.[Medline]
Pessoa L, McKenna M, Gutierrez E, and Ungerleider LG. Neural processing of emotional faces requires attention. Proc Natl Acad Sci USA 99: 1145811463, 2002.
Puce A, Allison T, Gore JC, and McCarthy G. Face-sensitive regions in human extrastriate cortex studied by functional MRI. J Neurophysiol 74: 11921199, 1995.
Rossion B, Caldara R, Seghier M, Schuller AM, Lazeyras F, and Mayer E. A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. Brain 126: 23812395, 2003.
Rotshtein P, Malach R, Hadar U, Graif M, and Hendler T. Feeling or features: different sensitivity to emotion in high-order visual cortex and amygdala. Neuron 32: 747757, 2001.[CrossRef][Web of Science][Medline]
Saxe R and Kanwisher N. People thinking about thinking people. The role of the temporo-parietal junction in "theory of mind." Neuroimage 19: 18351842, 2003.[CrossRef][Web of Science][Medline]
Schweinberger SR, Burton AM, and Kelly SW. Asymmetric dependencies in perceiving identity and emotion: experiments with morphed faces. Percept Psychophys 61: 11021115, 1999.[Web of Science][Medline]
Schweinberger SR and Soukup GR. Asymmetric relationships among perceptions of facial identity, emotion, and facial speech. J Exp Psychol Hum Percept Perform 24: 17481765, 1998.[CrossRef][Web of Science][Medline]
Sergent J, Ohta S, and MacDonald B. Functional neuroanatomy of face and object processing. A positron emission tomography study. Brain 115: 1536, 1992.
Sergent J, Ohta S, MacDonald B, and Zuck E. Segregated processing of facial identity and emotion in the human brain: a PET study. Vis Cogn 1: 349369, 1994.
Surguladze SA, Brammer MJ, Young AW, Andrew C, Travis MJ, Williams SC, and Phillips ML. A preferential increase in the extrastriate response to signals of danger. Neuroimage 19: 13171328, 2003.[CrossRef][Web of Science][Medline]
Tiberghien G, Baudouin JY, Guillaume F, and Montoute T. Should the temporal cortex be chopped in two? Cortex 39: 121126, 2003.[Web of Science][Medline]
Tranel D, Damasio H, and Damasio AR. A neural basis for the retrieval of conceptual knowledge. Neuropsychologia 35: 13191327, 1997.[CrossRef][Web of Science][Medline]
Vogels R, Biederman I, Bar M, and Lorincz A. Inferior temporal neurons show greater sensitivity to nonaccidental than to metric shape differences. J Cogn Neurosci 13: 444453, 2001.[CrossRef][Web of Science][Medline]
Vuilleumier P, Armony JL, Driver J, and Dolan RJ. Effects of attention and emotion on face processing in the human brain: an event-related fMRI study. Neuron 30: 829841, 2001.[CrossRef][Web of Science][Medline]
Vuilleumier P, Armony JL, Driver J, and Dolan RJ. Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nat Neurosci 6: 624631, 2003a.[CrossRef][Web of Science][Medline]
Vuilleumier P, Mohr C, Valenza N, Wetzel C, and Landis T. Hyperfamiliarity for unknown faces after left lateral temporo-occipital venous infarction: a double dissociation with prosopagnosia. Brain 126: 889907, 2003b.
Wada Y and Yamamoto T. Selective impairment of facial recognition due to a haematoma restricted to the right fusiform and lateral occipital region. J Neurol Neurosurg Psychiatry 71: 254257, 2001.
Whalen PJ, Rauch SL, Etcoff NL, McInerney SC, Lee MB, and Jenike MA. Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. J Neurosci 18: 411418, 1998.
Winston JS, O'Doherty J, and Dolan RJ. Common and distinct neural responses during direct and incidental processing of multiple facial emotions. Neuroimage 20: 8497, 2003a.[CrossRef][Web of Science][Medline]
Winston JS, Vuilleumier P, and Dolan RJ. Effects of low-spatial frequency components of fearful faces on fusiform cortex activity. Curr Biol 13: 18241829, 2003b.[CrossRef][Web of Science][Medline]
Wojciulik E, Kanwisher N, and Driver J. Covert visual attention modulates face-specific activity in the human fusiform gyrus: fMRI study. J Neurophysiol 79: 15741578, 1998.
Worsley K, Marrett S, Neelin P, Vandal AC, Friston KJ, and Evans AC. A unified statistical approach for determining significant signals in images of cerebral activation. Hum Brain Mapp 4: 5873, 1996.[CrossRef][Web of Science]
Young AW, Newcombe F, de Haan EH, Small M, and Hay DC. Face perception after brain injury. Selective impairments affecting identity and expression. Brain 116:941959, 1993.
This article has been cited by other articles:
![]() |
M. A. Pinsk, M. Arcaro, K. S. Weiner, J. F. Kalkus, S. J. Inati, C. G. Gross, and S. Kastner Neural Representations of Faces and Body Parts in Macaque and Human Cortex: A Comparative fMRI Study J Neurophysiol, May 1, 2009; 101(5): 2581 - 2600. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Aguado, A. Garcia-Gutierrez, and I. Serrano-Pedraza Symmetrical interaction of sex and expression in face classification tasks Atten Percept Psychophys, January 1, 2009; 71(1): 9 - 25. [Abstract] [PDF] |
||||
![]() |
C. S. Carter, D. M. Barch, R. Gur, R. Gur, A. Pinkham, and K. Ochsner CNTRICS Final Task Selection: Social Cognitive and Affective Neuroscience-Based Measures Schizophr Bull, January 1, 2009; 35(1): 153 - 162. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. ELLAMIL, J. M. SUSSKIND, and A. K. ANDERSON Examinations of identity invariance in facial expression adaptation Cogn Affect Behav Neurosci, September 1, 2008; 8(3): 273 - 281. [Abstract] [PDF] |
||||
![]() |
A. E. Pinkham, J. B. Hopfinger, K. Ruparel, and D. L. Penn An Investigation of the Relationship Between Activation of a Social Cognitive Neural Network and Social Functioning Schizophr Bull, July 1, 2008; 34(4): 688 - 697. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. L. LoPresti, K. Schon, M. D. Tricarico, J. D. Swisher, K. A. Celone, and C. E. Stern Working Memory for Social Cues Recruits Orbitofrontal Cortex and Amygdala: A Functional Magnetic Resonance Imaging Study of Delayed Matching to Sample for Emotional Expressions J. Neurosci., April 2, 2008; 28(14): 3718 - 3728. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Xu, P. Dayan, R. M. Lipkin, and N. Qian Adaptation across the Cortical Hierarchy: Low-Level Curve Adaptation Affects High-Level Facial-Expression Judgments J. Neurosci., March 26, 2008; 28(13): 3374 - 3383. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. P. Ewbank, W. A.P. Smith, E. R. Hancock, and T. J. Andrews The M170 Reflects a Viewpoint-Dependent Representation for Both Familiar and Unfamiliar Faces Cereb Cortex, February 1, 2008; 18(2): 364 - 370. [Abstract] [Full Text] [PDF] |
||||
![]() |
T.-W. Lee, R. J. Dolan, and H. D. Critchley Controlling Emotional Expression: Behavioral and Neural Correlates of Nonimitative Emotional Responses Cereb Cortex, January 1, 2008; 18(1): 104 - 113. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Kriegeskorte, E. Formisano, B. Sorger, and R. Goebel Individual faces elicit distinct response patterns in human anterior temporal cortex PNAS, December 18, 2007; 104(51): 20600 - 20605. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Rotshtein, P. Vuilleumier, J. Winston, J. Driver, and R. Dolan Distinct and Convergent Visual Processing of High and Low Spatial Frequency Information in Faces Cereb Cortex, November 1, 2007; 17(11): 2713 - 2724. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Furl, N. J. van Rijsbergen, A. Treves, K. J. Friston, and R. J. Dolan Experience-dependent coding of facial expression in superior temporal sulcus PNAS, August 14, 2007; 104(33): 13485 - 13489. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Kristjansson, P. Vuilleumier, S. Schwartz, E. Macaluso, and J. Driver Neural Basis for Priming of Pop-Out during Visual Search Revealed with fMRI Cereb Cortex, July 1, 2007; 17(7): 1612 - 1624. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Gilaie-Dotan and R. Malach Sub-exemplar Shape Tuning in Human Face-Related Areas Cereb Cortex, February 1, 2007; 17(2): 325 - 338. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Harris and K. Nakayama Rapid Face-Selective Adaptation of an Early Extrastriate Component in MEG Cereb Cortex, January 1, 2007; 17(1): 63 - 70. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. J Tate, H. Fischer, A. E Leigh, and K. M Kendrick Behavioural and neurophysiological evidence for face identity and face emotion processing in animals Phil Trans R Soc B, December 29, 2006; 361(1476): 2155 - 2172. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Ng, V. M. Ciaramitaro, S. Anstis, G. M. Boynton, and I. Fine Selectivity for the configural cues that identify the gender, ethnicity, and identity of faces in human cortex PNAS, December 19, 2006; 103(51): 19552 - 19557. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Schiltz, B. Sorger, R. Caldara, F. Ahmed, E. Mayer, R. Goebel, and B. Rossion Impaired Face Discrimination in Acquired Prosopagnosia Is Associated with Abnormal Response to Individual Faces in the Right Middle Fusiform Gyrus Cereb Cortex, April 1, 2006; 16(4): 574 - 586. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Sayres and K. Grill-Spector Object-Selective Cortex Exhibits Performance-Independent Repetition Suppression J Neurophysiol, February 1, 2006; 95(2): 995 - 1007. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. L. Hefter, D. S. Manoach, and J. J.S. Barton Perception of facial expression and facial identity in subjects with social developmental disorders Neurology, November 22, 2005; 65(10): 1620 - 1625. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. A. Pinsk, K. DeSimone, T. Moore, C. G. Gross, and S. Kastner Representations of faces and body parts in macaque temporal cortex: A functional MRI study PNAS, May 10, 2005; 102(19): 6996 - 7001. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |