Attractive faces can be considered to be a form of visual reward. Previous imaging studies have reported activity in reward structures including orbitofrontal cortex and nucleus accumbens during presentation of attractive faces. Given that these stimuli appear to act as rewards, we set out to explore whether it was possible to establish conditioning in human subjects by pairing presentation of arbitrary affectively neutral stimuli with subsequent presentation of attractive and unattractive faces. Furthermore, we scanned human subjects with functional magnetic resonance imaging (fMRI) while they underwent this conditioning procedure to determine whether a reward-prediction error signal is engaged during learning with attractive faces as is known to be the case for learning with other types of reward such as juice and money. Subjects showed changes in behavioral ratings to the conditioned stimuli (CS) when comparing post- to preconditioning evaluations, notably for those CSs paired with attractive female faces. We used a simple Rescorla-Wagner learning model to generate a reward-prediction error signal and entered this into a regression analysis with the fMRI data. We found significant prediction error-related activity in the ventral striatum during conditioning with attractive compared with unattractive faces. These findings suggest that an arbitrary stimulus can acquire conditioned value by being paired with pleasant visual stimuli just as with other types of reward such as money or juice. This learning process elicits a reward-prediction error signal in a main target structure of dopamine neurons: the ventral striatum. The findings we describe here may provide insights into the neural mechanisms tapped into by advertisers seeking to influence behavioral preferences by repeatedly exposing consumers to simple associations between products and rewarding visual stimuli such as pretty faces.
Faces convey a wealth of information and are perhaps the most important visual stimuli for humans in social environments (Adolphs 2001). The attractiveness of a face is a feature that we can perceive quite automatically (Olson and Marshuetz 2005), can subsequently motivate our behavior in terms of mate choice (Rhodes et al. 2005), and bias our beliefs about others' personality and expected success in life (Dion et al. 1972). The effect of attractiveness on human behavior has been documented in the workplace, where it has been shown that attractive individuals enjoy higher salaries (Hamermesh and Biddle 1994) and better employment prospects (Dipboye et al. 1977). These observations have led to the suggestion that preference for facial attractiveness may have evolved to enhance reproductive success (Rhodes 2006; Thornhill and Gangestad 1999).
Recent evidence indicates that attractive faces may act as a form of visual reinforcer as human subjects are prepared to work to gain access to them (Aharon et al. 2001). Although much is now known about the neural circuitry involved in processing the perceptual (Haxby et al. 2000; Ishai et al. 2005; Kanwisher et al. 1997) and affective aspects of facial stimuli (Adolphs 2001; Blair 2003; Whalen et al. 1998), the neural substrates of facial attractiveness are much less well understood. Nevertheless some preliminary studies investigating processing of facial attractiveness have implicated brain regions known to be involved in reward processing, such as the orbitofrontal cortex and ventral striatum (Aharon et al. 2001; Kranz and Ishai 2006; McClure et al. 2003; O'Doherty et al. 2003a,b; Rolls 2000).
Here we aim to address the manner in which facial attractiveness can influence one important aspect of human behavior: behavioral preference. Attractive faces have long been used in advertising as a means of modulating behavioral preferences for specific products. Indeed marketing research has shown that people will evaluate products more favorably when they are presented alongside physically attractive models (Baker and Churchill 1977; Smith and Engel 1968). One possible mechanism for this preference modulation effect is through classical conditioning, whereby an arbitrary neutral stimulus acquires affective value through repeated pairing with a stimulus that has preestablished value such as an attractive face.
In this study, we set out to elucidate the neural mechanisms of this phenomenon by scanning human subjects with functional magnetic resonance imaging (fMRI) while they learned an association between arbitrary affectively neutral visual stimuli (fractals) and attractive and unattractive male and female faces. Before and after the conditioning procedure, we took ratings of pleasantness and preference for the arbitrary fractal stimuli to establish whether behavioral preferences for these stimuli could be modulated as a function of conditioning with attractive faces.
We aimed to characterize the neural processes underlying learning of these preference associations. Modern learning theories propose that such reward-dependent learning is driven by the degree of surprise or unpredictability of a rewarding outcome, or more specifically, errors in predictions of reward (Rescorla and Wagner 1972). Electrophysiological studies in non-human primates implicate the phasic firing of midbrain dopaminergic neurons in encoding reward-prediction errors (Schultz et al. 1997). fMRI studies of human learning have found evidence of reward-prediction error-related activity in known projection sites of dopaminergic cells, especially the ventral striatum, during learning with other forms of natural and abstract rewards such as juice or money (McClure et al. 2003; O'Doherty et al. 2004; Tobler et al. 2006). Given that attractive faces can also be considered as a form of reward, we hypothesized that learning with attractive faces would also engage brain structures known to be involved in reward-prediction error coding such as the ventral striatum.
Twenty-eight subjects participated in this study (15 females and 13 males), ranging in age from 18 to 27 [mean: 20.8 ± 2.24 (SD) yr]. All subjects gave informed consent for the study, which was approved by the local research ethics committee. Due to technical difficulties (for 1 subject the experiment stopped during the study due to a software problem, and for 2 other subjects, part of the data set was lost in transfer), three subjects were excluded from the imaging analysis (n = 25), one subject was excluded from response time analysis (n = 27), and one subject's preference data were lost (n = 27).
The visual conditioned stimuli (CS) were complex abstract fractal images, and the unconditioned stimuli (UCS) were photographs of human male and female faces, attractive and unattractive. A set of 148 faces were previously rated by a separate group for attractiveness on a scale from 1 to 7 (O'Doherty et al. 2003b). Based on these ratings, eight faces were chosen to make up each of four conditions: female-attractive, male-attractive, female-unattractive, and male-unattractive. The faces had forward head position, and gazed forward with neutral to mildly happy expressions. The face images were masked to remove hair, were adjusted to be of approximately equal size and luminance, and centered in a 450 × 450-pixel gray background. We also used six abstract fractals, each centered in a170 × 170-pixel gray background. Stimuli were presented at a screen resolution of 800 × 600. Example stimuli are shown in the time course of a conditioning trial in Fig. 1A, and additional example face stimuli are shown in Fig. 1B. Stimuli were presented using Cogent 2000 developed by the Cogent 2000 team at the Functional Imaging Laboratory and the Institute of Cognitive Neuroscience and Cogent Graphics developed by John Romaya at the Laboratory of Neurobiology at the Wellcome Department of Imaging Neuroscience.
Subjects first completed a questionnaire in which they were asked to describe their sexual orientation by choosing from a set of labels (heterosexual, homosexual, bisexual, transgender, polyamorous, none). They were also asked to rate on a 10-point scale how interested they are in having sex with men and women and how sexually attractive they find men and women.
BEHAVIORAL MEASURES OF LEARNING AND PREFERENCE MODULATION.
During conditioning, attractive and unattractive faces were paired with affectively neutral fractal pictures. Subjects were first exposed to the fractal stimuli before conditioning to obtain pleasantness ratings and preference rankings. Pleasantness ratings were performed once subjects were installed in the scanner before fMRI data collection began. Subjects were first shown a screen with six fractal images; they were then shown each of the fractals once in random order and asked to verbally report a pleasantness rating between −10 and +10 (where −10 = very unpleasant, 0 = neutral, 10 = very pleasant). Next, subjects were presented with pairs of fractals and asked to indicate which of the two they preferred by pressing the left or right button on a two-button pad. Pairs were presented in random order with each combination presented three times and fractals randomly assigned to the left or right side of the screen. Subjects responded to a total of 45 pairs with each fractal appearing a total of 15 times.
During the conditioning procedure, subjects were asked to respond with a button press to indicate which side of the screen the fractal stimulus was presented on each trial. During conditioning, subjects were presented with each fractal a total of 48 times. These reaction times provided an additional on-line measure of conditioning (O'Doherty et al. 2006).
After conditioning, the preference and ratings tasks were repeated in that order; subjects were given the additional instruction that they should not try to match their previous answers but rather respond according to their present evaluation.
To assess explicit awareness of the contingencies, subjects were shown each of the six fractals in random order and asked how likely they thought it was that the fractal had been paired with an attractive face, using a scale from 0 to 10 (where 0 = not at all likely, and 10 = very likely). Subjects were also asked how unlikely it was that a stimulus was paired with an attractive face. We then asked subjects how likely and unlikely it was that each fractal had been paired with an unattractive face.
EVALUATION OF ATTRACTIVENESS OF FACE STIMULI.
The final task in the experiment was to evaluate the attractiveness of the faces. Subjects were presented with each of the 32 faces in random order and asked to verbally report a subjective rating of facial attractiveness on a scale from −10 to +10.
BEHAVIORAL DATA ANALYSIS.
We used differential ratings and preferences as an index of conditioning. We hypothesized that fractals paired with attractive faces would increase in pleasantness and become more preferred over fractals paired with unattractive faces especially for fractals paired with faces opposite in gender to the subject. A preliminary inspection of these data indicated that they were not normally distributed. Consequently, we used nonparametric statistics for all behavioral analyses in this study.
The pair-wise preference results were ranked based on the number of times a fractal was chosen as preferred, and category differences in ranking changes before and after conditioning were compared (e.g., change in rankings for fractals paired with attractive female faces compared with unattractive female faces).
Four of the fractals were randomly assigned to be paired with faces from one of the four face gender/attractiveness categories, and two were assigned to never be paired with any faces. The fractal/face categories were: attractive female, attractive male, unattractive female, unattractive male, and unpaired. Each trial began with the presentation of a fractal image, randomly displayed either to the left or right of a central fixation cross. This fractal remained on the screen for 1.5 s. On reinforced CS+ trials, after 1 s, a picture of a face appeared in the middle of the screen next to the fractal. The two appeared together for 500 ms, the fractal then disappeared while the face remained on the screen for another full second, followed by a fixation cross for 500 ms. The duration of each trial was 3 s with the face and fractal each presented for 1.5 s with 0.5 s of overlap. We chose to use a delay conditioning paradigm with a short interstimulus interval to maximize conditioning efficacy. The time course of a CS+ trial is shown in Fig. 1. To enhance conditioning, the first three trials of each condition were reinforced CS+ trials in which the face followed the fractal, whereas for the remainder of the experiment, 50% of trials were reinforced. To obtain a trial-based behavioral measure of learning during the study and also to ensure that subjects attended to the task, subjects were instructed to press the left or right button on a two button pad, to indicate which side of the screen the fractal appeared. They were also instructed to keep their attention directed toward the center of the screen throughout the experiment. There were 48 trials of each type, 50% of which were reinforced, and each of the eight faces in a category was presented up to three times. Along with the 288 conditioning trials, we included a set of 96 null events, during which the fixation cross was presented for 3 s, to mimic the effect of a jittered intertrial interval and facilitate separation of neural responses from consecutive trials. Trials were randomly ordered, and the total duration of the conditioning session was ∼20 min.
PREDICTION ERROR SIGNALS.
We used a simple trial-based Rescorla-Wagner rule to model trial-by-trial prediction errors in learning (Rescorla and Wagner 1972). This model uses a prediction error signal δ, which reflects the difference between the value of the outcome received on a given trial (R) and the value of the expected outcome on that trial (V): δ = R –V. The expected value V is then updated by adding δ weighted by a learning rate α: V = V + αδ.
In a follow-up region of interest analysis, we employed a real-time extension of the Rescorla-Wagner learning rule, a temporal difference model (Sutton 1988; Sutton and Barto 1990), in which the prediction error shifts backward in time from the face presentation to the cue presentation. We divided trials of each type in to early, middle, and late epochs and modeled the prediction error signal at the time of the face, 0.5 s before the face, and time of cue (1 s before the face), respectively.
The specific values used in our implementations of the model were the following: we modeled the presentation of a face with R = 1, the omission of a face with R = 0 (for both attractive and unattractive faces) and derived the learning rate (α) from subjects' behavioral responses. We used reaction times (responses to the conditioned stimuli) as a trial-by-trial measure of learning to derive model parameters from subjects' behavior. Reaction times have previously been shown to be modulated as a function of conditioning, and changes in reaction times over time have previously been found to correlate with reinforcement learning models (Critchley et al. 2002; Gottfried et al. 2002, 2003; Seymour et al. 2004). We derived learning signals for each subject based on their individual conditioning histories for a range of learning rates α (ranging from 0.01 to 0.5). For each type of trial, we averaged log-adjusted trial-by-trial response times across subjects and fit these to a regression model that included the averaged learning signal curves. To account for general changes in reaction time that would occur over the experiment, we included an additional regressor as a covariate of no interest that reflected the change in reaction times across the experiment in the neutral trials (specifically a spline-smoothed fit of the averaged reaction times from the unpaired trials). This method allowed us to determine the learning rates that gave the best fit to subjects' behavior (on average across subjects). We used the learning rates resulting from this procedure to model the fMRI data for all subjects, by regressing these signals against the brain imaging data as described below (the specific learning rates are given in results).
FMRI SCANNING PROCEDURE.
fMRI data were acquired on a Siemens AG (Erlangen, Germany) 3T TRIO MRI scanner; blood-oxygenation-level-dependent (BOLD) contrast was measured with gradient echo T2* weighted echo-planar images (EPI). Imaging parameters were optimized to minimize signal dropout in medial ventral prefrontal and anterior ventral striatum: we used a tilted acquisition sequence at 30° to the AC-PC line (Deichmann et al. 2003), and an eight-channel phased array coil that yields an ∼40% signal increase in this area over a standard coil. The first 5 volumes of 620 were discarded to permit T1 equilibration. Other parameters were as follows: in-plane resolution, 3 × 3 mm; slice thickness, 3 mm; repetition time, 2 s; echo time, 30 ms; field of view, 192 × 192 mm. A T1 weighted structural image was also acquired for each subject.
IMAGING DATA ANALYSIS.
fMRI data were preprocessed in SPM2 (http://www.fil.ion.ucl.ac.uk/spm/software/spm2/). Images were corrected for slice acquisition time within each volume, motion corrected by aligning to the first volume (Friston et al. 1995) and unwarped to correct for estimated movement-related deformations in the EPI field (Andersson et al. 2001). They were normalized to a standard EPI template in Montreal Neurological Institute space, and spatial smoothing was applied using a Gaussian kernel with full width at half-maximum of 8 mm. The normalization parameters estimated for each subject were also applied to their T1-weighted structural scans.
Statistical analysis was carried out using the general linear model with the canonical hemodynamic response function (HRF) as a basis set. We describe results from two main analyses, designed to examine stimulus driven effects and learning related effects respectively. In the first analysis, fractal and face presentation events were modeled as delta functions. In the second analysis, prediction error signals were entered as parametric regressors for each trial type at the time a face would be presented, independently of whether a face had actually been shown. For all models, the six ongoing motion parameters estimated during realignment were included as regressors of no interest. The results from each subject were taken to the random effects level by applying t-tests between contrast images to produce group statistical parametric maps. We focused our analyses on brain regions of interest, specifically the striatum, orbitofrontal cortex and amygdala.
FACE ATTRACTIVENESS RATINGS.
Consistent with previous studies using the same set of faces (O'Doherty et al. 2003b), subjects rated the faces in the attractive category as significantly more attractive than those in the unattractive category, for both female (Wilcoxon signed-rank test |Z| = 4.264, n = 28, P < 0.001) and male faces (|Z| = 4.623, n = 28, P < 0.001). Gender differences were observed in evaluations of male faces as female subjects rated them as significantly more attractive than did male subjects (Mann-Whitney |Z| = 2.374, n = 15, 13, P < 0.05 and |Z| = 2.097, n = 15, 13, P < 0.05, attractive and unattractive, respectively). There were no significant gender differences in evaluations of female faces. Attractiveness evaluations are shown in Fig. 2A.
SEXUAL ORIENTATION QUESTIONNAIRE.
Based on self-reports of sexual orientation, our subject group consisted of 14 heterosexual females and 1 bisexual female, 11 heterosexual males, and 2 bisexual males. Labels of sexual orientation were corroborated by ratings of attraction and sexual interest to the opposite sex: female heterosexual subjects rated their attraction to males to be on average 8 ± 0.6, whereas male heterosexual subjects rated their attraction to females as 9 ± 0.28. Similar scores were obtained on ratings of sexual interest in the opposite sex: 7.47 ± 0.41 in female subjects for males, 8.5 ± 1.5 in male subjects for females. The bisexual subjects rated their level of attraction and sexual interest for the opposite sex within the same range as the heterosexual subjects, and were therefore included in all analyses described here unless explicitly stated otherwise.
CHANGES IN PLEASANTNESS RATINGS OF STIMULI AS A FUNCTION OF CONDITIONING.
Significant differences in pleasantness ratings for fractal stimuli were found from before to after conditioning for the stimuli paired with attractive female faces (Wilcoxon signed-rank test, |Z| = 2.169, n = 28, P < 0.05) across all subjects (both male and female). This effect was also significant in the subgroup of male subjects (Wilcoxon signed-rank test, |Z| = 1.992, n = 13, P < 0.05) but not female subjects. We did not find a similar effect for the fractals paired with male faces in any of the subject groups. Male and female subjects showed no significant differences in pleasantness ratings. Figure 2B shows differences in pleasantness ratings from before to after conditioning for stimuli paired with attractive and unattractive faces plotted for all subjects and males and females separately.
CHANGES IN BEHAVIORAL PREFERENCE FOR FRACTAL STIMULI AS A FUNCTION OF CONDITIONING.
In male subjects, the increase in preferences for fractals paired with highly attractive female faces was significantly greater than for those paired with unattractive female faces (Wilcoxon signed-rank test, |Z| = 2.428, n = 13, P < 0.05), although this effect was not significant across all subjects. On the other hand, we did not find a similar effect in female subjects for those fractals paired with male faces. No significant effects were found in either male or female subjects for stimuli paired with same-sex attractive faces, and no significant gender differences in preference ratings were observed. Figure 2C shows differences in preference rankings for the fractal stimuli as a function of conditioning, plotted separately for stimuli paired with male and female faces, and groups of all, male and female subjects.
CORRELATIONS BETWEEN REACTION TIMES AND LEARNING MODEL.
Our regression analysis showed that the Rescorla-Wagner model with the best-fitting learning rate was significantly correlated with changes in subjects' reaction time data over the experiment for all four trial types in which subjects learned the predictive value of the fractal cues. The learning rates obtained for each trial type were [attractive female: 0.026 (R2 = 0.59, P < 0.05), attractive male: 0.04 (R2 = 0.43, P < 0.05), unattractive female: 0.038 (R2 = 0.48, P < 0.05), unattractive male: 0.04 (R2 = 0.57, P < 0.05)]. Subject averaged reaction times are shown separately for low (0.0–0.2)- and high (0.2–0.5)-value trials for each condition in Fig. 3. This figure shows that for all four face-paired conditions, in both genders, there is a slowing in reaction times as model-predicted reward value increases. The mean reaction times for each condition are 469.19 ± 88.66 (SE) ms (attractive female) 462.42 ± 87.38 ms (attractive male) 465.16 ± 87.90 ms (unattractive female), and 460.33 ± 86.99 ms (unattractive male).
PREDICTION ERROR CONTRASTS.
To identify brain regions responding to prediction errors during learning with attractive compared with unattractive faces, we performed a linear contrast between prediction error signals for attractive and unattractive faces.
We found significant activation in one of our a priori regions of interest: nucleus accumbens (NAcc) as shown in Fig. 4 A ([−9 15 −3], z = 3.38 and [9 15 −9], z = 3.12; significant at P < 0.001, uncorrected). These areas survive small volume correction using a sphere of 8-mm radius defined around coordinates derived from a previous demonstration of reward prediction error activity in the NAcc ([−11 11 −2] and [11 11 −2]) (Knutson et al. 2005). The peak in the left NAcc is also significant in the contrast of learning with opposite-sex attractive compared with opposite-sex unattractive faces in all subjects ([−9 15 −6], z = 3.52; P < 0.001 uncorrected), and the subset of heterosexual subjects ([−9 15 −6], z = 3.79; P < 0.001 uncorrected). The contrast between learning with same-sex attractive compared with unattractive faces did not show any significant activations. Activations for prediction error contrasts are shown in Table 1.
The prediction error contrast for learning with opposite-sex attractive compared with unattractive faces also showed activity in some of our other a priori regions of interest, namely bilateral medial orbitofrontal cortex ([−6, 33, −9], z = 3.63; P < 0.001 uncorrected) and ([9, 33, −12], z = 3.22; P < 0.001 uncorrected), and caudate nucleus ([9, 15, 6], z = 3.37; P < 0.001 uncorrected).
In the contrast of prediction error for learning with attractive compared with unattractive faces, we also found significant effects in the right and left inferior frontal gyrus (see Table 1). These areas remain significant when we restrict this contrast to opposite-sex faces.
TEST FOR LEARNING RELATED CHANGES OVER TIME.
To establish whether activity in NAcc is associated with a temporally evolving learning signal as opposed to nonlearning-related effects induced by the presence or absence of a face, we performed an additional analysis on the time-series data extracted from the peak voxel in NAcc (at [9 15 −9]). For this, we included in the analysis a prediction error regressor that temporally shifted from the time of face presentation to the time of cue presentation, using a real-time extension of the Rescorla Wagner learning rule: temporal difference learning (Sutton and Barto 1990). We included in the same analysis a regressor at the time of face presentation, only when faces were actually presented. The temporal difference prediction error signal was a significantly better fit to activity in the NAcc than the face regressor at P < 0.05, suggesting that activity in this structure reflects dynamic learning related changes and not merely effects relating to the presence of absence of a face.
SEPARATE PREDICTION ERRORS DURING LEARNING WITH ATTRACTIVE AND UNATTRACTION FACES.
Although comparing prediction error responses during learning with attractive and unattractive faces produced robust differences, due to the opposing direction of the prediction error signal, we also examined learning signals in response to both attractive and unattractive faces independently. A simple contrast for areas showing a positive correlation with the prediction error signal for attractive faces produced a peak in the right NAcc (at [6 15 −12]) that survived correction for small volume at P < 0.05 FDR corrected in a 5-mm sphere centered on the peak identified above ([9 15 −9]). A simple contrast to detect areas showing a negative correlation with the prediction error signal for unattractive faces, also produced a peak in the left NAcc (at [−6 12 −3]) which survived correction for small volume at P < 0.05 FDR corrected in a 5 -mm sphere centered on the NAcc peak identified above ([−9 15 −3]).
We found evidence for a positive correlation with prediction error signals during learning with both attractive and unattractive faces in the amygdala, another of our a-priori regions of interest: for attractive faces in right amygdala ([24 0 −25], z = 3.31 P < 0.001 uncorrected) and for unattractive faces in right amygdala ([18 −6 −21], z = 4.02 P < 0.001 uncorrected) and left amygdala ([-18 −6 −18], z = 3.26 P < 0.001 uncorrected).
We also extracted time courses for peak voxels in the region of individual subjects' left NAcc, and averaged over attractive and unattractive trials, subtracting the averaged trials for which no face was presented from the averaged trials for which a face was presented. The resulting time courses are shown in Fig. 4C (left) and indicate a positive increase in BOLD signal for positive prediction errors and decrease in signal for negative prediction errors. Figure 4C (middle) shows that the NAcc responds positively to the presentation of an attractive face and negatively to the omission of an attractive face, whereas Fig. 4C (right) shows that for unattractive faces this relationship is inverted with increased activation seen to the omission of a face.
PREDICTION ERROR RESPONSES TO LEARNING WITH SAME AND OPPOSITE-SEX FACES.
We explored prediction error responses to fractals paired with opposite- and same-sex faces by conducting a post hoc statistical analysis on the contrast estimates derived from the left ventral striatum (plotted in Fig. 4B) in heterosexual subjects. In male subjects, we found a significant difference in responses for attractive compared with unattractive female faces (|t| = 3.01, dof = 8, P < 0.05), but no difference for male faces, whereas in female subjects, we found a significant difference in contrast estimates for attractive compared with unattractive male faces (|t| = 3.16, dof = 12, P < 0.05) but not female faces. Pooling male and female subjects we found a significant effect of attractiveness when subjects were presented with opposite (|t| = 4.31, dof = 21, P < 0.001) but not same-sex faces.
MAIN EFFECT OF ATTRACTIVENESS.
We also tested for regions responding to receipt of the attractive faces themselves. For this, we performed a linear contrast of attractive-unattractive faces at the random effects level (Fig. 5 A) and found significant effects in medial OFC ([12 39 −9] z = 2.93) extending into medial prefrontal cortex, a region previously shown to be responsive to the receipt of attractive faces (Aharon et al. 2001;O'Doherty et al. 2003b). The OFC area survived correction for small volume at P < 0.05 FDR corrected in an 8-mm sphere centered around coordinates from a previous study of facial attractiveness (at [16 45 −11]) (from Aharon et al. 2001).
A number of other regions show responses to facial attractiveness (clusters >5 voxels significant at P < 0.001 uncorrected are tabulated in Table 2), including the left nucleus accumbens at [−9 15 −6] (z = 4.28; P < 0.001 uncorrected). The NAcc activity is in the same region we found to be responsive to prediction error. A post hoc inspection of the time course plots from these two regions shows that the NAcc demonstrates a response profile consistent with a reward prediction error and not face presentation per se, as this region not only increases after presentation of an attractive face but also increases after the omission of an unattractive face (Fig. 4C). On the other hand, the OFC area only showed increased activity to the presentation of an attractive face and showed no change in activity to any other condition, suggesting that this area is responding to the receipt of an attractive face and not a prediction error (Fig. 5B).
The impact an attractive face can have on human behavior, from product choice (Baker and Churchill 1977) to hiring preference (Dipboye et al. 1977), has been well documented. However, to our knowledge, this study marks the first demonstration of modulation of behavioral preference for a neutral visual stimulus by conditioned association with an attractive face. Subjects in our study rated neutral fractal images as significantly more pleasant after they had been repeatedly paired with attractive female faces. Our finding of a modulation of behavioral preference to previously neutral stimuli as a function of conditioning with attractive faces resonates with other studies that have found similar effects through acquisition of conditioned associations with other types of reinforcers such as food and money (Cox et al. 2005; Johnsrude et al. 2000; O'Doherty et al. 2006).
By measuring neural activity with fMRI while subjects acquired this association, we were able to observe learning-related activity in the brain as the association was formed. We found that reward-prediction errors were engaged in the ventral striatum, differentially for stimuli paired with attractive compared with unattractive faces. Prediction errors have been observed during learning with other types of reward, such as juice and money (Delgado et al. 2000; Tobler et al. 2006). The observation that attractive faces also engage these signals further reinforces the proposal that attractive faces can be considered to be a form of visual reward (Aharon et al. 2001; O'Doherty et al. 2003b). The present result also provides insight into the mechanism by which attractive faces transfer their rewarding properties to other stimuli.
It is notable that increases in activity occurred in the striatum in response to positive prediction error signals after the unexpected presentation of an attractive face, but the opposite effect was found in response to the unexpected presentation of an unattractive face in which case a decrease in signal was observed. These findings suggest that ventral striatum shows a very different response profile to prediction error signals during learning with attractive as opposed to unattractive faces. These results are similar to effects found for prediction error signals generated during learning with monetary reward and punishment (Kim et al. 2006). These results are also compatible with response profiles reported in striatum in fMRI studies involving delivery of monetary reward and punishment, whereby ventral striatum has been shown to increase in response to receipt of monetary reward and decrease in response to receipt of monetary loss (Delgado et al. 2000). By contrast, a different pattern of responses has been observed in ventral striatum in response to prediction errors produced during learning with other types of reinforcers such as somatosensory pain and even nonpreferred flavors (O'Doherty et al. 2006; Seymour et al. 2004). In these cases, an increase in signal has been reported in ventral striatum after the unexpected delivery of a cue signaling subsequent pain or unpleasant flavor. Thus ventral striatum appears to show very different neural responses as a function of learning with different types of reinforcers. This raises the question as to the nature of the difference between reinforcers that leads to such divergent response profiles. One possibility is that ventral striatum responds differently to learning with primary as opposed to secondary reinforcers. Money can be considered to be a secondary or learned reinforcer, whereas pain and food can be argued to be primary reinforcers (Rolls 2000). However, facial attractiveness is often suggested to be a primary reinforcer as judgments of facial attractiveness are suggested to be culturally invariant (Cunningham et al. 1995), and attractiveness has been argued to signal reproductive fitness (Rhodes et al. 2005). As a consequence, the fact that attractive faces and money are similar in the way they activate the striatum would appear to argue against a primary versus secondary reinforcer account of differential striatal function. An alternative possibility is that ventral striatum is involved not in learning about the sensory properties or abstract value of unconditioned stimuli but instead learns associations between arbitrary stimuli and the unconditioned responses produced by an unconditioned stimulus. Differences in the nature of the unconditioned responses produced by different reinforcers could potentially account for differential activity in the striatum. Future studies will be needed to investigate this possibility further.
Although we found an overall effect of attractiveness on prediction error activity in the ventral striatum, we also found that in this area the effect was significant when heterosexual subjects were presented with opposite-sex faces but not same-sex faces. That is, prediction error responses were enhanced when learning about attractive faces of the opposite sex in both genders. This suggests that ventral striatum may be involved in mediating learning about attributes linked to sexual preference as opposed to learning about more general aspects of visual esthetics (Kranz and Ishai 2006).
In contrast to the nucleus accumbens, the amygdala showed positive correlations with prediction error signals during learning with both attractive and unattractive faces, consistent with previous findings of a role for amygdala in conditioning involving both appetitive and aversive stimuli (Buchel 1998; Gottfried et al. 2003; LaBar 1998; Paton et al. 2006). More generally, these results add to an extensive prior literature implicating the amygdala in processing stimuli of both positive and negative valence (Garavan et al. 2001; Hamann et al. 1999; Winston et al. 2003).
The results of this study also have important implications for understanding the underlying mechanisms by which product advertising can influence behavioral preference in the marketplace. Marketers have long attempted to bias consumer preference by pairing a particular product with another stimulus that is already highly valued, such as an attractive face. Indeed, changes in product evaluations and preference have been observed in behavioral experiments as a function of such pairing procedures (Gorn et al. 1993; Smith and Engel 1968). However, the precise mechanism by which preference modulation takes place has remained an open question. One possibility is that changes in preference evaluations occur through cognitive appraisal or top down modulation of affect (as in cognitive appraisal) (cf. Folkman et al. 1986). Another possibility is that preference evaluations occur as a function of classical conditioning (Baeyens et al. 1993). We directly tested this hypothesis using a classical conditioning study that, compared with previous neuroimaging studies of classical conditioning (Gottfried et al. 2002; Tobler et al. 2006), used a relatively long (1.5 s) UCS duration. Although studies examining the effect of UCS duration have shown mixed results (Bitterman et al. 1952; Kawai and Imada 1996; Tait et al. 1983), we chose a longer UCS presentation to increase the salience of the face stimuli. Our results provide evidence that the change in preference likely occurs as a function of classical conditioning by showing that similar neural mechanisms are engaged during evaluative preference modulation as are engaged during other types of classical conditioning. Moreover, the fact that evaluative preference modulation specifically engages prediction error signals in the ventral striatum, suggests that this procedure may recruit dopamine neurons in the midbrain as is known to be the case during learning with other kinds of reward in non-human primates (Hollerman and Schultz 1998). Consistent with the preceding suggestion, a recent fMRI study has shown that prediction error signals expressed in the ventral striatum during reward-learning can be modulated through pharmacological manipulation of dopamine levels in humans (Pessiglione et al. 2006), indicating that the source of such signals in human fMRI studies may in part be attributable to the afferent input from dopamine neurons.
To conclude, in the present study, we have found significant prediction error-related activity in the ventral striatum during conditioning with attractive compared unattractive faces. These findings suggest that an arbitrary stimulus can acquire conditioned value by being paired with pleasant visual stimuli just as with other types of reward like money or juice. Such a learning process elicits a reward-prediction error signal in a main target structure of dopamine neurons: the ventral striatum. The learning process we describe here may provide insights into the neural mechanisms used in advertising to influence behavioral preferences, whereby consumers are exposed repeatedly to simple associations between products and rewarding visual stimuli such as pretty faces.
This research is supported by a grant to J. O'Doherty from the Gimbel Discovery Fund in Neuroscience.
We thank M. Tyzska and S. Flaherty for assistance with the functional imaging.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2007 by the American Physiological Society