Risk and ambiguity are two conditions in which the consequences of possible outcomes are not certain. Under risk, the probabilities of different outcomes can be estimated, whereas under ambiguity, even these probabilities are not known. Although most people exhibit at least some aversion to both risk and ambiguity, the degree of these aversions is largely uncorrelated across subjects, suggesting that risk aversion and ambiguity aversion are distinct phenomena. Previous studies have shown differences in brain activations for risky and ambiguous choices and have identified neural mechanisms that may mediate transitions from conditions of ambiguity to conditions of risk. Unknown, however, is whether the value of risky and ambiguous options is necessarily represented by two distinct systems or whether a common mechanism can be identified. To answer this question, we compared the neural representation of subjective value under risk and ambiguity. fMRI was used to track brain activation while subjects made choices regarding options that varied systematically in the amount of money offered and in either the probability of obtaining that amount or the level of ambiguity around that probability. A common system, consisting of at least the striatum and the medial prefrontal cortex, was found to represent subjective value under both conditions.
For many of the decisions we make, we must choose between outcomes that are not certain. In some cases, we can at least assess the probabilities for different outcomes, such as when a fair coin is tossed, whereas in other cases, even the probabilistic structure of the possible outcomes is unknown. In economics, the first condition is termed “risk” and the second “ambiguity,” and a long line of studies has shown that most people are averse to both of these conditions. Risk aversion is the tendency to prefer high probabilities of low payoffs to low probabilities of high payoffs even if the expected value (EV, the product of probability and amount, see Glimcher 2008 and Weber and Camerer 1987 for reviews) is higher for the latter option (Bernoulli 1738/1954). For example, many people will take $50 for sure over a 50% chance of winning $120, although the EV of the risky prospect is $60 (Abdellaoui et al. 2007). Ambiguity aversion can be illustrated using the Ellsberg paradox (Ellsberg 1961). Imagine two urns, each containing 60 red and blue poker chips. In one urn, 30 of the chips are red and 30 are blue (risky urn). In the other urn, the composition of red and blue chips is unknown (ambiguous urn). When subjects are asked to choose one of the urns and bet on the color of a chip drawn from that urn, most prefer to bet on the risky urn, even if it offers a lower payoff than the ambiguous one. Note that the winning probability in both cases is the same. For the risky urn, the probability of drawing either color is 0.5. For the ambiguous urn, the probability of drawing a particular color is unknown, but because the subject picks the winning color, the probability is again 0.5. Still, most people prefer to avoid ambiguous options, a finding replicated many times (Camerer and Weber 1992). Ambiguity aversion has also been observed under conditions of “partial” ambiguity, where although the exact probability for a given outcome was not known, a range of possible probabilities could be estimated (e.g., between 0.2 and 0.8) (Becker and Brownson 1964; Curley and Yates 1985; Rustichini et al. 2005).
Risk and ambiguity aversions are two different phenomena of very different magnitudes: risk aversion is simply a trade-off between amount and probability according to each individual's taste. Ambiguity aversion, which tends to be much stronger, is a puzzling phenomenon: in the normative sense, it seems irrational; in the practical sense, it leads to highly disadvantageous results in many domains including health, finance, and legal issues (Camerer and Weber 1992). Although several explanations for ambiguity aversion have been suggested [e.g., sense of incompetence (Heath and Tversky 1991), comparative ignorance (Fox and Tversky 1995), informed opponent (Kuhberger and Perner 2003), other evaluation (Curley et al. 1986); see Camerer and Weber (1992) for review], its source remains unclear.
The marked difference in behavior between the two decision situations raises several questions at the neural level. One question is simply how the brain processes different situations in which the outcome is not known with certainty: are different circuits involved in the processing of risk and ambiguity or does a single circuit signal each state with a different pattern of activation? Another question is which neural structures mediate the common transition from ambiguity to risk that occurs when ambiguous probabilities are repeatedly sampled and the underlying probability distribution learnt. A final intriguing question is how the value of risky and ambiguous options is represented neurally, and whether distinct neural systems represent value under each condition.
To answer the first question Hsu and colleagues (2005) used functional MRI (fMRI) to compare neural activity in response to ambiguous and risky options and found higher activation in the orbitofrontal cortex (OFC) to ambiguity compared with risk. The authors have also found that patients with OFC lesions did not exhibit either risk or ambiguity aversion. Taking these results together, it seems that the same area, the OFC, is involved in the processing of both risk and ambiguity and that it has a role in signaling how much is unknown, showing higher activation when less is known.
Huettel and colleagues (2006) examined the second question by employing a design in which ambiguous lotteries were resolved at the end of each trial, allowing subjects to learn the hidden probability distribution over the course of the experimental session. They found higher activity for ambiguous compared with risky lotteries in inferior frontal gyrus (IFG), anterior insula and posterior parietal cortex. Activity in IFG was also correlated with the level of ambiguity aversion across subjects, suggesting that this area at least has a role in the process of resolving ambiguity. This result has recently been extended to negative outcomes in the absence of choice (Bach et al. 2009).
However, no previous study addressed the question of the neural representation of subjective value under risk and ambiguity. A similar question has been raised for other types of decisions, such as intertemporal choice (Kable and Glimcher 2007; McClure et al. 2004a), and has been the subject of much debate. Examining the representation of value under risk and ambiguity is therefore of interest both for understanding the neural processing of uncertainty and in relation to the representation of value in general.
Here we asked how the brain represents subjective value under risk and ambiguity, and specifically whether shared or distinct neural mechanisms underlie the encoding of value under these two conditions. To answer this question, we had subjects make decisions under different levels of either risk or ambiguity while we measured neural activity using fMRI. We then used behavioral data to estimate the subjective value1 (SV) that each option had to each individual subject and looked for neural correlations with that measure separately under risk and ambiguity.
Our results reveal several areas, including the striatum and the medial prefrontal cortex (MPFC), the activity of which was significantly correlated with SV. Crucially, within the limits of 3T fMRI, all the areas that provided information about SV in ambiguous trials also provided such information in risky trials and vice versa. The results extend previous reports of the neural processing of risky and ambiguous choices by outlining a unified valuation system that represents SV under both risk and ambiguity.
Twenty-two healthy volunteers (14 women, 20 right-handed, ages: 19–35) participated in experiment 1, and 15 healthy volunteers (10 women, all right-handed, ages: 20–41) participated in experiment 2. Of the subjects who participated in experiment 1, 18 participated in two scanning sessions and 4 participated in a single session. Data from four subjects in experiment 1 and four subjects in experiment 2 were discarded: four for excessive head motion (>2 mm), one for problems in data acquisition (spikes in the images), two because their behavior was not lawful in the amount of the options (i.e., they did not prefer higher amounts to lower ones), suggesting that they failed to understand the task, and one whose behavior could not be fit with our behavioral model. Procedures were in compliance with the safety guidelines for MRI research and approved by the University Committee on Activities Involving Human Subjects at New York University. All subjects had normal or corrected to normal vision and provided written informed consent.
fMRI at 3T (Allegra, Siemens, Erlangen, Germany) was used to measure blood-oxygen-level-dependent (BOLD) changes in cortical activity. During each fMRI scan, a time series of volumes was acquired using a T2*-weighted EPI pulse sequence (TR: 2,000 ms, TE: 30 ms, flip angle: 75°, 36 3-mm slices with no inter-slice gap, in-plane resolution 3 × 3 mm, FOV: 192 mm). Images were acquired using a custom RF coil (NM-011 transmit head coil, NOVA Medical, Wakefield, MA). In addition, T1-weighted high-resolution (1 × 1 × 1 mm3) anatomical images were acquired with an MP-RAGE pulse sequence and used for volume-based statistical analysis. To minimize head movement, subjects’ heads were stabilized with foam padding. Stimuli were projected onto a screen at the back of the scanner, and subjects viewed them through a mirror attached to the head coil.
EXPERIMENT 1: CHOICE UNDER RISK AND AMBIGUITY.
In each trial, subjects were presented with a lottery of a varying winning probability or ambiguity level and a varying amount. Subjects had to indicate whether they wanted to play that lottery or whether they preferred to play a reference lottery, which was the same for all trials (50% chance of winning $5). The reference lottery was presented to the subjects before the beginning of the experiment. The changing lottery appeared on the screen in the form of an “urn” painted partly red and partly blue (Fig. 1A). Subjects were told beforehand that all the urns that they would see during the experiment contain a total number of 60 poker chips but that the relative numbers of red and blue chips would be different in different urns. The percentages of red and blue chips were indicated by the red and blue regions of the urn. Numbers next to the red and blue areas represented the amounts of money that could be made if a chip of that color were drawn from the physical urn to which the display corresponded. Those amounts were in an imaginary currency, the exchange rate to dollars of which was 10 to 1. For example, in Fig. 1A, if the subject draws a red chip, she will win $18, whereas she will win nothing if a blue chip is drawn.
Each stimulus was presented for 2 s, followed by a 6-s delay period during which a white fixation dot was presented at the center of the screen. The fixation dot then changed into green, cuing subjects to press one of two buttons on a response box to indicate their choice between the lottery on the screen and the reference. The response had to be made within 1.5 s and was followed by a 0.5-s visual presentation of the pressed button (Fig. 1A). The buttons assigned for the reference and for the option on the screen were counterbalanced across subjects. Trials were interleaved with 10-s fixation periods.
In half of the trials, part of the urn was hidden by a gray occluder, which was always placed over the center of the image (Fig. 1B, left). The probability of drawing a chip of a certain color was therefore incompletely known or ambiguous (ambiguous trials). For example, in the leftmost urns in Fig. 1B, 25% of the chips are occluded, and thus the probability of drawing a red chip can be anywhere between 37.5% (if all the chips behind the occluder are blue) and 62.5% (if all the chips behind the occluder are red). Similarly, the probability of drawing a blue chip can also be anywhere between these two values. Increasing the occluder size increases the ambiguity level or the range of possible probabilities for drawing a red or blue chip. Three different occluder sizes (covering 25, 50, or 75% of the urn) were used in the experiment.
In the other half of the trials, the entire urn was visible such that subjects had complete information about the ratio of red and blue chips in the urn (risky urns; Fig. 1B, right). Three winning probabilities were used, (0.13, 0.25, and 0.38), each corresponding to the worst possible probability that could have been encountered under one of the ambiguity levels.
Subjects were told that each image on the screen represented a physical bag containing physical poker chips in it. Each unique image corresponded to one unique physical bag. Thus each and every time they encountered a 25% ambiguous display, they were making choices about draws from the same bag. The three ambiguous bags were sealed and were presented to the subjects before the beginning of the experiment to ensure that the subjects were convinced that the number of red and blue chips could not be adjusted by the experimenters after the subjects had made their choices. Subjects signed their names across the seal and checked the bags at the end of the experiment to make sure that they had remained sealed throughout and after the experiment.
In half the trials presented to the subjects from each bag red was associated with winning a positive amount of money and blue yielded a zero amount. In the other half of trials, the contingencies were reversed. It is important to note that this design ensured that the objective winning probability in all the ambiguous trials, averaging across the 50% of red-winning and 50% of blue-winning trials, was effectively fixed at 0.5.
Five amounts (5, 9.5, 18, 34, and 65 dollars) were used at each risk and ambiguity level, yielding 60 unique trial types [(3 probabilities + 3 ambiguity levels) × (5 amounts) × (2 colors)]. The amounts were varied slightly (±$0.1) on each trial to prevent subjects from developing automatic responses to particular lotteries.
Subjects participated in one or two sessions of six scans each. Each scan started with a 10-s fixation period followed by a single choice trial that was modeled separately in the subsequent analysis. Thirty choice trials were then presented in a random order, thus resulting in 630-s-long scans. Each pair of scans contained a single repetition of each unique choice. Subjects went through a few practice trials, first outside of the scanner and then inside before beginning the actual experiment.
As subjects had been informed at the beginning of the experiment, at the end of the experiment, three trials from each session were randomly selected and played for real money. To select each of those trials, subjects first tossed a die to select one of the six scans in the session and then drew a numbered chip from an opaque bag containing 31 chips. The number on the chip indicated which trial in that scan would be played for real money. Subjects then drew a chip from the bag chosen by them in each of the trials and were paid according to the chip's color and the payment contingency on that trial. Those earnings were made in addition to the show-up fee. Subjects were informed that they would conduct all of these procedures before the experiment began.
Providing subjects with real monetary rewards was crucial because it has often been shown that subjects can behave differently when they are making real decisions for real money versus when they are making “as-if” decisions that have no financial impact on their lives (Smith 1991). On the other hand, paying each and every choice would have been highly problematic: first, either the cumulative payoff would have become unfeasibly expensive or the value of each decision becomes so small that they begin to approach the as-if study. Furthermore, if one played every trial for a real payoff, then the outcome of the entire “portfolio” of choices would be close to a certainty. To take a simple example of this portfolio effect, if a subject goes through 100 trials in which the winning probability is 25%, then on average 25% of the trials will pay off, and the variability around this average will be very small. Our payment mechanism ensures that on the one hand subjects treat every trial as if they will be paid according to the outcome of that trial (because they do not know in advance which trials will be selected) and on the other hand they will not be able to create a portfolio. We also note that this is the standard payoff mechanism employed in behavioral and experimental economics.
EXPERIMENT 2: CHOICE UNDER RISK—BROAD RANGE OF AMOUNTS AND PROBABILITIES.
Experimental design was similar to experiment 1 except that all trials were unambiguous and that a broader range of probabilities (0.2, 0.4, 0.6, or 0.8) and amounts (11, 12, 15, 25, 40, 100, 170, or 250 dollars) was used, resulting in 64 unique trial types (4 probabilities × 8 amounts × 2 colors). The reference option was also changed (to ensure well-centered indifference points) and consisted of a sure bet of $10.
All subjects participated in six scans, which took place in one scanning session. Each scan started with a 10-s fixation period followed by a single-choice trial that was discarded from the neural analysis. Thirty-two choice trials were then presented in a random order, thus resulting in 670-s-long scans. At the end of the experiment, three trials were randomly selected and played for real money similar to experiment 1.
To derive the risk and ambiguity attitudes of each individual subject, we modeled the SV of each option using a power function (Kahneman and Tversky 1979), which takes into account the effect of ambiguity on the perceived probability (Gilboa and Schmeidler 1989) where p is the objective probability (experiment 1: 0.13, 0.25, or 0.38 for risky options, 0.5 for ambiguous options, and 0.5 for the reference; experiment 2: 0.2, 0.4, 0.6, or 0.8 for the lotteries and 1 for the sure bet), A is the ambiguity level (the fraction of the total probability that is unknown: 0 for risky options, 0.25, 0.5, or 0.75 for ambiguous trials and 0 for the reference), V is the amount that can be won (experiment 1: 5, 9.5, 18, 34, or 65 dollars for the lotteries and 5 dollars for the reference; experiment 2: 11, 12, 15, 25, 40, 100, 170, or 250 dollars for the lotteries and 10 for the sure bet), and α and β are subject-specific risk and ambiguity attitude parameters, respectively. We selected this representation not based on any belief that this function uniquely describes ambiguity aversion but simply because it provides a simple and behaviorally predictive assessment of the relative values of risky and ambiguous lotteries at a subject-by-subject level. Note also that the probability range, 0.13–0.5, largely obviated the need to consider a probability weighting function (e.g., Tversky and Kahneman 1992), which has been shown to be significant only for probabilities that are close to 0 or 1 (Prelec 1998).
Using maximum likelihood, the choice data of each subject was fit to a single logistic function of the form where Pv is the probability that the subject chose the variable lottery, SVF and SVV are the SVs of the fixed and variable options, respectively, and γ is the slope of the logistic function, which is a third subject-specific parameter.
fMRI data were analyzed with the BrainVoyager QX software package (Brain Innovation, Masstricht, Netherlands) and with additional in-house software written in Matlab (MathWorks, Natick, MA). Preprocessing of functional scans included discarding the first three volumes, slice scan time correction, inter- and intrasession three-dimensional motion correction and removal of low frequencies up to five cycles per scan (linear trend removal and high-pass filtering), and spatial smoothing using a Gaussian filter (8-mm full-width at half-maximum value, FWHM). The images were then co-registered with each subject's high-resolution anatomical scan, rotated into the AC-PC plane, and normalized into Talairach space (Talairach and Tournoux 1988).
Statistical analysis was based on a general linear model (GLM) (Friston et al. 1995). The time course of activity of each voxel was modeled as a sustained response during each trial, convolved with a standard estimate of the hemodynamic impulse response function (Boynton et al. 1996).
The main model consisted of five predictors: three dummy predictors for mean activation in risky trials, ambiguous trials, and the first trial of each scan, and two parametric predictors for the SV of risky and ambiguous trials. The SV of each trial was calculated using the individual subject- and session-specific α and β, which were obtained from the behavioral fit. Because the reference option was always the same, we used the SV of the variable lottery alone in the predictor. The parametric predictors were normalized together to a range of 0–1. Activation during intertrial intervals (10-s fixation periods) served as baseline. The activity time course of each voxel in each scan was converted to percent signal change (PSC), and the model was independently fit to each voxel's PSC, yielding five coefficients for each subject, including one for SV under ambiguity and one for SV under risk. These results were used in a group random-effects analysis, which tested whether the mean effect at each voxel was significantly different from zero across subjects. The maps in Fig. 4 highlight voxels that showed a significant effect for SV under ambiguity (top) or under risk (bottom).
Regions of interest (ROIs) were defined using the first scanning session only and included voxels that both passed the per-voxel statistical threshold and were part of clusters of at least six contiguous functional voxels. Time course from the second session was then sampled in each ROI and averaged across all voxels in the ROI, and the GLM was fit to that mean time course. Figure 5 presents the coefficients of the parametric predictors obtained in the ROI-based GLM. Note that because one data set was used to localize ROIs and another to sample activation, the results are statistically unbiased.
To look for activation that is correlated with the ambiguity level, we constructed an additional model in which the parametric predictors were replaced by three new predictors: amount, probability, and ambiguity level. Figure 7 highlights voxels in which the correlation with the ambiguity level in the first session was significant in a random-effects group analysis.
The main analysis was similar to experiment 1 but with only three predictors: two dummy predictors for mean activation and for the first trial in each scan and one parametric predictor for SV. The maps in Fig. 6 highlight voxels that showed a significant effect for SV in a random-effects group analysis.
Three additional models were calculated in which the SV predictor was replaced by: the amount offered in each trial, the winning probability in each trial, and the EV of each trial (the product of amount and probability). Supplementary Figs. S3A and S42 present the voxels that showed a significant effect using each of these models. ROIs for Supplementary Fig. S3B were defined using the SV GLM in odd scans, and the three different GLM's were then fit to the mean time course of even scans in each ROI.
PSYCHOPHYSIOLOGICAL INTERACTION ANALYSIS (PPI).
To examine the possible effect of experimental condition on the connectivity patterns between different areas, we conducted PPI analyses (Friston et al. 1997). In each model, one of the predictors was the time course sampled from one of the ROIs identified in the experiment, a second predictor was a task-related contrast: either mean activity under ambiguity compared with mean activity under risk or SV under ambiguity compared with SV under risk, and a third predictor consisted of the interaction between the activation and contrast predictors. An additional predictor consisted of the global activation averaged across the entire brain. With such an analysis, a significant coefficient of the interaction in a target voxel can be interpreted either as a modulation exerted by the task on the connectivity pattern between that voxel and the seed or as a modulation exerted by the activation in the seed on the task-related activation in the target voxel.3 Supplementary Fig. S6 highlights voxels in which either the coefficient of the seed activation (Fig. S6A) or the coefficient of the interaction term (B) were significantly different from 0 in a random-effects group analysis.
Calculation of significance values in the activation maps was based on the individual voxel significance and on the minimum cluster size (Forman et al. 1995). The probability of a false positive was determined from the frequency count of cluster sizes within the entire brain using a Monte Carlo simulation. To achieve a corrected threshold of P < 0.05, a per-voxel threshold of P < 0.001 and a cluster size of 21 functional voxels were used. However, an almost identical activation maps was obtained when a much smaller cluster size (6 functional voxels) was used with the exception of an additional cluster identified in the right amygdala.
The time course presented in Fig. 7 was obtained from averaging the time course in all the voxels within the ROI. The first three time points of each trial served as baseline for calculating the PSC. Repetitions of each condition were averaged within subject and finally across subjects.
On each trial, subjects chose between a variable lottery, which changed in payoff amount and either probability or ambiguity level, and a reference lottery (0.5 probability of winning $5). In risky trials (those with ambiguity = 0), the winning probability of the variable option was precisely indicated by the graphics of the stimulus (0.13, 0.25, or 0.38; Fig. 1B, right), whereas in ambiguous trials, part of the information about probability was missing, rendering the trial probability partially ambiguous (25, 50, or 75% occlusion; Fig. 1B, left). Subjects made between 180 and 360 choices in one or two scanning sessions. All subjects performed the task well only missing on average 2 ± 3 (SD) trials per session.
To model the SV that each option had to each individual subject, we used the well-known model of Gilboa and Schmeidler (Gilboa and Schmeidler 1989) (1) where SV is the SV, p is the objective probability, A is the ambiguity level, V is the amount, and α and β are subject-specific parameters for risk and ambiguity attitudes, respectively, with β effectively capturing the relative values a given subject places behaviorally on ambiguous versus risky lotteries (for more details, see methods).
For a subject who is unaffected by ambiguity, β will be 0, and the model will be reduced to a power utility function of a lottery the winning probability of which is 0.5 in all of the ambiguous lotteries we examined. A subject who is averse to ambiguity will have a positive β and will behave as if the winning probability is <0.5 and a subject who seeks ambiguity will have a negative β and will behave as if the winning probability is >0.5. Similarly, a subject who is risk-neutral will have an α of 1, a risk-averse subject will have an α <1, and a risk-seeking subject will have an α >1.
The choice data of each subject was fit to a single logistic function of the form (2) where Pv is the probability that the subject chose the variable lottery, SVF and SVV are the SVs of the fixed and variable options, respectively, and γ is the slope of the logistic function, or equivalently a noise parameter.
This function characterized well the behavior of all but one subject (median: R2 = 0.67, range: 0.37–0.87) and parameters estimated by the function were stable across sessions (Supplementary Fig. S1). Most subjects exhibited both risk and ambiguity aversion (as reported previously in the literature), and they varied widely in the degree of these aversions (mean α: session 1, 0.6 ± 0.2; session 2, 0.7 ± 0.3; range, 0.27–1.20; mean β: session 1, 0.7 ± 0.2; session 2, 0.6 ± 0.3; range, −0.01–1.03). Figure 2 presents the choice curves of three subjects. Subject 1 was strongly averse to risk (α = 0.49) and extremely averse to ambiguity (β = 0.93). Note that a β of 1 indicates a subject who behaves as if the entire probability indicator hidden behind the gray bar in the display ranges against the possibility of them winning the lottery. In other words, they behave as if the winning probability is the worst possible probability commensurate with the display (see Maccheroni et al. 2006 for a mathematical treatment of this belief). Because the objective probabilities in the risky (unambiguous) lotteries we examined were specifically chosen to be equal to the worst possible probabilities in the ambiguous displays, the choice curves under ambiguity for such an extremely ambiguity-averse subject should be identical to her choice curves under risk. Indeed, subject 1's choice curves under ambiguity closely resemble her choice curves under risk. For example, in the highest ambiguity condition (75%), this subject chose the reference when the ambiguous option offered as much as $34. Because the objective winning probability in all of the ambiguous conditions was 0.5, just like the winning probability of the reference lottery, this subject has effectively preferred (in a normative sense) a 0.5 chance of winning $5 over a 0.5 chance of winning $34.
Subject 2 had a similar degree of aversion to risk (α = 0.50). However, this subject was less averse to ambiguity (β = 0.67), which can be seen from the fact that her ambiguity curves are closer to each other when compared with the ambiguity curves of subject 1. For a perfectly rational decision-maker (in the normative sense), β will be 0, and she will behave according to the objective probability, which is 0.5 for all ambiguity levels. For the lotteries we examined, such a chooser would show choice curves for ambiguous lotteries that were all completely overlapping (regardless of her degree of risk aversion) because the true probability of winning is the same in all three conditions. Subject 3 was less risk averse compared with the first two subjects, relying much more on the EV of the lotteries to guide her choices (α = 0.87) but was strongly averse to ambiguity (β = 0.82).
These few examples suggest that subjects’ attitudes toward risk and ambiguity are independent of each other. Indeed looking at the population we did not find a significant correlation between the levels of risk and ambiguity aversion across subjects (R2 = 0.052, P = 0.2, Fig. 3) in our admittedly small sample, similar to previous studies (e.g., Hogarth and Einhorn 1990), but it should be noted that an analysis of a much larger sample does suggest a weak behavioral correlation between risk and ambiguity attitudes (Bossaerts et al. 2009).
Finally, note that by choosing a particular model to represent the relative values of risky and ambiguous lotteries, we did not mean to make any claims at the mechanistic level. What we were looking for was a compact representation of the behavioral data that we could use for the neural analysis. Any of the numerous models that have been suggested for the SV of ambiguous options (Camerer and Weber 1992) could have been used, as long as the selected function predicted the data well. To demonstrate this point, we also fit the data using a model in which the effect of ambiguity on the subjective probability is exponential rather than linear (Hsu et al. 2005) (3) and obtained almost identical fits (Supplementary Fig. S2). Our results were thus robust to SV model choice.
The parameters obtained from the behavioral fit were used to infer the SV that each combination of amount, probability, and ambiguity level had for each individual subject as specified by the behaviorally derived equation shown in the preceding text. This measure of SV served as a common currency for value in different trials such that we could now directly compare the neural coding of value under conditions of both risk and ambiguity to assess the independence of the neural substrates for risk and ambiguity encoding.
The SVs inferred from behavior were thus used to construct two separate predictors, SV of ambiguous trials and SV of risky trials. Both predictors were included in a single GLM and were used to search for brain areas the activity of which was correlated with SV under ambiguity and for areas the activity of which was correlated with SV under risk (Fig. 4). Significant correlation with SV under ambiguity (random effects group analysis, n = 20, P < 0.001 per voxel, P < 0.05 corrected for cluster size) was found in the caudate, extending to the ventral striatum, the anterior cingulate extending to the medial prefrontal cortex (MPFC), the posterior cingulate cortex (PCC), left amygdala and anterior superior temporal sulcus (STS) (Fig. 4, top, and Table 1). No significant correlation with SV under risk was observed at the same statistical threshold. However, reducing the threshold (P < 0.01 per voxel) revealed significant correlation in similar regions of the striatum and the MPFC at the same cluster size (Fig. 4, bottom, and Table 1). Correlation was not observed in caudate, the PCC, the STS, or the amygdala even at the more liberal threshold.
This initial finding raises the possibility that areas of the caudate, the PCC, the STS or the left amygdala might be uniquely activated under conditions of ambiguity but not under conditions of risk. To test this hypothesis, we inspected the beta coefficients of the two predictors (the strength of the correlations between the fMRI response and the risky and ambiguous SVs) averaged across ROIs in these areas. To have unbiased estimates of the locations of these ROIs, we used the first scanning session performed on each subject to localize the areas that were correlated with SV under ambiguity and then examined the activity of these areas for correlations with both risky and ambiguous SV in the second session (Fig. 5A). If one or more of these areas uniquely encodes the effects of ambiguity on SV, then this analysis should reveal a statistically significant difference between the ambiguous and risky trials. Alternatively, if an area encodes information about SV under both risky and ambiguous conditions and shows no significant difference between risky and ambiguous trials, then we cannot conclude we have identified an ambiguity-specific activation from the fact that a whole-brain regression yielded higher correlations under one condition than under another. This approach thus enables us to both cross-validate the correlation with SV under ambiguity and to ask whether the same areas that provide information about SV under ambiguity also provide information about SV under risk. Using only the first scan to define ROIs, we still obtained significant activations in each area of interest at the same thresholds used to visualize these areas with the entire dataset as described in the preceding text.
When we performed this analysis, we found that the correlation with SV under ambiguity in the second session was statistically significant in the MPFC, the striatum, the PCC, and the left amygdala (P < 0.05, 1-tailed paired t-test, n = 15) but not in the STS (P = 0.2). Importantly, activity in the MPFC, the striatum, and left amygdala was also correlated with SV under risk (P < 0.05). A trend in the same direction was found in the PCC but did not reach significance (P = 0.09). Finally, we found no significant differences between risky and ambiguous conditions in any of these areas (P = 0.2).
For completeness, we also examined ROIs defined by the SV of risky trials for a discrepancy between risky and ambiguous trials. In a similar manner, we thus used the first session to localize areas the activity of which was correlated with SV under risk and examined their activation as a function of risky and ambiguous SV in the second session. Using the first session to define ROIs, we obtained significant clusters of risky SV-related activation in the MPFC and the striatum. We found that activation in both of these areas was also significantly correlated with SV under both risk and ambiguity in the second session (P < 0.05), and we found no statistically significant difference between activations to risk and ambiguity in either of these areas (P = 0.2).
Thus all of the areas in which the BOLD signal provided reliable information about SV under risk also provided reliable information about SV under ambiguity, and all but one of the areas that provided reliable information about SV under ambiguity also provided information about SV under risk. However, although the strength of the correlation between activity in these areas and SV under risk was similar to the strength of the correlation under ambiguity, the significance of the activation maps was lower. This is actually not surprising: bear in mind that for most subjects, β was <1, such that the overall SV of the risky choice set was lower than that of the ambiguous choice set, which might have led to overall lower activation levels for risk. To verify that this is indeed the case, we performed a second experiment in which we concentrated on the risky condition and increased the range of possible probabilities (0.2, 0.4, 0.6, or 0.8) and amounts (11, 12, 15, 25, 40, 100, 170, or $250) we examined, such that the EVs (and thus also the SVs) of the lotteries were much higher than in the original experiment. On each trial, subjects chose between one combination of amount and probability and a sure bet of $10. The behavior of each subject was fit using Eq. 2, where SVs were estimated using Eq. 1 with A = 0 (no ambiguity). As in experiment 1, α varied substantially across subjects (mean, 0.6 ± 0.3; range, 0.17–1.10). Using the fit parameters, we again inferred the SV that each trial had for each subject and constructed a predictor to search for areas that were correlated with SV. As expected, the results were much more significant now: using the same threshold that was used for ambiguous trials in the original experiment, with a smaller number of subjects, we now obtained significant correlation in regions of the MPFC, the striatum, and the PCC (random effects analysis, n < 11, P < 0.001 uncorrected, P < 0.05 corrected for cluster size). Reducing the cluster size slightly revealed an additional, more ventral, cluster in MPFC (Fig. 6). Reducing the threshold slightly also revealed correlation in left amygdala (Fig. 6, P < 0.005, uncorrected).
The broader dynamic range of SVs in experiment 2 also reduced the correlation between SV and its principal components (amount and probability), thus allowing us to compare the contribution of each component to the observed activity. We therefore constructed two additional models for our fMRI analysis. In each model, the SV predictor was replaced by a predictor of either the objective payoff amount or the (objective) probability presented on that trial. Comparing the correlation maps obtained using each of these models shows that activity in all of the ROIs was more strongly correlated with SV than with either objective amount or probability (Supplementary Fig. S3), reaching significance in the MPFC and the PCC for SV versus probability and in bilateral striatum for SV versus amount (P < 0.05, 1-tailed paired t-test, n = 11). Furthermore, using a fourth model in which the SV predictor was replaced by an EV predictor (EV, the objective product of amount and probability), we found that despite of the very tight correlation between SV and EV, activity in both the right striatum and the left amygdala was better correlated with SV than with EV (Supplementary Fig. S4) although this difference did not reach significance. Thus SV describes the activity in all of the reported ROIs at least as well as each of its components or their product.
Additionally, we note that the similar neural representation of SV under risk and ambiguity by no means implies that there was no difference in the neural circuitry underlying these two types of decisions. Such difference is in fact inevitable given that subjects were aware of the context of each trial and exhibited different choice behavior under each condition. Indeed higher mean activation for choice under ambiguity compared with choice under risk was observed in bilateral lateral OFC (lateral OFC; left: −42, 53, 7; right: 39, 56, 11; P < 0.05, corrected for cluster size; Supplementary Fig. S5), a finding compatible with previous results (Hsu et al. 2005). This suggests that the lateral OFC might have a role in signaling the level of missing information. If this is the case, we should expect activation in this area to increase as a function of the ambiguity level. Inspecting the time course from these ROIs we did not find such an effect. However, when we searched directly for correlation with the ambiguity level, we did find an adjacent area in left OFC that exhibited higher activation levels for higher ambiguity levels as well as for lower winning probabilities (tal: −42, 49, 2; P < 0.05, corrected for cluster size, Fig. 7).
Finally, we also examined the connectivity patterns between the major structures identified in the experiment, namely the MPFC, the striatum, and the left amygdala. Using psychophysiological interaction analyses (PPI) (Friston et al. 1997) with each of these ROIs as a seed, we investigated whether the functional connectivity between these areas and the rest of the brain is modulated by the task (see methods). No such modulation was observed when either the MPFC or the striatum were used as a seed. However, using the left amygdala, we did observe such modulation in two areas (Supplementary Fig. S6): correlation with the left striatum was stronger under ambiguity, whereas correlation with a region in the vicinity of the junction between the IFG and the precentral sulcus was stronger under risk. These results suggest that although the amygdala participates in the representation of value under both risk and ambiguity, its exact role in the processing of each type of information may well be distinct as has been previously hypothesized (Hsu et al. 2005).
Choice behaviors under risk and under ambiguity are markedly different. Here we showed that despite this difference in behavior, SV under both risk and ambiguity is represented in a similar manner in several brain areas. Using each individual's behavior, we estimated the SV that each risky and ambiguous lottery had for that subject and searched for neural activity that was correlated with that measure. Significant correlations were found in the MPFC, the striatum, the PCC, and the amygdala. Importantly, no brain area provided unique information about SV under one condition only. Rather all the areas that conveyed information about SV under risk at the signal-to-noise ratios of a 3Tesla scanner also conveyed information about SV under ambiguity and vice versa, and no significant difference was found between correlation with SV under risk and SV under ambiguity in any of the areas.
Our results are compatible with previous studies that examined different aspects of the neural processing of risk and ambiguity. Hsu and colleagues reported higher BOLD activation for risky choices compared with ambiguous ones in the striatum (Hsu et al. 2005). Our finding of correlation to SV in the striatum is compatible with such differential activation because the ambiguous choice set in the Hsu and colleagues study most likely had a lower aggregate SV than their risky choice set as indicated by the choice behavior of their subjects. Those authors also reported higher activation for ambiguous choices compared with risky ones in the OFC, a finding that a similar analysis of our dataset replicates (Supplementary Fig. S5). Finally, the areas reported in the papers by Huettel and colleagues (2006) and Bach and colleagues (2009) were also found to be active in our experiment (Supplementary Fig. S7). In the previous studies, these areas were more active for ambiguity than for risk, and the authors therefore hypothesized that they might have a role in the process of resolving ambiguity. Compatible with this hypothesis, in our design, which did not allow subjects to resolve the ambiguous probabilities, the level of activation was similar for risk and for ambiguity.
Our study extends the previous studies of risk and ambiguity, however, in a novel way. Our results identify a stage in neural processing at which the value of all of those different risky and ambiguous options is represented in a unified manner. This stage is composed of the MPFC, the striatum, the PCC, and the amygdala, all of which have been previously implicated in the representation of value in both humans and monkeys.
In monkey, value-related neurons have been identified in the striatum in both the caudate (Lau and Glimcher 2008) and the putamen (Samejima et al. 2005). In humans, activity in the striatum has been shown to rise above baseline for unexpected rewards, drop below baseline for unexpected punishments (Delgado et al. 2000; Kuhnen and Knutson 2005), and scale with the magnitude of both (Delgado et al. 2003). The striatum is also active for reward and punishment predicting cues, scaling with the amount (Breiter et al. 2001; Knutson et al. 2001a, 2003), the probability (Hsu et al. 2009), and the EV of the predicted outcome (Hsu et al. 2005; Luhmann et al. 2008; Preuschoff et al. 2006; Tobler et al. 2007; Tom et al. 2007). The subjective nature of the value representation in the striatum was exemplified in recent studies that reported correlation with marginal utility (Pine et al. 2009) as well as reference dependence of the activation (De Martino et al. 2009) and changes in activation that corresponded to changes in post choice estimates of future hedonic experience (Sharot et al. 2009). The striatum also responds to the anticipation of primary rewards (O'Doherty et al. 2002) and its activity is correlated with behavioral preferences, such as juice preferences (O'Doherty et al. 2006), meal pleasantness ratings (Small et al. 2003), and product preferences (Knutson et al. 2007). Finally, the ventral striatum responds to both immediate (McClure et al. 2004a, 2007) and delayed rewards, tracking the subjectively discounted value of future rewards (Kable and Glimcher 2007).
Activation in MPFC exhibits similar properties. Like the striatum, it is active for both receipt of reward (Knutson et al. 2001b, 2003; Kuhnen and Knutson 2005) and expected reward, and its activity is correlated with the EV of the expected rewards (Knutson et al. 2005) and with the subjectively discounted value of future rewards (Kable and Glimcher 2007) as well as with the actual outcome level (Luk and Wallis 2009). Activity in the MPFC is also correlated with behavioral preferences (McClure et al. 2004b), incorporating various factors that affect each individual's valuation of different options (Hare et al. 2009). Recent studies reported the overlapping representations of action and stimulus values in the MPFC (Glascher et al. 2009) as well as an overlapping representation of the value of different types of goods (Chib et al. 2009). Finally, a similar picture emerges from data obtained from the adjacent OFC. Neurons in this area have been shown to encode the value of offered, chosen, and received goods (Padoa-Schioppa and Assad 2006, 2008; Tremblay and Schultz 1999; Wallis and Miller 2003), including goods of different classes (FitzGerald et al. 2009). Activity in this areas reflects subjects’ willingness to pay for offered food items (Plassmann et al. 2007) as well as their reported experienced pleasantness (Plassmann et al. 2008).
Both the striatum and the MPFC receive reward-related dopaminergic inputs from midbrain neurons that have been shown to encode the reward prediction error (RPE) or the difference between the received and expected reward (Bayer and Glimcher 2005; Schultz et al. 1997) compatible with a role for these areas in the representation of value (but see Hare et al. 2008).
The PCC is reciprocally connected to the MPFC, and both are parts of the brain's “default” system, which has been hypothesized to attend to internal body and mental states (Goldberg et al. 2006; Raichle et al. 2001). Like the MPFC and the striatum, the PCC responds both to receipt and to anticipation of reward and its activity is correlated with reward size (McCoy et al. 2003) including discounted value of future rewards (Kable and Glimcher 2007) as well as with social valuations (Schiller et al. 2009).
The amygdala is also reciprocally connected to the MPFC, and in addition it projects to the ventral striatum (Sah et al. 2003). The amygdala has been shown to represent both positive and negative values in both humans and animals. It responds positively to reward (Elliott et al. 2003; O'Doherty et al. 2003) and to anticipation of reward (Small et al. 2008), and it plays a role in representing the current value of a reward during learning of stimulus-reward associations (Baxter and Murray 2002; Paton et al. 2006) and in devaluation (Gottfried et al. 2003).
To conclude, we found that areas previously implicated in the representation of value also represent SV under risk and ambiguity. Despite the marked difference between the conditions of risk and ambiguity, at least at the fMRI resolution we employed there were no areas that provided unique information about SV in only one condition, suggesting a unified evaluative system that uses a common currency to represent value under different conditions.
This work was supported by grants from the McDonnell Foundation to P. W. Glimcher and the Seaver Foundation.
We thank J. Kable, E. DeWitt, and D. Schiller for fruitful discussions and comments, B. Lau for help with the behavioral fit, S. Shaw for help with the scanning, and K. Sanzenbach and the Center for Brain Imaging at NYU for technical support.
↵1 We use the term “subjective value” in a most general sense that can encompass both risky and ambiguous options. For risky choices, the subjective value is simply proportional to the expected utility of each choice, i.e., the product of probability and utility. For ambiguous choices, the subjective value is meant to capture an expected utility-like notion that could, in principle, also include the notion of subjective probability but does not depend on it. In other words, subjective value is the overall value of a certain option for an individual subject, taking into account any possible parameter of that option, including the parameters varied in this study, namely amount, probability and ambiguity.
↵2 The online version of this article contains supplemental data.
↵3 Because we did not have a good estimate of the hemodynamic response function (HRF) in each region of each subject, we chose not to deconvolve the HRF out of the BOLD signal sampled from the seed ROI. While such deconvolution can provide more correct results when the HRF is precisely known, it could seriously hamper the results when the HRF is misestimated (Gitelman et al. 2003). It should be noted, however, that for a slow experimental design, the results of the two methods, using the true HRF, should be very similar (Gitelman et al. 2003).
- Copyright © 2010 the American Physiological Society