Experience-Dependent Neural Integration of Taste and Smell in the Human Brain

Dana M. Small, Joel Voss, Y. Erica Mak, Katharine B. Simmons, Todd Parrish, Darren Gitelman


Flavor perception arises from the central integration of peripherally distinct sensory inputs (taste, smell, texture, temperature, sight, and even sound of foods). The results from psychophysical and neuroimaging studies in humans are converging with electrophysiological findings in animals and a picture of the neural correlates of flavor processing is beginning to emerge. Here we used event-related fMRI to evaluate brain response during perception of flavors (i.e., taste/odor liquid mixtures not differing in temperature or texture) compared with the sum of the independent presentation of their constituents (taste and/or odor). All stimuli were presented in liquid form so that olfactory stimulation was by the retronasal route. Mode of olfactory delivery is important because neural suppression has been observed in chemosensory regions during congruent taste–odor pairs when the odors are delivered by the orthonasal route and require subjects to sniff. There were 2 flavors. One contained a familiar/congruent taste–odor pair (vanilla/sweet) and the other an unfamiliar/incongruent taste–odor pair (vanilla/salty). Three unimodal stimuli, including 2 tastes (sweet and salty) and one odor (vanilla), as well as a tasteless/odorless liquid (baseline) were presented. Superadditive responses during the perception of the congruent flavor compared with the sum of its constituents were observed in the anterior cingulate cortex (ACC), dorsal insula, anterior ventral insula extending into the caudal orbitofrontal cortex (OFC), frontal operculum, ventral lateral prefrontal cortex, and posterior parietal cortex. These regions were not present in a similar analysis of the incongruent flavor compared with the sum of its constituents. All of these regions except the ventrolateral prefrontal cortex were also isolated in a direct contrast of congruent − incongruent. Additionally, the anterior cingulate, posterior parietal cortex, frontal operculum, and ventral insula/caudal OFC were also more active in vanilla + salty minus incongruent, suggesting that delivery of an unfamiliar taste–odor combination may lead to suppressed neural responses. Taken together with previous findings in the literature, these results suggest that the insula, OFC, and ACC are key components of the network underlying flavor perception and that taste–smell integration within these and other regions is dependent on 1) mode of olfactory delivery and 2) previous experience with taste/smell combinations.


Flavor perception arises from the central integration of peripherally distinct sensory inputs (taste, smell, texture, temperature, sight, and even sound of foods) (Bartoshuk and Beauchamp 1994). Neuroimaging studies of taste and smell have shown that independent presentation of a tastant or an odorant produces overlapping activation in regions of the insula (Cerf-Ducastel et al. 2001; Gottfried et al. 2002a; Poellinger et al. 2001; Savic et al. 2000; Small et al. 1999, 2003), which may correspond to the primary gustatory area, the amygdala (Anderson et al. 2003; Gottfried et al. 2002a; Small et al. 1997a,b, 2003; Zald and Pardo 1997; Zald et al. 1998) and the orbitofrontal cortex (OFC) (Francis et al. 1999; Gottfried et al. 2002a; O'Doherty et al. 2000; Poellinger et al. 2001; Savic et al. 2000; Small et al. 1997a, 1999, 2003; Sobel et al. 1998; Zald and Pardo 1997; Zald et al. 1998; Zatorre et al. 1992), thought to house the secondary taste and secondary smell cortical regions and the anterior cingulate cortex (de Araujo et al. 2003; O'Doherty et al. 2000; Royet et al. 2003; Savic et al. 2000; Small et al. 2001, 2003; Zald and Pardo 1997; Zald et al. 1998). In accordance with these findings in humans, single-cell recording studies in monkeys have identified both taste and smell-responsive cells in the insula/operculum (Scott and Plata-Salaman 1999) and OFC (Rolls and Baylis 1994; Rolls et al. 1996). Notably, whereas olfactory stimulation regularly activates the putative primary gustatory region, gustatory stimulation does not appear to activate the primary olfactory cortex, which is located in the piriform cortex in both primates (Price et al. 1991) and humans (Zatorre et al. 1992). This suggests differential input of gustatory and olfactory systems in flavor perception.

Rolls and Baylis (1994) recorded cells in the caudal two thirds of the monkey OFC that responded to visual, olfactory, and gustatory stimulation. They identified unimodal taste, smell, and visual cells that were concentrated in 3 different areas. However, considerable overlap occurred and bimodal cells were interspersed throughout the 3 areas. One striking feature of all bimodal cells tested was their selective responses for congruent stimuli. For example, a cell responding optimally to the glucose taste solution also responded optimally to fruit-related odors rather than to salmon or other odors that were less likely to be previously experienced with a sweet-tasting solution. Similarly, a bimodal taste/visual cell responding to glucose also responded to the sight of sweet foods such as a banana or a syringe that had been used to deliver blackcurrant juice, but not to the sight of pliers, which had never been associated with a sweet taste.

The neurophysiology of taste/odor integration is paralleled by perceptual experience. Odors can enhance the perceived intensity of a taste but only if the odor is perceptually similarto the taste (Frank et al. 1989; Schifferstein and Verlegh 1996). Similarly, Dalton et al. (2000) found that the subthreshold summation occurred between simultaneously presented tastes and odors only if they were congruent taste/odor pairings. Finally, odors can acquire “tastelike” properties after repeated pairings with a taste [i.e., strawberry odor is often described as sweet even though the odor does not activate taste receptors (Prescott 1999; Stevenson and Prescott 1995; Stevenson et al. 1998, 1999)]. Together these observations underscore the role of experience in forming the neural representation of flavor.

There are only a handful of neuroimaging studies examining the sensory components of flavor perception (Cerf-Ducastel and Murphy 2001; Cerf-Ducastel et al. 2001; de Araujo et al. 2003; Small et al. 1997a). In 1997 we reported massive deactivation in cortical chemosensory regions when tastes were perceived simultaneously with orthonasally presented odors (Small et al. 1997a). However, when odors are naturally experienced in combination with taste it is because food is in the mouth. In this case the odor is sensed retronasally as volatiles released to the nasopharynx and not by the orthonasal route. Rozin (1982) suggested that olfaction could be divided into 2 functionally distinct senses: one sense for identifying objects at a distance (ortho) and another that contributes to flavor and food identification in the mouth (retro). Several authors have argued that retro- and orthonasal olfaction differ only in the efficiency by which odors are delivered to the olfactory epithelium (Pierce and Halpern 1996; Voirol and Dagnet 1986). Consistent with this notion, Cerf-Ducastel and Murphy performed an functional magnetic resonance imaging (fMRI) study examining brain response to retronasally perceived odors delivered as liquid to the mouth and reported activation in many of the areas identified as important in orthonasal olfaction (Gitelman et al. 1999; Gottfried et al. 2002a; Royet et al. 2003; Savic et al. 2000; Sobel et al. 1998; Zatorre et al. 1992). However, these authors did not include orthonasal stimulation in the paradigm and so a direct comparison between brain regions activated by orthonasal and retronasal olfaction was not possible. In contrast, Heilmann and Hummel (2001) endoscopically inserted nasal cannula into their subject's noses, enabling vaporized odor to be delivered to the external nares (orthonasal) and retropharynx (retronasal). Using this sophisticated method of odor delivery they were able to show distinct olfactory-evoked related potentials in healthy human subjects to vaporized odor delivered by the retronasal versus the orthonasal route. More recently, in a study we performed in collaboration with Hummel and colleagues and found preliminary evidence for differential engagement of chemosensory regions by ortho- compared with retronasal olfactory stimulation (Gerber et al. 2003). Notably, the comparison of retronasal odor versus odorless baseline and orthonasal odor versus odorless baseline each yielded similar activations to those reported by Cerf-Ducastel and Murphy. This convergence indicates that, although the neural correlates of these forms of olfactory processing may be divisible, they clearly share many common neural substrates.

There is also evidence that orthonasal and retronasal olfaction may interact differently with gustatory processing. Slotnick and colleagues (1997) showed that odors can potentiate a taste aversion but only when they are presented retronasally. This indicates that retronasal odors may have a greater ability to influence the gustatory or flavor neural code. In addition, using fMRI, de Araujo and colleagues (2003) measured brain response to retronasally delivered odors in combination with a taste. In contrast to our 1997 study in which odors were delivered orthonasally in combination with a taste, they did not observe brain deactivation during simultaneous taste/smell stimulation. Instead, they reported activation in an anterior region of orbitofrontal cortex (OFC) that was not activated by taste or smell alone (thus providing evidence for a positive interaction), and suggested that the region was therefore important in integration of taste and smell.

Taken together, the results presented above suggest that taste/smell integration is strongly dependent on the mode of olfactory delivery (orthonasal vs. retronasal) and previous experience with of taste/odor pairings. To date, no study has examined the role of experience on taste–odor integration. The goal of the present study was therefore to use fMRI to evaluate brain response to a retronasally presented odor (vanilla) in combination with a congruent taste (sweet) versus an incongruent taste (salty). We were particularly interested in determining whether superadditive responses could be identified that were specific to the congruent (and not incongruent) taste/smell stimulus. A superadditive response occurs when the neural activity evoked in response to bimodal stimulation is greater than the summed activity evoked by independent presentation of its unimodal components. Such responses have been observed during cross-modal perceptions in other sensory modalities and have come to represent a hallmark of multisensory integration (e.g., Calvert et al. 2001; Stein 1998). Given the likely role of experience in taste/smell integration and the studies showing overlapping representation of taste and smell in the insula, OFC, and anterior cingulate cortex, we predicted that superadditive responses in these regions would result from perception of the vanilla/sweet mixture but not the vanilla/salty mixture. We also predicted that taste/smell integration would recruit heteromodal neocortical regions, such as the posterior parietal cortex and superior temporal sulcus, which are known to be involved in multisensory integration between other modalities (Bushara et al. 2003; Calvert 2001; Calvert et al. 2001; Gottfried and Dolan 2003; Macaluso et al. 2000), and thus reflect some degree of shared neural circuitry for all forms sensory integration.



The Northwestern University Institutional Review Board approved the study and 20 subjects gave informed consent. Subjects were excluded if they made excessive movements (5 subjects) or if their ratings did not fall in the target range (i.e., they rated the familiar mixture as unfamiliar) (4 subjects). The remaining 11 subjects were all right-handed [average handedness score of 84 according to the modified Edinburgh inventory (Oldfield 1971)]; 3 were men and 8 women, with a mean age of 26 and mean education of 17 yr. Taster status was evaluated by having subjects rate the intensity of 6-n-propylthiouracil (PROP) with the general labeled magnitude scale (Green et al. 1996). All subjects were average tasters with a mean PROP rating of 38 (Prutkin et al. 2000).


Twenty subjects underwent echo planar imaging on a 3-T Seimens Trio magnet while tasting 6 different solutions made from stimuli provided by Unilever Research and distilled water (see Fig. 1). These included 2 odorless taste solutions (1.8 × 10−1 M sucrose = sweet; 1.8 × 10−1 M NaCl = salty), one tasteless odor solution (vanillin 10 drops in 10 ml of distilled water), 2 mixtures (sucrose + vanillin = congruent and NaCl + vanillin = incongruent), and one tasteless/odorless solution [with similar ionic components to saliva (O'Doherty et al. 2001a)]. All subjects had sampled these solutions in a training session and provided ratings of intensity, familiarity, and pleasantness. Only subjects who identified the congruent solution as familiar and the incongruent solution as unfamiliar were asked to participate in the neuroimaging study. All of the solutions were rated in the mild to moderate range on intensity. We chose not to use strong stimuli because previous studies of multisensory integration show more profound integrative effects with weaker stimuli (Stein 1998). To ensure that the vanilla solution had no taste, subjects were asked to attempt to distinguish the vanilla solution from water while holding their nose. When a subject could detect a difference (>3 correct responses on 5 forced choice tests) the concentration was reduced. If the solution no longer had a detectable retronasal odor (a rating of ≥3) we did not ask that subject to participate in the fMRI portion of the experiment. Therefore all subjects scanned could not differentiate the vanilla solution from water when retronasal olfaction was blocked.

FIG. 1.

Paradigm. A long-event related design was used so that we could dissociate the event of interest (tasting) from movement related to swallowing. Five different solutions were delivered (sucrose, saline, vanillin, sucrose/vanillin, and saline vanillin) plus one tasteless stimulus [with similar ionic components to saliva (O'Doherty et al. 2001b)]. Events began with delivery of one of these 5 solutions as a 0.5-ml bolus over a 5-s period (marked as “stimulus” in the diagram). Subjects were then instructed to hold the liquid in their mouth until a 400-Hz tone played for 5 s. Subjects were informed that they should swallow only when they heard this tone (thus they had a 5-s window to swallow; marked “tone” in the diagram). Tone was followed by a 5-s rinse, which was in turn followed by a second tone, signaling that the subject should swallow again. Dotted light gray line indicates the predicted hemodynamic response function (hrf). By signaling the subject to swallow 15 s after taste delivery, the peak of the hrf should be free from contamination by movement related to swallow. First 15 s is modeled as the event of interest and the second 15 s as an event of no interest. Each stimulus was presented twice per run and a total of 7 runs were performed. Runs lasted for 6 min and 13 s (12.6 s to equilibrate).

A long-event–related design was used to enable a dissociation of the event of interest (tasting) from movement related to swallowing (Fig. 1). This procedure was previously successfully used (Small et al. 2003) for this purpose. Stimuli were all delivered as 0.5 ml of solution over 5 s (Fig. 1) by beverage tubing attached to a series of programmable syringe pumps (Braintree Scientific BS- 8000) controlled using MATLAB. Intensity, familiarity, and pleasantness ratings were collected after the first, third, and seventh scans and are presented in Fig. 3. We chose to have subjects make ratings between rather than during scans to avoid confounds related to decision making, which is known to involve OFC (Elliott et al. 2000). This method has been successful in the past (Small et al. 2003), although others have obtained reliable results while subjects perform tasks during scanning (Cerf-Ducastel and Murphy 2001; Cerf-Ducastel et al. 2001).

A susceptibility-weighted single-shot echo planar sequence was used to image the regional distribution of the blood oxygenation level–dependent (BOLD) signal with TR = 2100 ms, TE = 20 ms, flip angle = 80°, FOV = 220, matrix = 64 × 64, slice thickness = 3 mm, and acquisition of 40 contiguous slices. A TE = 20 ms was chosen to reduce susceptibility and distortion effects while maintaining sensitivity to BOLD signal changes. Further reduction in signal loss in the OFC region was obtained by using a localized shimming procedure, whereby a volume of interest was selected to include the OFC (see Fig. 2). The Siemens standard shimming software optimized the field homogeneity in this region, which resulted in an increase in the T2* in the OFC. Slices were acquired in an interleaved mode to reduce the cross talk of the slice selection pulse.

FIG. 2.

Bold sensitivity maps (Parrish et al. 2000) in the amygdala (left at cross hatch) and orbitofrontal cortex (OFC; right at cross hatch). Top row: data from a single subject who participated in our first functional MRI (fMRI) study of chemosensation (Small et al. 2003) on a 1.5-T Seimens Sonata scanner. In that study, we were able to successfully isolate activity in the amygdala and OFC in group random effects analyses. Bottom row: blood oxygenation level–dependent (BOLD) detectability map created from the data of a single subject performing the same task as in our previous study but on the 3-T Seimens Trio scanner using the imaging protocol used in the current study. Purple: ability to detect ≥0.5% signal change; blue: ≥1%; green: ≥2%; yellow: ≥4 signal change, given the number of trials collected, and α and β = 0.05. These data show that our ability to detect the BOLD signal is better at 3 T (used in the current study) with local shimming, at which we are able to detect >0.5% signal change throughout almost all of the OFC and amygdala.

At the beginning of each functional run, the MR signal was allowed to equilibrate over 6 scans for a total of 12.6 s, which were then excluded from analysis. For each subject, a high-resolution, T1-weighted 3D volume was acquired in <8 min (MP-RAGE with a TR/TE of 2100 /2.4 ms, flip angle of 15°, TI of 1,100 ms, matrix size of 256 × 256, FOV of 22 cm, slice thickness of 1 mm). Using SPM2 (Welcome Department of Cognitive Neurology, London, UK), the functional images were time-acquisition corrected to the slice obtained at 50% of the TR. All functional images were then realigned to the scan immediately before the anatomical T1 image. The images (anatomical and functional) were then normalized to the Montreal Neurological Institute template (MNI-305), which approximates the anatomical space delineated by Talairach and Tournoux (1988). Functional images were smoothed with a 7-mm FWHM isotropic Gaussian kernel. For the time series analysis on all subjects, a high-pass filter was included in the filtering matrix (according to convention in SPM2) to remove low-frequency noise and slow drifts in the signal, which could bias the estimates of the error. A global covariate was also included in the design matrix. When making inferences about signal decreases, we examined whether the global estimator was collinear with any columns of the design matrix. Condition-specific effects at each voxel were estimated using the general linear model. The response to events was then modeled by a canonical hemodynamic response function, consisting of a mixture of 2 gamma functions that emulate the early peak at 5 s and the subsequent undershoot. The temporal derivative of the hemodynamic function was also included as part of the basis set to provide a better model of the data (Henson et al. 2002).

Postprocessing of the neuroimaging data from data from the 11 subjects was performed with SPM2 (Wellcome Department of Neurology, London) using group random effects analyses. We applied Gaussian random field theory as implemented in SPM2 (Friston et al. 1994; Worsley and Friston 1995). A main analysis of [congruent − (vanilla + sweet)], using a t-map threshold of P = 0.001 (uncorrected) was performed to identify the flavor network and heteromodal regions involved in generalized cross-modal integration (i.e., not specific to flavor). Because activations in these regions are predicted, we used an uncorrected P-value of <0.001 to determine significance. The regions identified in the main analysis were then used to perform directed searches in the secondary analyses using 10-mm spheres surrounding the peaks identified in the main analysis. This procedure allowed us to determine the specificity of the response identified in the main analysis so that we could determine whether these regions were activated to a greater extent by the familiar/congruent versus the unfamiliar/incongruent taste–odor combination and whether there was evidence of suppression at these loci during perception of the unfamiliar/incongruent combination. Peaks were considered significant at P < 0.05 (with respect to voxels) corrected for multiple comparisons (Worsley et al. 1996). In this case, significance means that the effect was observed at the same location as identified in the original analysis. Because subsequent analyses focus only on regions identified in this initial analysis we used a reduced t-map threshold of P = 0.005 uncorrected k >5 (i.e., the cluster threshold was set to show peaks with a minimum of 5 activated voxels). The secondary analyses were thus performed to evaluate the role of experience on the network identified in the initial analysis. Secondary analyses included 1) the main analysis masked exclusively with [incongruent − (vanilla + salty)] to identify regions that responded in a superadditive fashion only if the bimodal mixture was familiar; 2) a direct comparison of the familiar and novel mixtures [congruent − incongruent]; 3) identification of regions that show a depressed response to the novel mixture compared to its summed unimodal constituents [(vanilla + salty) − incongruent]. A similar procedure was followed for [incongruent − (vanilla + salty)]. Because 2 of the follow-up analyses are not independent from the main analysis {[congruent − incongruent] and [Congruent − (vanilla + sweet) masked by incongruent − (vanilla plus salty)]}, the significance of these peaks was determined by using a threshold of P < 0.001 uncorrected.


Behavioral data


Mean intensity, familiarity, and pleasantness ratings are presented in Fig. 3. A repeated-measures ANOVA showed that there was no effect of time [F(2,14) = 0.63; P = 0.55] or rating × time interaction [F(4,28) = 0.73; P = 0.58], showing that subjects did not habituate to the stimuli over the course of the experiment. There was a stimulus × rating interaction [F(8,56) = 12.8; P < 0.001]. As predicted, the incongruent stimulus was rated as more unfamiliar than all other stimuli (vanilla, P = 0.05; sweet, P = 0.004; salty, P = 0.03; congruent, P = 0.002). The congruent stimulus was more familiar than the incongruent (P = 0.002) and the vanilla (P = 0.04). The vanilla was also less familiar than the sweet and vanilla solutions (P = 0.04 for both), and less intense that all other stimuli (sweet, P = 0.05; salty, P = 0.002; congruent, P = 0.006; incongruent, P = 0.001). There was no difference in intensity ratings between the congruent and incongruent stimulus (P = 0.1); however, the congruent was rated as more intense than the salty (P = 0.03) and vanilla (P = 0.002) and the incongruent as more intense than the vanilla (P = 0.001). The incongruent was also rated as more unpleasant than vanilla (P = 0.007), sweet (P = 0.00001), and congruent (P = 0.001), whereas the congruent was more pleasant that the salty (P = 0.005) taste and the vanilla solution (P = 0.04). Salty was also less pleasant than sweet (P = 0.001) but the intensity ratings between salty and sweet were not significantly different (P = 0.35). Although differences in pleasantness exist between the congruent and incongruent mixtures it is unlikely that the differential activations are attributable to this difference because the salty is rated as unpleasant as the incongruent solution, yet greater activation is observed to salty plus vanilla than incongruent (see following text).

FIG. 3.

Average ratings of familiarity, intensity, and pleasantness. Error bars represent SE. Familiarity and pleasantness ratings were made on a 21-point scale (10 = very familiar/delicious; 5 = familiar/pleasant; 0 = neutral; −5 = unfamiliar/unpleasant; −10 = very unfamiliar/awful) and intensity a 10-point scale (10 = extremely intense; 0 = no taste). Ratings were collected before the first scan, after the 3rd scan, and after the 7th scan.

Neuroimaging data


First we sought to identify the neural network for flavor processing in the human brain. To do this we searched for brain regions with a superadditive response to the congruent mixture compared with the sum of its parts [congruent − (vanilla + sweet)]. This contrast revealed activation in the anterior dorsal and anterior ventral insula extending into the orbitofrontal cortex, frontal operculum, anterior cingulate cortex (ACC, 2 peaks), ventrolateral prefrontal cortex, and posterior parietal cortex (2 peaks) (Fig. 4 and Table 1).

FIG. 4.

Flavor network: activations isolated in the group random effects analysis of [congruent − (taste + smell)]. A: activation in dorsal anterior insula (dINS) and ventral anterior insula (vINS)/caudal OFC. Graphs show parameter estimates associated with each of the 5 conditions (C = congruent, I = incongruent, S = sweet, N = salty, V = vanilla) in the dorsal insula (top) and ventral insula (bottom). B: activation in the anterior cingulate cortex (ACC) and the graph of parameter estimates from that region in the 5 conditions. C: activation in the frontal operculum and ventrolateral prefrontal cortex with the graph of parameter estimates in the frontal operculum for the 5 conditions. D: activation in the posterior parietal cortex (PPC) (intraparietal sulcus IPS) and parameter estimates at that location for each of the 5 conditions.

View this table:

Flavor network


To evaluate the role of experience on taste–smell integration we first applied an exclusive mask to the original analysis {[congruent − (vanilla + sweet)] but not [incongruent − (vanilla + salty)]} to ensure that the regions identified in the original analysis show a superadditive response only when the flavor is composed of a familiar taste–smell combination. The flavor network was probed by searching for activity within a 10-mm sphere around each of the 8 original peaks. All of the regions in the original analysis showed a selective superadditive effect, except for the more anterior peak in the posterior parietal cortex (Table 1, column 6).

Next, we identified regions of the flavor network that responded significantly greater to congruent compared with the incongruent mixture by performing a direct contrast [congruent − incongruent] using small volume corrections defined as 10-mm spheres around the 8 original peaks. This produced activation in 5 of the original 8 regions including the anterior dorsal insula, anterior ventral insula/OFC, frontal operculum, anterior cingulate cortex, and posterior parietal cortex (posterior peak) (Table 1, column 7). This procedure allowed us to determine whether there were differences between congruent and incongruent at the loci identified in the main analysis. However, we note that [congruent − incongruent] and {[congruent − (vanilla + sweet)] but not [incongruent − (vanilla + salty)]} are not independent observations from the main analysis. Therefore we also checked the uncorrected P-value associated with these peaks. With the exception of the anterior cingulate cortex in (congruent − incongruent), all peaks were significant at P < 0.001 uncorrected.

Finally, to verify that brain activity was not being driven by greater activity to the pleasant sucrose versus the unpleasant salty solution in our regions of interest, we formally contrasted these 2 conditions. We first used a t-map threshold of P < 0.005 and a cluster threshold of k >5 voxels and then displayed the results again using a t-map threshold of P < 0.01 and a cluster threshold of k >3 to verify the result. The only region of activity that has previously been associated with chemosensory stimulation was in the temporal operculum at z = 3.5 (−54, 3, −15) (Small et al. 2003). This peak may reflect quality-specific coding or differences in intensity or pleasantness between the 2 solutions. It is somewhat surprising that this contrast did not produce differential activity within the OFC, given that we previously observed valence-specific responses in this region (Small et al. 2003). One possible explanation is that the affective ratings were not as polarized in the current study as they were in our previous report. This result suggests that the superadditive effects reported above are unlikely attributable to differences in the intensity or pleasantness of the 2 tastants used.


Calvert and colleagues (2001) suggested that simple linear summation may be an insufficient criterion to demonstrate multisensory integration with fMRI because the BOLD signal represents the averaged response from a large population of cells within the same area. However, they propose that experience-dependent integration can be concluded if a depressed response to an incongruent combination can be demonstrated in the same region that showed a superadditive response to a congruent combination. In this situation, one can assume that the same population of cells responds differentially depending on previous experience with particular combinations. To determine whether any of the regions of the flavor network can satisfy this stringent criterion, we performed the analysis [(vanilla + salty) − incongruent], again using small volume corrections to evaluate activity in 10-mm spheres around the 8 original peaks. A depressed response to the incongruent mixture relative to the sum of its constituents was observed in the anterior cingulate cortex, frontal operculum and posterior parietal cortex. Because a single-cell recording study in primates had identified multimodal cells within the anterior portion of the insula/caudal OFC that respond to congruent but not incongruent tastes and smells (Rolls and Baylis 1994) we lowered our cluster threshold to k >3 and observed a weak depression within this region.


Although the incongruent mixture was included mainly to show that superadditive responses were specific to previously experienced taste/smell combinations, we also acknowledge that it was possible that there would be brain activation related specifically to the perception of this novel taste–smell combination. To investigate this possibility we performed 3 analyses. First, we identified regions with a greater response to the incongruent mixture compared with the sum of its constituents [incongruent − (vanilla + salty)]. The behavioral ratings show that the vanilla was less intense and more pleasant than the incongruent mixture, whereas no differences in pleasantness or intensity were observed between salty and incongruent. In contrast, both the vanilla and the salty were rated as significantly more familiar than the incongruent solution. Thus the comparison attempts to isolate brain activity related to the integration of 2 familiar sensory inputs into an unfamiliar whole but does not allow us to rule out differential activity related to differences in intensity or pleasantness of the vanilla odor. Activation was observed at the junction of the temporal and parietal lobules, ventral striatum, and the anterior ventral insula (Table 2 and Fig. 6). Notably, the anterior ventral insula peak did not overlap at all with the anterior ventral insula peak identified in the congruency analyses (as would be expected, given the use of the incongruent exclusive mask in the congruent analysis) (Table 1). These 2 insular peaks are color-coded and presented side by side in Fig. 6B. We also performed a direct comparison between incongruent and congruent mixtures. The activation at the temporal parietal junction was significantly greater in response to the incongruent solution. A small peak was also observed in the ventral insula, but only when the cluster threshold was dropped to k >3. Differential activation was not found in the ventral striatum.

View this table:

Integration of familiar parts into a novel flavor


Evidence for a role of mode of olfactory delivery in taste–odor integration

Previous studies have shown that independent presentation of a taste or an orthonasally presented odor results in overlapping activation of the insula, OFC, and anterior cingulate cortex (de Araujo et al. 2003; Faurion et al. 1998; Frey and Petrides 1999; Gottfried and Dolan 2003; Gottfried et al. 2002a,b; Kinomura et al. 1994; O'Doherty et al. 2001a; Poellinger et al. 2001; Rolls et al. 2003; Royet et al. 2003; Savic and Gulyas 2000; Savic et al. 2000; Small et al. 1999, 2003; Zald and Pardo 1997; Zald et al. 1998, 2002; Zatorre et al. 1992). We also know that retronasal olfactory stimulation activates in these same regions (Cerf-Ducastel and Murphy 2001; de Araujo et al. 2003). Therefore it is clear that unimodal experience of all 3 classes of chemosensory perception (taste, orthonasal olfaction, retronasal olfaction) result in activation of overlapping regions of the brain in humans. However, it appears that the 2 classes of olfactory stimulation interact differently with taste. When a taste is perceived simultaneously with an orthonasally presented odor, significant deactivation occurs in the OFC, insula, and ACC (Small et al. 1997a). In contrast, the current results show that when a taste is perceived simultaneously with a retronasally presented odor, a superadditive response in these same regions ensues instead of deactivation (Fig. 4 and Table 1).

Studies of visual and auditory cross-modal integration show that cross-modal integrative processes are dependent on the nature of the correspondence between the different sensory inputs (for excellent reviews, see Calvert 2001; Stein 1998). Concordant stimulation in time and space may lead to multisensory integration, in which the response to multimodal stimulation is greater than the summed response of independent stimulation with the components (Meredith and Stein 1996; Meredith et al. 1987). If temporal and spatial factors are important in multisensory integration then it would follow that odors perceived orthonasally (by the nose) in combination with taste (experienced by the mouth) should lead to neural competition, whereas odors perceived retronasally (by the mouth) in combination with taste should lead to integration. Although this suggestion is in line with the current results, those from de Araujo et al. (2003) and our previous results showing suppression during simultaneous perception of a taste with an orthonasally presented odor (Small et al. 1997a), validity of the claim clearly relies on confirmation from a study in which both modes of taste–odor integration are compared directly.

In accordance with this suggestion and the current results, de Araujo and colleagues (2003) reported a superadditive response to simultaneous perception of a congruent taste and retronasally presented odor. However, in contrast with our study, in which superadditive responses were observed in the ACC and anterior ventral insula/caudal OFC, the superadditive response observed in their study occurred in the far anterior OFC (although they did note that the insula, caudal OFC, and ACC coactivated to unimodal taste and olfactory stimulation). In addition, we also found superadditive responses in a number of heteromodal neocortical regions including the intraparietal sulcus, frontal operculum, and ventrolateral prefrontal cortex. It is not immediately clear what might account for the discrepant location of superadditive responses. However, there are a number of differences between the paradigms that might offer a clue.

First, de Araujo et al. (2003) collected affective ratings during scanning whereas we did not. Many studies have shown that the affective value of chemosensory stimuli appears to be represented in the more caudal regions of the OFC, ACC, and insula (Anderson et al. 2003; Gottfried et al. 2002a; O'Doherty et al. 2001b; Rolls, 2003; Royet et al. 2003; Small et al. 2001, 2003; Zald and Pardo, 1997; Zald et al., 1998, 2002). In addition, there is now a growing body of literature highlighting the importance of paralimbic regions, especially caudal OFC, in affective decision making (Elliott et al. 2000, 2003). For example, O'Doherty and colleagues (2003) found that the anterior ventral insular region extending into the caudal OFC is significantly activated during passive tracking of reward value as well as during reward decision making. It is possible that deciding on an affective value for a stimulus and then choosing the appropriate response may activate these circuits, thus obscuring superadditive responses. This possibility is more likely, given that in the analysis (sweet/strawberry − sweet + strawberry) the amount of decision making is not balanced, with 2 sets of judgments being made for the unimodal stimuli (i.e., one for taste and one for smell) and only one judgment being made for the bimodal mixture.

A second possibility is that the stimuli may not have been equivalently intense. Although very different scales were used, it appears that our stimuli may have resulted in weaker perceptions. In the current study, the sucrose, vanilla, and congruent solution were all rated as weak to moderately intense, whereas in the study by de Araujo and colleagues (2003) the sweet and the sweet/strawberry mixture were rated as intense and the strawberry odor presented alone as moderately intense. Because Meredith and Stein (1986) showed that enhanced neural responses to audiovisual stimuli are inversely related to stimulus intensity (a phenomenon they termed “inverse effectiveness”), it is conceivable that stimulus intensity may also be an important factor in taste–smell integration. Thus in accordance with the discrepant results, weak taste–smell combinations may lead to greater neural enhancement than stronger taste–smell combinations. However, one caveat remains with this interpretation; we did not observe group superadditive response in the anterior OFC. We did see responses in a slightly more caudal region in some subjects (y =36–44) but the region did not survive group analysis. This same region has been shown to be important in olfactory–visual integration (Gottfried and Dolan 2003) (y = 39). It is therefore likely that this region is important in chemosensory integration, including taste–odor integration. Future studies will be important in determining the validity of our speculation.

A role for experience in taste–odor integration

The results from the current study also provide the first evidence that superadditive responses to bimodal taste and retronasal olfaction may be critically dependent on prior experience with a given taste–odor pair. Superadditive responses were highly dependent on the congruence of the taste–smell pair. Specifically, superadditive responses in the ACC, dorsal insula, ventral insula, frontal operculum, and intraparietal sulcus occurred during congruent (vanilla/sweet − its constituents (vanilla + sweet) masked exclusively by incongruent (vanilla/salty) − its constituents (vanilla +salty), and in congruent − incongruent. Moreover, many of these regions were also activated more in vanilla + salty − incongruent (vanilla/salty), suggesting that the incongruent mixture may actually have resulted in deactivation. This is particularly interesting because orthonasal odor/taste combinations also lead to deactivations, suggesting that neural suppression may be a common feature of chemosensory neural competition. Finally, a complex interaction was observed in the anterior ventral insula/caudal OFC, which showed the above pattern in one area and the opposite response in an immediately adjacent region: greater activity in incongruent − its constituents and incongruent − congruent (Fig. 6).

Taken together the data suggest that previous experience with a taste–odor pair influences whether neural integration or competition occurs in specific brain areas. This does not imply that these regions are also responsible for the perception of familiarity or novelty. In fact, the data presented in Figs. 4A and 5 show that the response in the ventral insula/OFC to the saline is greater than the response to the sweet solution, yet the saline is more unfamiliar than sweet (Fig. 3). In addition the congruent solution produces significantly greater activation than the sweet solution yet there are no differences in familiarity, intensity, or pleasantness ratings of these stimuli. In contrast, perception of the vanilla solution alone is judged as less familiar and less pleasant and less intense than when it is sampled with the sweet taste (i.e., congruent) and also results in less activation in the ventral insula. Thus the data suggest that the superadditive response is at least partially independent of these perceptual dimensions. If this is true, then the response within this and other reported regions likely reflects an interaction between multimodal stimulation and previous experience of a taste–odor pair such that these superadditive responses are dependent on taste–odor learning but not on the perceptual evaluation of the stimuli.

FIG. 5.

Group data showing experience-dependent superadditive responses in (top) the ACC and (bottom) ventral insula/OFC. y-axis = fitted response in arbitrary units (percentage change for each condition) and x-axis = peristimulus time in seconds. Response at each location is plotted for each event with: congruent = turquoise; incongruent = yellow; vanilla = red; sweet = blue; and salty = green. To create these graphs the ACC and ventral insula/OFC peaks from the random effects analysis of [congruent − (vanilla + sweet)] were used as starting locations to search for peaks in the individual data sets for this same contrast. Only peaks in the indvidual subject t-maps that were within 20 mm of the group random effects peaks were included. Significant activations in each of these areas were obtained for 9 of the 11 subjects. Plots were then made of the fitted event-related response for each event and each session. These data were then extracted and averaged across sessions and subjects. Because the data used to make the graphs differ from the data used to generate the random effects analyses (9 vs. 11 subjects), these time-series plots differ slightly from the plots of the parameter estimates presented in Fig. 4. In both areas the congruent stimulus clearly produces a response greater than the sum of its parts (sweet + vanilla), whereas the incongruent stimulus does not produce a response greater than the sum of its parts (salty + vanilla). In both areas the congruent stimulus clearly produces a response greater than the sum of its parts (sweet + vanilla), whereas the incongruent stimulus does not produce a response greater than the sum of its parts (salty + vanilla). This pattern represents experience-dependent taste–odor integration. Responses in the ventral insula/OFC cannot be accounted for by familiarity alone because salty produces more of a response than the sweet taste in the ventral insula (z = 3.6 at 36, 21, −12), yet it is rated as less familiar (Fig. 3). Neither can this activity be related only to affective valence or intensity, given that the congruent and sweet solutions are rated as similarly pleasant and intense (Fig. 3) yet there is significantly more activation here in congruent > sweet with z = 3.7 at 36, 15, −12. Congruent also produced more activation in the anterior insula than the vanilla (z = 4.2 at 33, 30, −6) and ACC [z = 2.7 (ns) at 15, 33, 27]. However, in this case the vanilla is less pleasant and less familiar than the congruent. The top graph suggests a similar relationship with the ACC; however, the direct comparison of congruent > sweet revealed small nonsignificant differences (congruent > sweet: z = 2.8 at 24, 27, 39; salty > sweet: z = 2.9 at 18, 30, 29). Note that the y-axis represents arbitrary units so that negative values do not necessarily reflect deactivations. Rather, negative deviations indicate responses less than the grand mean of all responses observed in the data set.

That taste–odor integration depends on experience is consistent with electrophysiological studies in monkeys and psychophysical evidence collected in humans. Rolls and Baylis (1994) identified bimodal and multimodal taste, smell, and visual cells in a region extending from the anterior insula into the caudal orbitofrontal region. One striking feature of all bimodal cells tested was their selective responses for congruent stimuli. For example, a cell responding optimally to the glucose taste solution also responded optimally to fruit-related odors rather than to salmon or other odors that were less likely to be previously experienced with a sweet-tasting solution. The neurophysiology of taste–odor integration is paralleled by perceptual experience. Odors can enhance the perceived intensity of a taste but only if the odor is perceptually congruent with the taste (Frank et al. 1989; Schifferstein and Verlegh 1996). Similarly, Dalton et al. (2000) found that the subthreshold summation of a taste and an odor occurs only if the stimuli are congruent. These observations underscore the role of experience in forming the neural representation of flavor. Other evidence to suggest that multisensory integration in the chemosensory system may be sensitive to experience comes from a recent fMRI study showing enhanced convergence of congruent compared with incongruent visual/olfactory stimuli (Gottfried and Dolan 2003) in an anterior region of orbitofrontal cortex, and in an analysis from the de Araujo study showing that activation in a similar region correlating with subjects' ratings of taste–odor congruency (de Araujo et al. 2003).

Although previously experienced taste–smell combinations come to be experienced as flavors and are associated with food, organisms also encounter taste–smell combinations that are novel and may either represent a potential source of nutrients or a noxious substance that should be avoided. Studies in animals suggest that the basal forebrain and ventral striatum play an important role in evaluating novel foods (Williams et al. 1993; Wilson and Rolls 1990). In the current study subjects sampled a taste–smell combination that was rated as significantly more unfamiliar than either of its constituents experienced by themselves (Fig. 3). In accordance with the animal data, comparison of the activity associated with this incongruent condition compared with the sum of the activity of evoked by its constituents resulted in brain activity in the ventral striatum. However, activity was also observed in the anterior ventral insula/caudal OFC and temporal–parietal junction, suggesting that neural events within these regions also contribute to the integration of 2 familiar sensory inputs into an unfamiliar whole. In contrast to our previous study of taste–smell integration, in which activation was observed within 2 regions of the basal forebrain in response to novel taste–smell pairs (Small et al. 1997a), we did not observe activity within this region. One reason for our failure to observe activity here may be that, in the earlier PET study, subjects were presented with many novel combinations during a single 60-s PET scan, whereas in the current study the incongruent solution was repeatedly experienced throughout the pilot session and fMRI experiment without any ill effects.

Finally, it is noteworthy that a double dissociation of function within 2 discrete regions of anterior ventral insula/caudal OFC was observed (Fig. 6). One region showed a superadditive response to a congruent taste–smell combination and not to the incongruent combination and the other a superadditive response to the incongruent taste–smell combination and not the congruent combination. In addition, direct contrasts of the 2 bimodal conditions revealed significantly greater or less activity within the respective regions. Interestingly, the responses in the region with the superadditive response to the congruent stimulus appeared to be independent of perceptual experiences of the stimuli, whereas the responses within the region showing the superadditive response to the incongruent taste–odor combination appeared related to the dimension of novelty/familiarity (Fig. 6).

FIG. 6.

Integration of familiar stimuli into a novel flavor. Activations from the group random effects analysis of [incongruent − (vanilla + salty)]. A: activity in the anterior ventral insula and a graph of the parameter estimates at this location for each of the 5 conditions (C = congruent, I = incongruent, S = sweet, N = salty, V = vanilla). B: activation from [congruent − (vanilla + sweet)] in turquoise and from [incongruent − (vanilla + salty] in pink. Note that these peaks are adjacent, nonoverlapping activations within the ventral insula/caudal OFC. C: activation at the parietal–temporal junction and a graph of the parameter estimates from each of the conditions at this location. D: activity in the ventral striatum and graph of parameter estimates at this location across the 5 conditions.

We also note that the salty taste produced as significant an activation in the insula and frontal operculum as did the congruent mixture (Fig. 4). This was unlikely related to unpleasantness of the saline because the incongruent was rated as similarly intense but did not activate this area. The finding may be related to recent psychophysical results reported by Laing and colleagues (2002). In this study subjects were asked to identify taste and odor qualities in taste–taste and taste–odor mixtures. Mixtures contained 2–5 different components (i.e., sweet, salty, octanol). The results indicated that the identity of the individual components of a mixture is generally lost if that mixture contains 3 or more components. This was true for all stimuli except salty, which maintained its identity even in the 5-component taste–taste mixture and the 4-component taste–odor mixture. It is not immediately clear why salt taste would have greater representation than the other tastes; especially because single-cell studies generally report sweet tastes make the most effective stimulus (Scott and Plata-Salaman 1999)—even in taste–taste mixtures (Plata-Salaman et al. 1996). Although it is possible that there are interspecies differences, future studies will need to confirm the reliability of this finding. There are several reasons why we believe it unlikely that the inclusion of saline in a taste–odor combination discounted the possibility of integration into superadditive responses. First, the strong response to saline was not consistent across all regions showing superadditive responses (Fig. 5), with the congruent mixture producing greater activation in the ACC and intraparietal sulcus compared with the salty taste alone and the incongruent stimulus, which contained a salty taste. Second, despite containing salty as one of its components, the incongruent mixture resulted in reduced activation compared with the congruent mixture in the ACC, IPS, frontal operculum, and insula. Nevertheless, it will be important to replicate these results with different stimulus pairs to evaluate the generalizability of these findings.

Neocortical responses

At the cortical level, multimodal integration is thought to be a product of an interaction between sensory-specific cortex and heteromodal neocortex (Calvert 2001) with increases in heteromodal regions either being associated with increases (Calvert et al. 1999; Macaluso et al. 2000) or decreases (Bushara et al. 2003; Laurienti et al. 2002) in unimodal sensory-specific cortex. The posterior parietal cortex is consistently activated during cross-modal binding (Bushara et al. 2001, 2003; Calvert 2001; Calvert et al. 2001; Gottfried and Dolan 2003; Macaluso et al. 2000). In accordance with multisensory integration in other modalities, we found experience-dependent integration in the posterior parietal cortex (intraparietal sulcus) in addition to the heteromodal paralimibic sensory regions (anterior cingulate, anterior dorsal insula, anterior ventral insula/caudal OFC).

We also observed a strong experience-dependent superadditive response in the frontal operculum. This region is considerably more lateral than the region we (Small et al. 1997a, 2003) and others (Faurion et al. 1998; Francis et al. 1999; Kinomura et al. 1994; O'Doherty et al. 2001a; Zald et al. 1998, 2002) observed in response to gustatory stimulation. However, responses within this lateral portion of frontal operculum have been reported during perception of a multisensory food stimulus (Del Parigi et al. 2002; Small et al. 2001). Because the frontal operculum does not appear to play any consistent role in cross-modal integration (Calvert 2001), it is possible that it plays a specific role in multisensory binding during flavor processing.


Although there has been considerable controversy surrounding the potential laterality of chemosensory systems, with some studies suggesting right hemisphere dominance (Barry et al. 2001; Savic and Berglund 2000; Small et al. 1997b, 1999; Zatorre et al. 1992) and others suggesting that laterality may depend on handedness (Cerf et al. 1998; Royet et al. 2003) or affective valence (Anderson et al. 2003; Small et al. 2003; Zald and Pardo 1997, 2000; Zald et al. 1998), it is striking that the activity we observed in the anterior ventral insula/caudal OFC occurred only in the right hemisphere. This asymmetry remained when the t-map threshold was dropped to P < 0.1. In accordance with arguments suggesting that pleasant taste and smell stimuli (which tend to be edible odors) frequently preferentially activate the right hemisphere, we have proposed a specific role for the right caudal orbitofrontal region in processing food-related chemosensory stimuli (Small et al. 2003).

In summary, taken together with previous findings in the literature the results from this study suggest that the insula, OFC, and ACC are key components of the network underlying flavor perception and that taste–smell integration within these and other regions is dependent on 1) mode of olfactory delivery and 2) previous experience with taste/smell combinations. This conclusion is in accordance with Rozin's proposal of olfaction as a dual sense modality (Rozin 1982) and Laing's suggestion that Rozin's model be expanded to include experience as a second critical component dictating functional classifications within chemosensory systems (Laing et al. 2002).


This work was supported by National Institutes of Health Grants R03 DC-006169-01 to D. M. Small and AG-0094 to D. R. Gitelman.


We thank Unilever Research for providing our stimuli.


  • The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.


View Abstract