JN Fuel your research with LabChart
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


J Neurophysiol 93: 3434-3441, 2005. First published February 2, 2005; doi:10.1152/jn.00555.2004
0022-3077/05 $8.00
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
93/6/3434    most recent
00555.2004v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (4)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Österbauer, R. A.
Right arrow Articles by Calvert, G. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Österbauer, R. A.
Right arrow Articles by Calvert, G. A.

Color of Scents: Chromatic Stimuli Modulate Odor Responses in the Human Brain

Robert A. Österbauer1, Paul M. Matthews1, Mark Jenkinson1, Christian F. Beckmann1, Peter C. Hansen2 and Gemma A. Calvert1,2,3

1Oxford Centre for Functional Magnetic Resonance Imaging of the Brain, John Radcliffe Hospital and 2University Laboratory of Physiology, University of Oxford, Oxford; and 3Department of Psychology, University of Bath, Bath, United Kingdom

Submitted 27 May 2004; accepted in final form 27 January 2005


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Color has a profound effect on the perception of odors. For example, strawberry-flavored drinks smell more pleasant when colored red than green and descriptions of the "nose" of a wine are dramatically influenced by its color. Using functional magnetic resonance imaging, we demonstrate a neurophysiological correlate of these cross-modal visual influences on olfactory perception. Subjects were scanned while exposed either to odors or colors in isolation or to color-odor combinations that were rated on the basis of how well they were perceived to match. Activity in caudal regions of the orbitofrontal cortex and in the insular cortex increased progressively with the perceived congruency of the odor-color pairs. These findings demonstrate the neuronal correlates of olfactory response modulation by color cues in brain areas previously identified as encoding the hedonic value of smells.


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
In everyday life, odors are often perceived together with visual cues. Both types of sensations interact to modulate the subjective experience of the stimuli from which they emanate. Behavioral studies demonstrated that the ability to correctly identify an odor relies heavily on visual inputs. For example, if subjects are presented with olfactory cues alone, only a few of the presented odors are identified correctly (Desor and Beauchamp 1974Go). There is a general lack of awareness of how difficult it can be to identify a substance by its odor alone because visual cues are usually present to facilitate identification. One visual feature with a particularly strong influence on odor perception is color. The color of a fruit, for example, provides an important visual cue about its ripeness and palatability.

Several reports of strong associations between certain odors and colors (Gilbert et al. 1996Go; Zellner and Kautz 1990Go) imply that such color-smell associations are most likely acquired and may be subject to variation between cultures. Nonetheless, some of these associations are particularly robust across subjects (e.g., yellow—lemon). Several color-smell associations are so compelling that an odor percept can change with color, e.g., subjects may perceive a cherry-flavored drink as orange-flavored if it is colored orange (DuBose et al. 1980Go). Furthermore, color not only facilitates odor identification but can also influence judgments of odor intensity and pleasantness. For example, both the perceived intensity and pleasantness of an appropriately colored solution (e.g., red—strawberry) is judged as higher than that of an inappropriately colored (e.g., green—strawberry) or a colorless solution even when the actual concentration of odorant remains constant (Zellner et al. 1991Go). More recently it has been found that tasteless red coloring added to a white wine induces a perceptual olfactory illusion that causes the wine to be described with olfactory terms typically used only for red wines (Morrot et al. 2001Go).

Despite the plethora of behavioral studies showing the influence of color on various olfactory judgments, the neurophysiological basis for these cross-modal interactions is still very poorly understood. In mammals, projections from the retina and olfactory bulb have been found to converge in the piriform cortex, the olfactory tubercle, the cortical region of the medial amygdala, and the lateral hypothalamus (Cooper et al. 1994Go). In primates, a prominent region for multisensory integration of olfactory and visual signals is the orbitofrontal cortex (OFC), where afferent inputs from both primary olfactory cortex and higher-order visual areas converge (Ongur and Price 2000Go). Additionally, populations of bimodal neurons responsive to both visual and olfactory stimulation have been reported within the OFC of non-human primates (Rolls and Baylis 1994Go). Recently, a study using functional magnetic resonance imaging (fMRI) in humans reported that congruent picture-odor combinations (such as the image of a bus and the smell of diesel) enhance neural activity, in the OFC and hippocampus (Gottfried and Dolan 2003Go). Whether similar regions participate in the synthesis of odors and lower level visual features such as color remains to be explored.

Although the precise neuronal mechanisms underlying the integration of visual and olfactory cues have not yet been studied in detail, some general principles characterizing cross-modal integration at the neuronal level have emerged from the study of other combinations of sensory modalities (Stein 1998Go). Specifically, when two or more sensory cues occur at the same time and in approximate spatial correspondence, the firing rate of a multisensory neuron to a stimulus in one modality can be measurably altered by the presence of a second stimulus in another modality. This is referred to as "multisensory integration." Rarely, this cross-modal modulation of the cell's output can exceed the sum of its response to either modality in isolation (a phenomenon referred to as superadditivity). Similar multisensory interactions have also been identified in the context of human neuroimaging experiments involving the chemical senses (de Araujo et al. 2003Go; Gottfried and Dolan 2003Go). In addition to the detection of cross-modal superadditive responses, systematic manipulation of the perceived congruency of two multisensory cues have also been shown to affect the height of the blood-oxygen-level-dependent (BOLD) response. In particular, increasingly better-matched cross-modal cues induce a corresponding enhancement of the hemodynamic response in areas thought to be involved in their combination. Parametric imaging designs such as these avoid some of the interpretation issues associated with superadditive interactions (see Calvert 2001Go and following text).

We used fMRI to investigate whether similar multisensory mechanisms might underlie the interaction of visual color and olfactory cues in the generation of an olfactory percept in humans. Based on findings of previous human neuroimaging studies of olfaction by others (Anderson et al. 2003Go; Gottfried and Dolan 2003Go; Gottfried et al. 2002aGo; Rolls et al. 2003Go; Zald and Pardo 2000Go; Zatorre et al. 1992Go), we hypothesized that if olfactory responsive areas including the piriform cortex, amygdala, and OFC participate in the integration of odor-color cues, activity in these regions should increase systematically with increasingly better matched odor-color pair trials as rated on an individual and subjective basis.


    METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Subjects

Ten healthy right-handed volunteers participated in this study (6 females and 4 males; mean age: 27 yr, age range: 22–35 yr). All participants gave written informed consent after having received the instructions for the study. One subject had to be discarded from the study because of excessive motion during scanning. Data analysis at the group level thus includes nine subjects. The study was approved by the Central Oxford Research Ethics Committee (C99.179).

Stimuli and task

In a behavioral experiment prior to scanning, 40 subjects were asked to match 1 of 17 different odors (Quest Intl.) to 1 of 10 different isoluminant colors presented on a computer screen and to rate the intensity, familiarity, and pleasantness of each odor and then attempt to identify it. Thirteen of the 17 odors were significantly often matched to one particular color (P < 0.01). Of these, the four color-odor pairs showing the most consistent match across subjects were used in the fMRI study. These congruent pairings were yellow—lemon, strawberry—red, spearmint—turquoise, and caramel—brown and were presented during scanning as well as the variably less congruent pairings of each odor with the remaining three colors. The four smells were highly matched for pleasantness, intensity, and familiarity (Table 1).


View this table:
[in this window]
[in a new window]
 
TABLE 1. Ratings of pleasantness, intensity and familiarity for the 4 odors

 
During scanning, subjects were presented for 6 s with a visual stimulus (yellow, red, turquoise, or brown), an olfactory stimulus (lemon, strawberry, spearmint, or caramel) or a bimodal visual-olfactory stimulus. The order of each of the three conditions was randomized. Each presentation was followed by a rest period of 30 s to avoid habituation to the odor stimuli. For the unimodal conditions, each of the four olfactory and color stimuli was presented three times. For the bimodal conditions, each odor was presented three times with a congruent color (i.e., lemon—yellow) and once with each of the remaining three colors. Overall, 12 visual, 12 olfactory, and 24 bimodal stimuli were presented over a period of 28 min 48 s.

Before scanning, subjects were informed that odors may occur in isolation or in the presence of a simultaneously presented color. In addition, color patches could also occur in the absence of an odor. They were also told that the order of these events would be random so they should maintain their attention to both channels equally throughout the experiment. Subjects were also instructed that on the bimodal conditions only, they were to rate each color-smell combination with respect to "goodness of fit between color and smell" on a rating scale ranging from 1 = very good to 4 = very bad using a four-button response box. Rating responses were cued by the word "rate" presented 15 s after the end of each bimodal presentation. To control for the motor response to the visual cue between the bi- and unimodal conditions, the same visual cue also appeared 15 s after each unimodal presentation, and subjects were instructed to press any of the four buttons. The advantage of a parametric design that requires a cross-modal congruency judgment is that it allows multisensory color-odor integration sites to be identified in the covariance analysis between bimodal trials and rating judgments. However, as the same judgment cannot be made in the unimodal conditions (for example, it is not possible for humans to rate the congruency between 2 simultaneously presented odors), contrasts between the bi- and unimodal conditions may be confounded by differences in the task requirements. Therefore superadditive effects [OV –(O + V)] must be interpreted cautiously.

Full-screen color images were generated using a video projector located outside the scanner room and projected onto a translucent screen placed directly outside the magnet bore. Subjects wore prism glasses so that they could see the screen while lying in the scanner. Olfactory stimuli were delivered using a custom-built, computer-controlled olfactometer. This was identical in design to that used by Rolls and colleagues (for a detailed description, see Rolls et al. 2003Go) so that switching between odorized and odorless air was free of any auditory, tactile, or thermal cues that could have alerted the subjects to the onset of odor delivery. Subjects were asked to breathe normally through their nose for the duration of the scan while an air stream of either odorized or odorless air was delivered at a flow rate of 6 l/s through a Teflon tube placed directly under the subject's nose. They were also informed that although during most presentations, the visual stimulus would be accompanied by an odor, they should refrain from sniffing at the onset of a color cue and continue to breathe normally. This instruction was included to avoid sniff-related activation of olfactory areas (Sobel et al. 1998Go) in the color-alone condition. Furthermore, in the absence of a color cue in the odor-alone condition, sniffing only at the onset of trials containing a color stimulus could have resulted in a potential confound of the comparison between bimodal color-odor and odor-alone conditions. Instead, during the scan, subjects were asked to detect the onset and offset of odors and to signal these events by making an appropriate ON-OFF button response. This method resulted in a slight jitter in the onset and duration of the odor trials that was taken into account for data analysis.

Data acquisition and analysis

Both functional and structural MRI images were acquired on a Siemens/Varian 3T system fitted with a birdcage head coil. For the functional data series, a total of 581 T2*-weighted echo-planar imaging (EPI) volumes were taken over a time period of 28 min 48 s. Each volume consisted of 25 continuous oblique coronal slices that were tilted ~20° toward the axial plane with an in-plane resolution of 3 x 2.5 mm and a thickness of 3 mm, covering the anterior half of the brain dorsally reaching a y coordinate of +4 of the Montreal Neurological Institute (MNI) standard space and a ventral coordinate of y = –12. This allowed coverage of all primary and secondary olfactory areas in OFC and the temporal lobes. Other imaging parameters were: TR = 3 s, 64 x 64 matrix, FOV 192 x 160 mm, TE = 25 ms, flip angle 90°. To minimize susceptibility artifacts in orbitofrontal brain regions, slices were acquired in the oblique coronal orientation (Deichmann et al. 2003Go) with a short echo time of 25 ms and a small in-plane voxel size. Additionally an automated frontal-weighted local shimming method was used (Wilson et al. 2002Go). This imaging protocol is optimized for fMRI studies of OFC function and has been extensively used by other groups on the same scanner (Rolls et al. 2003Go).

For registration into standard space, a whole brain T2*-weighted EPI volume (54 slices, TR = 7 s) and a high-resolution, whole brain T1-weighted morphological scan (1.5 mm slice thickness, 1.5 x 1.5 mm in-plane resolution) was acquired after the experimental paradigm.

Statistical image analysis was carried out using the FMRIB Software Library (www.fmrib.ox.ac.uk/fsl). The initial strategy for analyzing the bimodal condition was to first test for a linear correlation between the BOLD response and ratings of the congruency between smells and colors and then to assess superadditivity (i.e., where the activation to bimodal stimuli exceeds the sum of both unimodal stimuli). Prior to data analysis, the first 3 volumes were deleted. Analysis was carried out using the FMRIB expert-analysis tool (FEAT). The following preprocessing was applied: motion correction using MCFLIRT (Jenkinson et al. 2002Go); spatial smoothing using a Gaussian kernel of FWHM 5 mm; mean-based intensity normalization of all volumes by the same factor; nonlinear high-pass temporal filtering (Gaussian-weighted LSF straight line fitting, sigma = 36.0s). Each of the three event types (visual, olfactory, and bimodal) was modeled as a separate explanatory variable. A fourth explanatory variable with the same time course as the bimodal one was used to model the linear correlation between the BOLD response and the behavioral rating for the degree of matching. This variable was orthogonalized with respect to the bimodal one. Statistical analysis was carried out using FMRIB's improved linear model (FILM) with local autocorrelation correction (Woolrich et al. 2001Go).

For group analysis, the individual results were registered both to high-resolution anatomical MR images and to the MNI 152 standard image. Registration to high-resolution and standard images was carried out using FMRIB's linear image registration tool (FLIRT) (Jenkinson et al. 2002Go). Mixed-effects (often referred to as "random-effects") group analysis was carried out using FMRIB's local analysis of mixed effects (FLAME) software (Beckmann et al. 2003Go) with a cluster threshold of Z > 2.0 and a cluster significance threshold of P = 0.05 (corrected for multiple comparisons) (Forman et al. 1995Go; Friston et al. 1994Go; Worsley et al. 1992Go). This threshold was specifically chosen on the basis of pilot experiments that showed robust activation in olfactory areas in both temporal and orbitofrontal cortices using these parameters.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Behavioral

Analysis of the behavioral responses obtained during the fMRI experiment showed that for three odors (lemon, spearmint, and caramel), the color-smell combinations rated as most congruent in our preliminary experiment were indeed rated as significantly (Student's t-test P < 0.05) better matching than combinations of the same odor with any of the other three colors (Table 2). Although the strawberry odor showed a trend toward higher perceived congruence with red than the other three colors, the only statistically significant difference occurred between red and the turquoise. Note, however, that for the fMRI data analysis, each individual congruency rating for a bimodal trial was used to model the BOLD response (i.e., if a subject perceived the combination spearmint-brown as a very good match, then this was accounted for in the analysis model).


View this table:
[in this window]
[in a new window]
 
TABLE 2. Ratings of congruence for color-odor pairings

 
Unimodal stimulation

Presentation of odors alone produced bilateral activation in the piriform cortex/amygdaloid region (x, y, z = 14, –6, –30; Z score = 3.81 and x, y, z = –22, –2, –18; Z score = 3.29) and the putamen (x, y, z = 14, –4, –10; Z score = 3.83 and x, y, z = –6, 2, –8; Z score = 4.22; Fig. 1A). Lateralized activation was observed in the right orbitofrontal gyrus (x, y, z = 24/32/-24; Z score = 4.38) and in the left insular cortex (x, y, z = –44, 2, –14; Z score = 3.49). Both piriform and orbitofrontal areas are considered to be primary and secondary olfactory cortices in primates (Tanabe et al. 1975Go) and have been shown to activate in response to odorants in human neuroimaging studies (Zald and Pardo 2000Go). There was no significant activation of these regions when colors were presented alone.



View larger version (86K):
[in this window]
[in a new window]
 
FIG. 1. Group response to odor stimulation alone and brain regions displaying effects of superadditivity [bimodal – (vision + smell)]. Olfactory stimulation (A) evoked neural responses bilaterally in the piriform cortex/amygdala (white), the ventral striatum (blue), and the left insular cortex (red) as well as in the right orbitofrontal cortex (B). Superadditive responses were observed in medial orbitofrontal cortex (C, white) and the superior frontal gyrus (blue) and the anterior cingulate (D, white). Slice y coordinates are in the Montreal Neurological Institute (MNI) standard space and the right side of the brain is shown on the right side of the pictures.

 
Bimodal modulation and congruency

Using the ratings of relative color-odor congruency as a parametrically varying explanatory variable for brain activation changes, a network of brain areas exhibiting increasing activity with progressively higher perceived congruency was identified (Fig. 2). This network was entirely left lateralized and was mainly localized within the caudal orbitofrontal cortex around the olfactory sulcus (x, y, z = –16, 32, –4; Z score = 3.88), inferior frontal gyrus pars orbitalis (x, y, z = –20, 22, –16; Z score = 2.92) and gyrus rectus (x, y, z = –8, 16, –24; Z score = 3.34). Other brain regions exhibiting the same color-odor modulations were found in the anterior insular cortex (x, y, z = –32, 22, –8; Z score = 2.75), the frontal operculum (x, y, z = –54, 24, 4; Z score = 2.68), and the temporal pole (x, y, z = –56, 18, –8; Z score = 3.03).



View larger version (119K):
[in this window]
[in a new window]
 
FIG. 2. Brain regions showing odor-color modulation. A: the group activation map shows the main cluster of activation (red crosshairs) in the caudal region of the left orbitofrontal cortex (OFC; x, y, z = –8, 16, –24; Z score = 3.34) in all 3 slice orientations. Here, the blood-oxygen-level-dependent (BOLD) signal increased with greater perceived odor-color congruency. All activated voxels were rendered onto the standard MNI brain and the right side of the brain is shown on the right side of the pictures. B: the panel shows the parameter estimates for smell alone and the 4 different degrees of congruency of the color-odor pairs (ranging from 1 = very good to 4 = very bad) averaged across all 9 subjects. Black lines indicate the SE and asterisks significance levels of t-test (** = P < 0.01, * = P < 0.05, and NS = not significant).

 
Superadditivity

To assess whether indices of multisensory interactions similar to those previously reported in electrophysiological (Stein 1998Go) and neuroimaging experiments (Calvert 2001Go) could be identified in the current imaging experiment, we compared the BOLD response to odor-color pairings to the activation of odors and colors alone [bimodal – (odors + colors)]. This included the behavioral ratings in the overall model, so that the height of the BOLD response to a "very good fit" was modeled as larger than the response to a "very bad fit." Several regions displayed effects of superadditivity (Fig. 1, C and D): the medial wall of the superior frontal gyrus (x, y, z = –8, 48, 26; Z score = 4.48), the superior transverse frontopolar gyrus extending caudally into the anterior cingulate cortex (x, y, z = 4, 48, 8; Z score = 3.96 and x, y, z = –10, 48, 4; Z score = 3.83) and the OFC along the gyrus rectus (x, y, z = 4, 28, –19; Z score = 3.16 and x, y, z = –4, 35, –18; Z score = 3.07).

Neural suppression and incongruency

Incongruency in stimulus properties has been shown to suppress neuronal activity for audiovisual stimulus combinations (Calvert 2001Go) and congruent olfactory-taste stimuli have been reported to cause neural suppression (Small et al. 1997Go). To investigate whether similar effects are observable for olfactory-visual stimuli, we tested the linear correlation of the BOLD response with incongruency (i.e., brain areas that respond increasingly stronger the more incongruent the stimulus combinations are) and did not find an effect. Similarly, incongruent visual-olfactory stimuli did not significantly decrease the BOLD response below the levels observed to olfactory stimuli alone (Fig. 3).



View larger version (15K):
[in this window]
[in a new window]
 
FIG. 3. Time course of activation. The chart shows the group (n = 9) averaged time course of the percentage BOLD signal changes for both matching (rating of 1) and mismatching (rating of 4) odor-color trials as well as the odor alone trials in the peak voxel within the left OFC (x, y, z = –8, 16, –24). Stimulation started at time 0 and lasted for 6 s. As can be seen, congruent odor-color pairs elicited a significantly greater BOLD response than that observed to either incongruent pairs (t = 3.05, df = 8, P < 0.01) or odors in isolation (t = 3.26, df = 8, P < 0.01). Note, differences in the time to peak across conditions are most likely to reflect corresponding differences in BOLD amplitude (Miezin et al. 2000Go).

 

    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
The aim of this study was to probe the neurophysiological basis of visual influences on odor perception using fMRI. The presentation of odors in the absence of visual cues was first shown to stimulate the piriform/amygdaloid region, the right OFC and left insular cortex—consistent with previous olfactory neuroimaging experiments (Anderson et al. 2003Go; Gottfried et al. 2002aGo; Poellinger et al. 2001Go; Rolls et al. 2003Go; Zald and Pardo 2000Go). These areas are understood to correspond to human primary olfactory and associative olfactory cortices, respectively. A subset of these olfactory-responsive areas was also found to be sensitive to the perceived congruency of bimodal odor-color trials. Specifically, activation in the orbitofrontal and insular cortices increased in strength with increasingly higher ratings of the perceived congruency between specific odors and color patches. These findings are consistent with the hypothesis that color modifies the perception of odors at a relatively late stage of olfactory processing in heteromodal regions of the OFC and a region of the insular cortex previously implicated in flavor processing (de Araujo et al. 2003Go).

The detection of cross-modal modulatory effects in a network of areas including the OFC and insular cortex suggests that color may influence different aspects of olfactory processing. The OFC receives converging projections from multiple sensory modalities, including the primary olfactory cortex and the ventral visual pathway (Ongur and Price 2000Go; Rolls and Baylis 1994Go), making it a plausible initial site of information exchange between the visual and olfactory systems. Electrophysiological studies in non-human primates and imaging experiments in humans have implicated the OFC in stimulus-reinforcement associative learning using olfactory and visual stimuli (Gottfried et al. 2002bGo, 2003Go). For example, responses in the OFC to olfactory, gustatory, or visual cues previously associated with certain foods, are diminished when subjects are fed to satiety (Rolls 2001Go), and activity in this region has been shown to increase according to the perceived pleasantness of olfactory stimuli (Anderson et al. 2003Go). Together, these data contribute to the mounting evidence that it is specifically the rewarding properties of the sensory cues that are represented in the OFC. The systematic BOLD increase observed in the current study is therefore likely to reflect the physiological mechanism underlying the behavioral phenomenon whereby the more appropriate the color-odor combination, the greater is the perceived pleasantness of the odor (Zellner et al. 1991Go).

In addition to changing the hedonic value of odors, colors have also been shown to influence the perception of flavor (Rolls et al. 1982Go). Visuo-olfactory modulated responses in the current experiment were also detected in the left anterior insular cortex and the adjacent frontal operculum. These regions have previously been identified as areas of primary taste cortex in monkeys (Plata-Salaman et al. 1995Go; Yaxley et al. 1990Go) and are consistently activated during human imaging studies of taste (Frey and Petrides 1999Go; O'Doherty et al. 2001Go; Small et al. 1999Go, 2003Go). However, the agranular insular, located at the caudal border of the OFC, is also responsive to olfactory inputs (Weismann et al. 2001Go), and neuroimaging studies of olfaction have reported activation in this region (Cerf-Ducastel and Murphy 2001Go; de Araujo et al. 2003Go; Poellinger et al. 2001Go; Savic et al. 2002Go). Interestingly, this anterior region of the insular appears to be multimodal, i.e., it responds to both retro- and orthonasal olfactory stimuli as well as trigeminal inputs and pure tastes, leading to speculation that it must be involved in the cortical representation of flavor (de Araujo et al. 2003Go; Savic et al. 2002Go). To the extent that our experience of flavor is predominantly derived from our sense of smell, it seems highly plausible that color cues do not only modulate odor representations but also the putative taste/flavor sensations that may be automatically retrieved in the presence of food odors. In the current study then, we believe cross-modal modulations have an effect not only in areas of visual-olfactory processing but also drive the observed enhancement of neuronal activity in adjacent downstream regions involved in flavor perception.

A topical issue of debate in the multisensory literature relates to the appropriateness of different analytic criteria used for identifying a putative multisensory integration site. One relatively well-established approach has been to expose subjects to bimodal cues, matched or mismatched, along some parameter (e.g., time, space, or content) as well as to each modality independently and look for brain areas exhibiting positive and negative statistical interactions to congruent and incongruent bimodal cues, respectively [OV > O + V] and [OV < either O or V, whichever is the greater]. This strategy represents a reasonable attempt to capture BOLD responses that bear some resemblance to the known response properties (cross-modal response facilitation and suppression) of multisensory neurons. However, there are clearly theoretical complications involved in translating the behavior of individual neurons to neuronal population responses detected by fMRI (Beauchamp et al. 2004Go; Calvert 2001Go). In the current study, we adopted an alternative strategy for identifying multisensory integration sites, which was to present bimodal odor-color cues that were perceived to match to a varying degree and look for brain areas showing response changes that were correlated with the extent of perceived congruency. Parametric designs such as these avoid some of the inherent problems associated with calculation of interactions designed to mirror cross-modal response facilitation and suppression at the cellular level. One such problem relates to the difficulty in balancing task demands between bimodal and two independent unimodal conditions. For example, in the current study, it would have been unfeasible to match the congruency rating judgments required in the bimodal condition by having subjects similarly rate the co-presentation of two odors or two colors. Consequently, calculation of super- and subadditive responses in the current study may simply reflect greater attentional demands in the bimodal condition consistent with having to evaluate the congruency of two sensory cues and prepare to make a response. Nevertheless, in view of prevalence of these criteria in previous cross-modal imaging studies (Calvert 2001Go; Calvert et al. 2000Go; Foxe et al. 2002Go; Macaluso et al. 2000Go), we included this additional analysis for comparison purposes. This computation revealed that bimodal activations in the OFC did not only exhibit a linear correlation with subjective odor-color congruency ratings but exceeded the sum of the response to either odors or colors presented alone. The identification by both parametric and interaction analyses of the OFC is strongly suggestive of a prominent role for this area in the integration of odor-color cues. The additional detection, in the superadditive analysis only, of positive bimodal interactions in the anterior cingulate and superior frontal gyrus—areas previously implicated in studies manipulating attention (Badre and Wagner 2004Go; Kondo et al. 2004Go; Smith and Jonides 1999Go) is more likely to reflect the different task and associated attentional demands between the bi- and unimodal conditions.

The detection of BOLD responses resembling features of multisensory integration at the neuronal level, suggests that the same mechanism by which multisensory inputs are combined in non-human species—convergence onto sets of bimodal neurons (Stein 1998Go)—also underlies the integration of colors and odors. That such responses should have been identified in the OFC in the present experiment is consistent with electrophysiological studies of visual-olfactory or olfactory-gustatory integration in non-human primates (Rolls and Baylis 1994Go) and humans (de Araujo et al. 2003Go; Gottfried and Dolan 2003Go; Small et al. 2004Go). It is noteworthy, however, that in the current study there was no evidence of multisensory "response depression" in the presence of incongruent bimodal stimuli (i.e., where the bimodal condition falls below the level of activity elicited by a unimodal stimulus) as has been reported in previous neuroimaging studies of audiovisual (Calvert 2001Go) and visuo-tactile integration (Macaluso et al. 2000Go). Indeed, studies investigating cross-modal interactions between the chemical senses in humans have also found little or no evidence (de Araujo et al. 2003Go; Gottfried and Dolan 2003Go; Small et al. 2004Go) of suppression responses to incongruent inputs. At the cellular level, this feature of multisensory neurons in the superior colliculus has been shown to be mediated by the presence of inhibitory surrounds that prevent spatially discrepant bimodal stimuli from being co-localized (Stein 1998Go). This characteristic of multisensory integration, however, does not appear to be shared by putative multisensory flavor neurons in the OFC.

In a related imaging study by Gottfried and Dolan (2003)Go, olfactory responses in the OFC have also been shown to be modulated by co-presentation of semantically congruent or incongruent pictures associated with odors. Interestingly, as in the current study examining odor-color interactions, the precise location of these putative intersensory effects within the OFC differed depending on the analytic criteria chosen to identify cross-modal convergence zones. Of note is the observation that the peak activation associated with generic superadditive (OV > O + V) responses in the current study is both bilateral and more medially located than that detected in the odor-picture experiment. However, it does overlap closely with the activation showing a linear correlation with subjective postscan ratings in the Gottfied and Dolan (2003)Go study. The corresponding parametric model in the present study (albeit using ratings obtained during rather than post scanning) on the other hand, reveals that the region of OFC sensitive to subjective odor-color congruency matches lies both lateral and caudal to that implicated in the odor-picture study. Together, these observations would suggest that different visual parameters associated with objects (i.e., color and features) modulate olfactory-responsive neurons in adjacent OFC zones that are additionally context-sensitive (i.e., dependent on experimental task requirements).

In contrast to the study by Gottfried and Dolan (2003)Go, the current study failed to implicate a role for the hippocampus in odor-color integration. Due to our restricted coverage over the frontal third of the brain, we are not able to rule out the possibility that this structure is involved in the synthesis of odors with low-level sensory features such as color. However, an alternative possibility is that the hippocampus may only play a role in cross-modal binding when mediating the retrieval of semantic associations between odors and pictures—a question for further research.

Finally, previous neuroimaging studies of olfaction suggest stronger engagement of the right OFC during unimodal odor stimulation (Yousem et al. 1997Go; Zald and Pardo 2000Go; Zatorre et al. 1992Go). We observed the same trend toward right OFC activation in the odor alone condition, consistent with this notion. However, we found that modulation of the bimodal responses according to congruency was entirely left lateralized, indicating a functional segregation between the two hemispheres with respect to the type of olfactory task performed. This is consistent with the observation that hedonic judgments of odors elicit brain responses predominantly in the left OFC, whereas judgments of familiarity appear to recruit mainly the right OFC (Royet et al. 2001Go). Additionally, modulation of the BOLD response according to the perceived pleasantness of liquid flavor stimuli has been found in the left, but not right, OFC (Kringelbach et al. 2003Go). Last, cross-modal superadditive effects and enhancement of activity as a function of bimodal semantic congruency in Gottfried and Dolan's study (2003)Go were also observed to reside predominantly within the left OFC. Together, these data suggest that it is specifically the left OFC that integrates olfactory information with visual cues.

We believe the current findings provide a neurophysiological basis for behavioral effects such as changes in the perception of a white wine's odor when it is artificially colored red (Morrot et al. 2001Go) and the increased pleasantness of an odor when paired with an appropriate color (Zellner et al. 1991Go). A better understanding of how visual cues contribute to our sense of smell and taste could provide insights that will help guide the development of food products, particularly for those with an impaired sense of smell, a problem experienced by as many as 50% of the people >65 yr of age (Schiffman 1997Go).


    GRANTS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
This research was funded by a grant from the Sense of Smell Institute, New York. R. A. Österbauer is funded by the McDonnell Centre for Cognitive Neuroscience, Oxford, United Kingdom, and G. A. Calvert is funded by the Wellcome Trust. Additionally, we thank the Medical Research Council for support.


    ACKNOWLEDGMENTS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
We thank E. T. Rolls for helpful comments on this manuscript.


    FOOTNOTES
 
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Address for reprint requests and other correspondence: R. A. Österbauer, Oxford Centre for Functional Magnetic Resonance Imaging of the Brain, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK (E-mail: robert{at}fmrib.ox.ac.uk)


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Anderson AK, Christoff K, Stappen I, Panitz D, Ghahremani DG, Glover G, Gabrieli JD, and Sobel N. Dissociated neural representations of intensity and valence in human olfaction. Nat Neurosci 6: 196–202, 2003.[CrossRef][ISI][Medline]

Badre D and Wagner AD. Selection, integration, and conflict monitoring; assessing the nature and generality of prefrontal cognitive control mechanisms. Neuron 41: 473–487, 2004.[CrossRef][ISI][Medline]

Beauchamp MS, Lee KE, Argall BD, and Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41: 809–823, 2004.[CrossRef][ISI][Medline]

Beckmann CF, Jenkinson M, and Smith SM. General multilevel linear modeling for group analysis in FMRI. Neuroimage 20: 1052–1063, 2003.[CrossRef][ISI][Medline]

Calvert GA. Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11: 1110–1123, 2001.[Abstract/Free Full Text]

Calvert GA, Campbell R, and Brammer MJ. Evidence from functional magnetic resonance imaging of cross-modal binding in the human heteromodal cortex. Curr Biol 10: 649–657, 2000.[CrossRef][ISI][Medline]

Cerf-Ducastel B and Murphy C. fMRI activation in response to odorants orally delivered in aqueous solutions. Chem Senses 26: 625–637, 2001.[Abstract/Free Full Text]

Cooper HM, Parvopassu F, Herbin M, and Magnin M. Neuroanatomical pathways linking vision and olfaction in mammals. Psychoneuroendocrinology 19: 623–639, 1994.[CrossRef][ISI][Medline]

de Araujo IE, Rolls ET, Kringelbach ML, McGlone F, and Phillips N.Taste-olfactory convergence, and the representation of the pleasantness of flavor, in the human brain. Eur J Neurosci 18: 2059–2068, 2003.[CrossRef][ISI][Medline]

Deichmann R, Gottfried JA, Hutton C, and Turner R. Optimized EPI for fMRI studies of the orbitofrontal cortex. Neuroimage 19: 430–441, 2003.[CrossRef][ISI][Medline]

Desor JA and Beauchamp GK. The human capacity to transmit olfactory information. Percept Psychophys 16: 551–556, 1974.[ISI]

DuBose CN, Cardello AV, and Maller O. Effects of colorants and flavorants on identification, perceived flavor intensity and hedonic quality of fruit-flavored beverages and cake. J Food Sci 45: 1393–1399, 1980.[CrossRef]

Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, and Noll DC. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn Reson Med 33: 636–647, 1995.[ISI][Medline]

Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, and Murray MM. Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88: 540–543, 2002.[Abstract/Free Full Text]

Frey S and Petrides M. Re-examination of the human taste region: a positron emission tomography study. Eur J Neurosci 11: 2985–2988, 1999.[CrossRef][ISI][Medline]

Friston KJ, Worsley KJ, Frakowiak RSJ, Mazziotta JC, and Evans AC. Assessing the significance of focal activations using their spatial extent. Hum Brain Mapp 1: 214–220, 1994.

Gilbert AN, Martin R, and Kemp SE. Cross-modal correspondence between vision and olfaction: the color of smells. Am J Psychol 109: 335–351, 1996.[CrossRef][ISI][Medline]

Gottfried JA, Deichmann R, Winston JS, and Dolan RJ. Functional heterogeneity in human olfactory cortex: an event-related functional magnetic resonance imaging study. J Neurosci 22: 10819–10828, 2002a.[Abstract/Free Full Text]

Gottfried JA and Dolan RJ. The nose smells what the eye sees: cross-modal visual facilitation of human olfactory perception. Neuron 39: 375–386, 2003.[CrossRef][ISI][Medline]

Gottfried JA, O'Doherty J, and Dolan RJ. Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J Neurosci 22: 10829–10837, 2002b.[Abstract/Free Full Text]

Gottfried JA, O'Doherty J, and Dolan RJ. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301: 1104–1107, 2003.[Abstract/Free Full Text]

Jenkinson M, Bannister P, Brady M, and Smith S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17: 825–841, 2002.[CrossRef][ISI][Medline]

Kondo H, Osaka N, and Osaka M. Cooperation of the anterior cingulate cortex and dorsolateral prefrontal cortex for attention shifting. Neuroimage 23: 670–679, 2004.[CrossRef][ISI][Medline]

Kringelbach ML, O'Doherty J, Rolls ET, and Andrews C. Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness. Cereb Cortex 13: 1064–1071, 2003.[Abstract/Free Full Text]

Macaluso E, Frith C, and Driver J. Selective spatial attention in vision and touch: unimodal and multimodal mechanisms revealed by PET. J Neurophysiol 83: 3062–3075, 2000.[Abstract/Free Full Text]

Miezin FM, Maccotta L, Ollinger JM, Petersen SE, and Buckner RL. Characterizing the hemodynamic response: effects of presentation rate, sampling procedure, and the possibility of ordering brain activity based on relative timing. Neuroimage 11: 735–759, 2000.[CrossRef][ISI][Medline]

Morrot G, Brochet F, and Dubourdieu D. The color of odors. Brain Lang 79: 309–320, 2001.[CrossRef][ISI][Medline]

O'Doherty J, Rolls ET, Francis S, Bowtell R, and McGlone F. Representation of pleasant and aversive taste in the human brain. J Neurophysiol 85: 1315–1321, 2001.[Abstract/Free Full Text]

Ongur D and Price JL. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb Cortex 10: 206–219, 2000.[Abstract/Free Full Text]

Plata-Salaman CR, Scott TR, and Smith-Swintosky VL. Gustatory neural coding in the monkey cortex: acid stimuli. J Neurophysiol 74: 556–564, 1995.[Abstract/Free Full Text]

Poellinger A, Thomas R, Lio P, Lee A, Makris N, Rosen BR, and Kwong KK. Activation and habituation in olfaction—an fMRI study. Neuroimage 13: 547–560, 2001.[ISI][Medline]

Rolls BJ, Rowe EA, and Rolls ET. How sensory properties of foods affect human feeding behavior. Physiol Behav 29: 409–417, 1982.[CrossRef][Medline]

Rolls ET. The rules of formation of the olfactory representations found in the orbitofrontal cortex olfactory areas in primates. Chem Senses 26: 595–604, 2001.[Abstract/Free Full Text]

Rolls ET and Baylis LL. Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. J Neurosci 14: 5437–5452, 1994.[Abstract]

Rolls ET, Kringelbach ML, and de Araujo IE. Different representations of pleasant and unpleasant odours in the human brain. Eur J Neurosci 18: 695–703, 2003.[CrossRef][ISI][Medline]

Royet JP, Hudry J, Zald DH, Godinot D, Gregoire MC, Lavenne F, Costes N, and Holley A. Functional neuroanatomy of different olfactory judgments. Neuroimage 13: 506–519, 2001.[ISI][Medline]

Savic I, Gulyas B, and Berglund H. Odorant differentiated pattern of cerebral activation: comparison of acetone and vanillin. Hum Brain Mapp 17: 17–27, 2002.[CrossRef][ISI][Medline]

Schiffman SS. Taste and smell losses in normal aging and disease. J Am Med Assoc 278: 1357–1362, 1997.[Abstract]

Small DM, Gregory MD, Mak YE, Gitelman D, Mesulam MM, and Parrish T. Dissociation of neural representation of intensity and affective valuation in human gustation. Neuron 39: 701–711, 2003.[CrossRef][ISI][Medline]

Small DM, Jones-Gotman M, Zatorre RJ, Petrides M, and Evans AC. Flavor processing: more than the sum of its parts. Neuroreport 8: 3913–3917, 1997.[ISI][Medline]

Small DM, Voss J, Mak YE, Simmons KB, Parrish TR, and Gitelman DR. Experience-dependent neural integration of taste and smell in the human brain. J Neurophysiol 92: 1892–1903, 2004.[Abstract/Free Full Text]

Small DM, Zald DH, Jones Gotman M, Zatorre RJ, Pardo JV, Frey S, and Petrides M. Human cortical gustatory areas: a review of functional neuroimaging data. Neuroreport 10: 7–14, 1999.[ISI][Medline]

Smith EE and Jonides J. Storage and executive processes in the frontal lobes. Science 283: 1657–1661, 1999.[Abstract/Free Full Text]

Sobel N, Prabhakaran V, Desmond JE, Glover GH, Goode RL, Sullivan EV, and Gabrieli JD. Sniffing and smelling: separate subsystems in the human olfactory cortex. Nature 392: 282–286, 1998.[CrossRef][Medline]

Stein BE. Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Exp Brain Res 123: 124–135, 1998.[CrossRef][ISI][Medline]

Tanabe T, Iino M, and Takagi SF. Discrimination of odors in olfactory bulb, pyriform-amygdaloid areas, and orbitofrontal cortex of the monkey. J Neurophysiol 38: 1284–1296, 1975.[Abstract/Free Full Text]

Weismann M, Yousry I, Heuberger E, Nolte A, Ilmberger J, Kobal G, Yousry TA, Kettenmann B, and Naidich TP. Functional magnetic resonance imaging of human olfaction. Neuroimag Clin N Am 11: 237–250, 2001.

Wilson JL, Jenkinson M, de Araujo I, Kringelbach ML, Rolls ET, and Jezzard P. Fast, fully automated global and local magnetic field optimization for fMRI of the human brain. Neuroimage 17: 967–976, 2002.[CrossRef][ISI][Medline]

Woolrich MW, Ripley BD, Brady M, and Smith SM. Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage 14: 1370–1386, 2001.[CrossRef][ISI][Medline]

Worsley KJ, Evans AC, Marrett S, and Neelin P. A three-dimensional statistical analysis for CBF activation studies in human brain. J Cereb Blood Flow Metab 12: 900–918, 1992.[ISI][Medline]

Yaxley S, Rolls ET, and Sienkiewicz ZJ. Gustatory responses of single neurons in the insula of the macaque monkey. J Neurophysiol 63: 689–700, 1990.[Abstract/Free Full Text]

Yousem DM, Williams SC, Howard RO, Andrew C, Simmons A, Allin M, Geckle RJ, Suskind D, Bullmore ET, Brammer MJ, and Doty RL. Functional MR imaging during odor stimulation: preliminary data. Radiology 204: 833–838, 1997.[Abstract/Free Full Text]

Zald DH and Pardo JV. Functional neuroimaging of the olfactory system in humans. Int J Psychophysiol 36: 165–181, 2000.[CrossRef][ISI][Medline]

Zatorre RJ, Jones Gotman M, Evans AC, and Meyer E. Functional localization and lateralization of human olfactory cortex. Nature 360: 339–340, 1992.[CrossRef][Medline]

Zellner DA, Bartoli AM, and Eckard R. Influence of color on odor identification and liking ratings. Am J Psychol 104: 547–561, 1991.[CrossRef][ISI][Medline]

Zellner DA and Kautz MA. Color affects perceived odor intensity. J Exp Psychol Hum Percept Perform 16: 391–397, 1990.[CrossRef][ISI][Medline]




This article has been cited by other articles:


Home page
Chem SensesHome page
M Luisa Dematte, D. Sanabria, and C. Spence
Cross-Modal Associations Between Odors and Colors
Chem Senses, July 1, 2006; 31(6): 531 - 538.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
93/6/3434    most recent
00555.2004v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (4)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Österbauer, R. A.
Right arrow Articles by Calvert, G. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Österbauer, R. A.
Right arrow Articles by Calvert, G. A.