INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Salient features of the sensory environment tend to preferentially draw our attention and enter our awareness. The salience of a given stimulus reflects its potential relevance to behavior and is therefore influenced by behavioral context. However, salience may also depend on factors independent of behavioral context, such as stimulus intensity, frequency of appearance, or novelty. Lesion data, in conjunction with neuroimaging and electrophysiological studies, suggest that the identification of salient events in the sensory environment relies on a large-scale distributed network of brain regions with frontal, parietal, and cingulate components (Mesulam 1981, 1999; Swick and Knight 1998).

Many previous studies of stimulus salience have used an "oddball" paradigm examining the response to infrequent target, nontarget, or novel nontarget deviant stimuli embedded in a repetitive train of standard stimuli. Many of these studies have focused on the role of the prefrontal cortex (PFC). However, there is evidence that areas outside of PFC, such as the temporo-parietal junction (TPJ) and hippocampus, also play a role in detecting novel or otherwise salient stimuli (Knight 1996; Knight et al. 1989; McCarthy et al. 1997; Yamaguchi and Knight 1991). Furthermore, most studies using the oddball paradigm have presented infrequent or novel stimuli in the context of a particular task, which can alter the observed cortical response (Katayama and Polich 1998; Suwazono et al. 2000). Hence, the sensitivity of regions other than PFC to the salience of stimuli presented in a neutral behavioral context remains to be investigated.

We have previously identified a multimodal network of regions responsive to visual, auditory, and tactile stimulus changes in the absence of any task (Downar et al. 2000). This network is dominated by the right TPJ, a region encompassing the supramarginal gyrus (SMG), caudal parts of the superior temporal gyrus (STG), and dorsal-rostral parts of the occipital gyri. The network also includes smaller activations in inferior frontal gyrus (IFG), anterior insula, and cingulate and supplementary motor areas (CMA/SMA). These cortical regions correspond closely to the regions most frequently damaged in patients with hemineglect syndromes (Vallar 1998), suggesting that they may play a role not only in detecting sensory changes but more generally in identifying salient features of the sensory environment. This is consistent with evidence that the right TPJ activates during the detection of left and right visual targets and nontargets for saccades (Perry and Zeki 2000), particularly at unattended locations (Corbetta et al. 2000). It is also consistent with our recent finding that regions within the TPJ and anterior cingulate cortex (ACC) respond preferentially to behaviorally relevant rather than irrelevant changes in visual and auditory stimuli (Downar et al. 2001). These findings raise the question of whether the TPJ and the other multimodal regions are also sensitive to factors independent of behavioral context. If so, these regions may perform a general role in identifying salient events in the sensory environment across multiple modalities, whether their salience stems from behavioral context or not.

In the present study, we sought to determine whether the TPJ and other multimodally responsive regions are sensitive to the salience of stimuli across multiple sensory modalities in a neutral behavioral context. We used stimulus novelty to manipulate salience in a passive-perception task, to avoid imposing a particular behavioral context on the stimuli. We used whole-brain event-related functional magnetic resonance imaging (fMRI) to identify brain regions showing a greater response to novel versus familiar deviant stimuli across visual, auditory, and tactile sensory modalities.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Subjects

Subjects were six male and four female right-handed individuals, ages 23-46, with no prior history of neurological injury. All subjects gave informed written consent for the experimental procedures, approved by the institutional Review Committee.

Stimuli and task design

Visual stimuli were back-projected onto a screen viewed through an angled mirror in the head coil. Auditory stimuli were delivered via a piezoelectric speaker connected to a custom-built set of pneumatic headphones. Presentation of visual and auditory stimuli was computer-controlled. Visual stimuli consisted of centrally presented monochrome shapes (Fig. 1) covering 7-9° of the visual field. Auditory stimuli consisted of 2-s samples of abstract and everyday sounds (e.g., rising tones, chimes). Tactile stimuli consisted of brushing and tapping patterns applied to the lower right leg with the bristle and flat ends of a 1-cm² brush (e.g., brushing in an "X" pattern, tapping in a circular pattern). Subjects were instructed to keep their eyes closed during the auditory and tactile runs.

View larger version (24K): [in this window] [in a new window]   Fig. 1. Schematic of visual stimulus presentation schedule. A: the stimulus pool from which baseline, familiar, and novel stimuli were drawn randomly for each subject. B: sample sequences of visual stimulation for 3 subjects, with predicted hemodynamic responses for familiar (Fam) and novel (Nov) stimuli. Analogous schedules of presentation were used for stimuli in the auditory and tactile modalities.

Subjects underwent three separate fMRI runs for visual, auditory, and tactile sensory modalities. The order of runs was counterbalanced across subjects. In each modality, a continuous baseline stimulus (analogous to the standard stimuli in an oddball paradigm) was interrupted by briefly presented (2 s) deviant stimuli every 10, 12, or 14 s. The first 10 deviant stimuli were identical and served to establish a familiar stimulus against which to contrast novel stimuli. The next 20 deviant stimuli consisted of 10 presentations of the familiar stimulus and 10 novel stimuli in a pseudo-random order. The specific baseline, familiar, and novel stimuli were drawn at random for each subject from a pool of 12 stimuli (Fig. 1) to counterbalance any differences in stimulus intensity, size, etc., between novels, familiars, and baseline across subjects.

Imaging

A 1.5-T Echospeed MRI system (GE Medical Systems, Milwaukee, WI) and a standard quadrature head coil were used to obtain all images. For anatomical images, a T1-weighted 3D SPGR sequence (flip angle = 45°, TE = 5 ms, TR = 25 ms) was used to generate 124 1.5-mm-thick sagittal slices (256 × 256 matrix, 24 × 24 cm field of view). For functional imaging, 25 contiguous 4-mm-thick T2*-weighted axial slices were acquired with a gradient echo sequence using a single-shot spiral trajectory through k-space (Glover and Lee 1995), flip angle = 85°, TE = 40 ms, TR = 2000 ms, 64 × 64 matrix, 20 × 20 cm field of view. During each run, 183 functional volumes were acquired, of which the first three were discarded for signal equilibration.

Data processing and analysis

Data were analyzed using BrainVoyager 4.1 (Brain Innovation, Frankfurt). Data were motion-corrected using sinc interpolation, spatially smoothed with a 6-mm FWHM Gaussian filter, transformed into standard stereotactic space (Talairach and Tournoux 1988), and linear-interpolated to 3 × 3 × 3 mm resolution. Individual subjects' data were averaged together for group analysis.

Areas showing significantly greater responses to novel deviant stimuli versus familiar deviant stimuli in each modality were identified by linear correlation to the predicted hemodynamic response to novel stimuli minus the predicted hemodynamic response to familiar stimuli (Fig. 1). Thus, any regions identified in this comparison show an additional amplitude of response to novel deviants over and above any response they may show to familiar deviants; regions that respond equally strongly to both novel and familiar deviants are not revealed. Predicted hemodynamic responses were constructed using an empirically derived hemodynamic response waveform based on previous data (Downar et al. 2000). The initial 10 consecutive presentations of the familiar stimulus were excluded from this analysis, since the familiar stimulus may have retained some degree of novelty during these initial presentations.

Multimodal activations were identified by conjunction analysis of the visual and auditory and tactile maps, to ensure a response to novelty across all three modalities, rather than merely an exceptionally strong response in one modality but not in the others (Friston et al. 1999; Price and Friston 1997). A voxelwise conjoint P < 0.0001 threshold was used in the conjunction analysis to identify multimodal activations. Given this threshold and the total volume of the statistical map (1,502,673 mm³), approximately 150 1-mm³ voxels in the map would be expected to show activation due to type I errors. These voxels would also be expected to show clustering, as they were interpolated from 3 × 3 × 3 mm resolution, smoothed functional data. As a conservative measure to minimize false-positive activations, we therefore required a minimum cluster size of 150 contiguous interpolated 1-mm³ voxels.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

All subjects confirmed in debriefing sessions that they had remained awake and attentive during imaging and had been able to discriminate the stimuli presented in all sensory modalities, but had not covertly or overtly counted or otherwise responded to the stimuli.

Regions showing a significantly greater response to novel versus familiar stimuli across all three sensory modalities formed a cortical network with frontal, parietal, cingulate, and insular components (Table 1, Fig. 2). This network included two activations in the right TPJ: one in the SMG and one in the STG. Activations in the right IFG, right anterior insula, left ACC, and a small region in left inferior temporal gyrus were also observed.

View larger version (73K): [in this window] [in a new window]   Fig. 2. Multimodal novelty-sensitive brain regions. A: activations in the TPJ, IFG, ACC, and anterior insula showed a significantly greater response to novel than familiar stimuli. Activations represent group-average data for all 10 subjects. B: average event-related hemodynamic responses (±SE) to all novel and all familiar stimuli show strong responses to novel stimuli but nonsignificant responses to familiar stimuli in the TPJ, IFG, and ACC.

To compare the novelty-sensitive regions identified in the present study to the regions we previously identified as sensitive to behavioral context (Downar et al. 2001), we superimposed the activation maps for each study (Fig. 3). The superior novelty-sensitive TPJ activation partially overlapped a region in the SMG sensitive to the behavioral context of visual and auditory stimulus changes (Fig. 3A). In contrast, the more inferior novelty-sensitive TPJ activation bordered a region in the STG previously identified as responding similarly to all stimulus changes, regardless of behavioral context. The novelty-sensitive left ACC region (Fig. 3B) partially overlaps the anterior margin of a large region activated exclusively during changes in a task-relevant stimulus (requiring a motor response). The novelty-sensitive right IFG activation overlapped a large region of IFG responding similarly to all stimulus changes, regardless of behavioral context (Fig. 3C), and bordered a region activated exclusively during changes in a task-irrelevant stimulus (requiring motor inhibition).

View larger version (75K): [in this window] [in a new window]   Fig. 3. Novelty-related and generalized functions for TPJ, ACC, and IFG. The results of the present study are compared with a previous study on the effects of stimulus task-relevance. A: the novelty-sensitive TPJ subregion in the SMG partially overlaps a large right TPJ region sensitive to the task-relevance of nonnovel sensory events. The novelty-sensitive TPJ activation in the STG borders a region responding similarly to all sensory events, regardless of relevance. B: the novelty-sensitive left ACC region partially overlaps a large medial region spanning ACC, CMA, and SMA, which was activated exclusively during task-relevant stimulus changes. C: the novelty-sensitive region in right IFG partially overlaps a region activating exclusively during task-irrelevant sensory events and borders a region activating during both task-irrelevant and task-relevant sensory events.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

The results of the present study support the hypothesis that the TPJ, IFG, ACC, and anterior insula serve not only to detect events in the sensory environment across multiple modalities, but more specifically to identify salient (in this case, novel) sensory events, even when the salience is not due to behavioral context.

Two TPJ subregions were sensitive to stimulus novelty: one in the SMG and one in the STG. The observation of temporo-parietal sensitivity to stimulus novelty is consistent with electrophysiological data showing reduced P3a event-related potentials in patients with prefrontal or temporo-parietal lesions in response to novel stimuli in visual, auditory, or tactile modalities (Knight et al. 1989; Verleger et al. 1994; Yamaguchi and Knight 1992). The TPJ has also been implicated in the detection of visual target oddballs differing from the standard stimulus in form, location, or both (Marois et al. 2000). Right TPJ activation has also been observed during tasks involving cued saccades and shifts of attention to visual targets. The right SMG responds to both targets and nontargets appearing on the right or left (Perry and Zeki 2000), and the right TPJ responds in particular to targets appearing at unexpected locations (Corbetta et al. 2000). The TPJ responds to visual and auditory target oddballs requiring silent counting or button-pressing responses (Linden et al. 1999), to visual target but not distractor oddballs (Clark et al. 2000), and to changes in task-relevant visual and auditory stimuli (Downar et al. 2001). However, these stimuli all required some sort of behavioral response. The results of the present study identify a factor affecting the amplitude of TPJ response even in a neutral behavioral context: stimulus novelty. This finding, taken together with the results of previous studies, suggests that the TPJ plays a general role in identifying salient stimuli in the sensory environment across multiple modalities, whether the salience is due to the current behavioral context or not.

The proposed role of the TPJ in identifying salient environmental stimuli is consistent with evidence linking lesions of this region to hemineglect syndromes, in which salient stimuli contralateral to the lesion fail to capture attention and enter awareness (Driver and Vuilleumier 2001; Heilman et al. 1993; Vallar 1998). Although hemineglect is traditionally attributed to posterior parietal or temporo-parietal lesions, recent evidence indicates that lesions giving rise to "pure" hemineglect in the absence of visual-field deficits are centered on the STG (Karnath et al. 2001). Centers of lesion overlap in hemineglect reported in the study of Karnath et al. (2001) at Talairach coordinates (x, 59; y, 30; z 15) and (x, 51; y, 32; z 15) lie only 11 mm from the novelty-sensitive SMG activation identified in the present study. However, it should be noted that the novelty-sensitive STG activation identified in the present study lies 28-44 mm caudal to the centers of lesion overlap reported by Karnath et al. Nonetheless, the general correspondence between the TPJ activations identified in the present study and the neural correlates of hemineglect support a role for this region in identifying, attending to, and becoming aware of salient environmental stimuli.

The multimodal novelty-sensitive regions in the left ACC was unexpected, given that the task involved only passive observation. This region of the ACC is typically activated during response competition or interference, as in the Stroop task (Bush et al. 2000; Carter et al. 1999; Pardo et al. 1990). The ACC has not previously been implicated in the passive observation of novel stimuli. However, the neighboring left CMA and SMA respond to nonnovel, multimodal stimuli during passive observation (Downar et al. 2000). Moreover, a similar region of the ACC responds to painful stimuli, possibly due to their high motivational salience (Davis 2001; Hutchison et al. 1999). The left ACC activation in the present study may likewise reflect a greater motivational salience for novel versus familiar stimuli (Mesulam 1999). An alternative interpretation is that the left ACC is not part of the otherwise right-lateralized network of regions sensitive to stimulus novelty, but rather co-activates with this network for some other reason. For example, subjects may have treated novel events as targets for some unfocused response, despite instructions to observe the stimuli passively and despite debriefing reports denying overt or covert responses. ACC activation may simply have reflected the process of evaluating the novel stimulus and deciding not to respond to it. Consistent with this hypothesis, ACC activation has been reported for no-go trials in a go/no-go paradigm (Liddle et al. 2001). Further study will be required to elucidate the role of the ACC in detecting or responding to novel stimuli.

The novelty sensitivity of the right IFG is consistent with PFC responses to oddball or otherwise salient stimuli. Lesion data and electrophysiological studies suggest a PFC role in detecting and attending to novel oddball stimuli (Knight 1984; Knight and Nakada 1998). Similarly, neuroimaging studies have identified regions in PFC responsive to rare visual distractors (Clark et al. 2000) or novel auditory stimuli (Kiehl et al. 2001; Opitz et al. 1999a,b). However, this region has also been shown to activate on no-go trials during a go/no-go task and during cognitive set shifting, both of which require inhibition of prepotent motor responses (Konishi et al. 1998a,b, 1999). Taken together, these findings suggest that the prefrontal response to novel stimuli may reflect a more general process than the detection of novel sensory events. The IFG may play a broad role in evaluating the potential relevance of sensory stimuli and in inhibiting prepotent responses to stimuli not requiring a response. A role for the IFG in stimulus evaluation would be consistent with reports that P3a amplitude predicts the duration of viewing time for visual stimuli (Daffner et al. 1998).

As described in METHODS, subjects were instructed to keep their eyes open only for the visual run. It should be noted that some extrastriate visual areas show differences in baseline signal under eyes-closed versus eyes-open states (Raichle et al. 2001). However, these differences would be unlikely to influence the results of our analysis, which was based on the comparison of transient responses to stimulus events within each modality rather than tonic differences in baseline signal across modalities. Furthermore, the use of conjunction analysis across the three modalities would exclude any effects confined to just the visual as compared with the auditory and tactile modalities.

A multimodal network consisting of temporo-parietal, frontal, and cingulate components is thought to play a key role in identifying and evaluating salient events in the sensory environment (Knight et al. 1995; Mesulam 1981, 1998). The present study supports this proposal by showing that these regions show sensitivity to stimulus salience, even when the salience depends on factors independent of behavioral context (i.e., novelty). Thus the TPJ, ACC, IFG, and anterior insula may comprise a multimodal network that serves to identify, evaluate, and plan responses to potentially important sensory stimuli. This network may also serve more generally in mediating attention to and awareness of salient events in the sensory environment.