Spatiotemporal Activity of a Cortical Network for Processing Visual Motion Revealed by MEG and fMRI

S. P. Ahlfors, G. V. Simpson, A. M. Dale, J. W. Belliveau, A. K. Liu, A. Korvenoja, J. Virtanen, M. Huotilainen, R.B.H. Tootell, H. J. Aronen, R. J. Ilmoniemi


A sudden change in the direction of motion is a particularly salient and relevant feature of visual information. Extensive research has identified cortical areas responsive to visual motion and characterized their sensitivity to different features of motion, such as directional specificity. However, relatively little is known about responses to sudden changes in direction. Electrophysiological data from animals and functional imaging data from humans suggest a number of brain areas responsive to motion, presumably working as a network. Temporal patterns of activity allow the same network to process information in different ways. The present study in humans sought to determine which motion-sensitive areas are involved in processing changes in the direction of motion and to characterize the temporal patterns of processing within this network of brain regions. To accomplish this, we used both magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). The fMRI data were used as supplementary information in the localization of MEG sources. The change in the direction of visual motion was found to activate a number of areas, each displaying a different temporal behavior. The fMRI revealed motion-related activity in areas MT+ (the human homologue of monkey middle temporal area and possibly also other motion sensitive areas next to MT), a region near the posterior end of the superior temporal sulcus (pSTS), V3A, and V1/V2. The MEG data suggested additional frontal sources. An equivalent dipole model for the generators of MEG signals indicated activity in MT+, starting at 130 ms and peaking at 170 ms after the reversal of the direction of motion, and then again at ∼260 ms. Frontal activity began 0–20 ms later than in MT+, and peaked ∼180 ms. Both pSTS and FEF+ showed long-duration activity continuing over the latency range of 200–400 ms. MEG responses in the region of V3A and V1/V2 were relatively small, and peaked at longer latencies than the initial peak in MT+. These data revealed characteristic patterns of activity in this cortical network for processing sudden changes in the direction of visual motion.


The cerebral cortex processes information via networks of anatomically and functionally differing areas. In monkeys, anatomic feed-forward and feed-back connections suggest a hierarchical order among the cortical areas (Felleman and Van Essen 1991). Functional studies of onset latencies of neural activity within the areas indicate both serial and parallel processing (Nowak and Bullier 1997; Petersen et al. 1988; Raiguel et al. 1989; Schmolesky et al. 1998; Schroeder et al. 1998). The numerous interconnections suggest that multiple areas interact in the course of stimulus processing. A given network of brain areas can give rise to many functional operations through different temporal patterns of interactions between areas. A greater understanding of cortical information processing will be achieved through investigation of the functional properties of each area, the connectivity between areas, and the dynamic patterns of activity in networks of areas.

Electrophysiological methods like magnetoencephalography (MEG) and electroencephalography (EEG) provide measures that reflect neural ensemble activity in the millisecond time scale (e.g.,Hämäläinen et al. 1993; Regan 1989; Simpson et al. 1995; Williamson and Kaufman 1981). Estimating the locations of the brain sources of MEG and EEG activity, however, is problematic due to the nonuniqueness of the solutions. Modeling procedures are required that ideally incorporate as much a priori information as possible to maximize estimation accuracy. Functional magnetic resonance imaging (fMRI) and positron emission tomography (PET), which measure hemodynamic changes related to neural activity on the time scale of seconds, have a relatively high spatial resolution (e.g., Belliveau et al. 1991; Raichle 1987). Combining hemodynamic and electrophysiological information holds promise for imaging patterns of human brain activity in both space and time (Belliveau et al. 1993; George et al. 1995; Heinze et al. 1994; Korvenoja et al. 1999; Mangun et al. 1998; Menon et al. 1997; Simpson et al. 1993, 1995). In the present study, we developed and applied this combined approach to examine the temporal patterns of activity simultaneously in all cortical areas responding to changes in the direction of visual motion.

A sudden change in the direction of visual motion is a salient feature of visual information. Many cortical areas in monkey (Andersen et al. 1997; Boussaoud et al. 1990;Newsome et al. 1990) and human (Cheng et al. 1995; Cornette et al. 1998; Dupont et al. 1994; Tootell et al. 1995b, 1997; Zeki et al. 1991) participate in the processing of visual motion. Relatively little, however, is known about the processing of sudden changes in direction (Cornette et al. 1998). Previous fMRI studies have demonstrated that a pattern of expanding and contracting rings is an effective stimulus for activating a subset of the human visual motion areas, particularly MT+ in the occipito-temporal cortex (Tootell et al. 1995a, 1998). Here, we performed an fMRI-guided MEG source analysis to determine the dynamic patterns of cortical activity related to the processing of sudden changes in the motion direction of this stimulus. Our combined MEG-fMRI approach allowed us to identify areas in the cortical network responsible for processing this visual motion and to estimate temporal patterns of activity within and between them.


Subjects and stimuli

Four subjects (all male, aged 25–45 yr, normal or corrected-to-normal visual acuity) were studied with both MEG and fMRI. In the course of data analysis, one of the subjects was excluded due to movement artifact. One of the subjects (S3) was left-handed. Supporting data were obtained from three other subjects with MEG only and from two with fMRI only. The visual motion stimulus was 6° in diameter, consisting of a pattern of 10 concentric, expanding and contracting rings (see Fig.1 A). The direction of motion was reversed every 3 s, and the speed of motion was 2.4°/s. The contrast (L maxL min)/(L max +L min) was ∼5%; in the MEG experiment the mean luminance was 11 cd/m2 forsubjects S1 and S2 and 105 cd/m2 for S3 and S4. The purpose of the low contrast was to specifically enhance the response in MT+ relative to areas V1/V2 (Tootell et al. 1995b). The subjects were instructed to fixate on a stationary dot in the center of the screen during all recordings.

Fig. 1.

Visual motion-reversal evoked magnetic fields. A:distribution of the magnetoencephalographic (MEG) responses as a function of time, over the right hemisphere of subject 1. Radial motion stimulus consisted of a pattern of concentric rings (top right inset). Rectangles depict the locations of sensor pairs (top left). Responses to changes in the motion direction (expanding to contracting, and vice versa) were averaged together, 708 epochs in total. Top andbottom waveforms correspond to ∂B z/∂x and ∂B z/∂y, whereB z is the normal component of the magnetic field; and x, y, and z refer to the local coordinate system of the sensor units. The position of the vertical scale bar indicates the time of motion direction reversal. Vertical and horizontal electrooculograms (EOG) are shown bottom right. B: responses in 2 MEG channels enlarged (indicated by shading in A). Vertical markers indicate the latencies of prominent deflections (135, 170, and 260 ms).

MEG data acquisition

The MEG data were recorded with a whole-head 122-channel dc-SQUID device (Neuromag) with planar first-order gradiometric detectors (Ahonen et al. 1993). The analogue filter passband was 0.03–100 Hz; the sampling frequency was 397 Hz. MEG signals were acquired in epochs consisting of 200 ms preceding each reversal and 800 ms postreversal. Vertical and horizontal electrooculograms (EOG) were recorded to detect and discard epochs with eye movement or blink artifacts. For each subject, 500–700 epochs were averaged. The responses were low-pass filtered at 40 Hz, and the zero level (baseline) was set to the mean of the signal 100 ms preceding the reversal. Thus the MEG signal reflected transient responses evoked by the reversal of the direction of motion not the sustained activity related to the continuous motion, which was set as the baseline. The location of the head relative to the magnetometer was determined with the help of small marker coils attached to the head (Ahlfors and Ilmoniemi 1989).

Anatomic MR images

Locations of active regions were identified and visualized within high-resolution anatomic MRI of each subject, coregistered with MEG using digitized locations of the nasion and preauricular points. The three-dimensional anatomic images were obtained with a Siemens 1.5-T scanner using an MPRAGE sequence (TR = 9.7 ms; effective IT = 20 ms; TE = 4 ms; flip angle = 10°; voxel size, 1 × 1 × 1 mm3).


BOLD-contrast (blood-oxygenation-level dependent) fMRI images were obtained using a GE 1.5-T scanner with Advanced NMR echo planar, asymmetric spin echo sequence, a standard head coil, and a bite bar (Tootell et al. 1995b). In each run, 64 sets of 25 slices were collected (TR = 7 s; slice thickness, 5 mm; pixel size, 3 × 3 mm2). The pattern of stimulation alternated between periods of moving stimuli and stationary patterns in cycles of either 30 or 40 s. The activation time series for each voxel was Fourier transformed. Activation significance values were computed on a voxel-by-voxel basis by using F statistics based on a comparison between the Fourier domain amplitudes at the stimulation frequency (the 30- or 40-s cycles of motion vs. stationary, not the 3-s intervals between reversals) and the average amplitude at the other frequencies, except the harmonics of the stimulus frequency (Tootell et al. 1997). Active regions were identified on the basis of maps of significance value thresholded atP < 0.01. For comparisons with MEG dipole analysis, locations of foci within regions were determined by finding voxels of maximum significance value.

Blocked design with short intervals between reversals was required to eliminate confounding motion aftereffects that would occur with long periods of continuous motion. Although the event-related fMRI paradigm (Buckner et al. 1996; Dale and Buckner 1997; Friston et al. 1998; McCarthy et al. 1997; Menon et al. 1997) would make it possible to have a baseline comparable with the MEG, it was not feasible for this experiment. Event-related fMRI techniques employing randomized stimulation (Dale and Bruckner 1997) could not be used due to the fact that we have only one stimulus type. Blocked design was also desirable, because it provides a better signal-to-noise ratio. Pilot studies indicated that some regions had very low-amplitude activations; thus it was important to optimize the detection of activated areas. The stimulation of the brain was identical during fMRI and MEG measurements (i.e., continuous motion transiently interspersed with reversals of direction at 3-s intervals); however, the fMRI signals acquired with the blocked design represented the summation of responses to continuous motion and/or reversals. This was not problematic for using the fMRI foci to support the MEG source analyses (see fMRI-guided MEG source analysis).

Source analysis of MEG data

The spatiotemporal distribution of the neural activity underlying the measured MEG signals was modeled in terms of multiple equivalent current dipoles (see e.g.,Hämäläinen et al. 1993; Scherg 1990). An equivalent dipole is a model for localized electrical activity at the macroscopic scale in the brain. The electrical conductivity distribution of the head was assumed spherically symmetric; in this approximation, radially oriented sources produce no magnetic field outside the head (Grynzpan and Geselowitz 1973). The center of symmetry was chosen for each subject to match the local curvature of the posterior part of the inner surface of the skull.

In the source analysis of MEG data, first an independent multidipole model was determined based on the MEG data alone. Independent MEG source modeling is important because it will reveal sources not found by fMRI. Second, in the fMRI-guided MEG analysis, the results of the independent model were compared with the fMRI foci to evaluate the possibility of alternative MEG inverse solutions. In general, the limited understanding of the detailed relation between electrophysiological neural activity (as measured with MEG) and hemodynamic responses (fMRI) presents a fundamental problem for using fMRI to constrain the MEG solution. Rather than using the fMRI as a rigid constraint for the MEG sources, we emphasize the usefulness of fMRI in selecting more likely inverse solutions among the possible ones.


In the independent MEG analysis, the first step was to localize a single dipole with a nonlinear least-squares fit in the latency range of either the earliest or the most prominent MEG signal in a subset of sensor channels. In the second step, the contribution from this source was projected out of the measured signals using a signal-space projection technique (Tesche et al. 1995;Uusitalo and Ilmoniemi 1997), and another single dipole was fitted to the part of the data that was not explained by the first one. Third, these two dipoles were used as the initial guess for a two-dipole least-squares fit performed to the original nonprojected data. This sequence of a single-dipole fit, signal-space projection, and a multiple-dipole fit was repeated, adding dipoles until the remaining residual variance was of the same order of magnitude as the measurement noise. The explanation rate was expressed in terms of the goodness-of-fit g = 1 − Σ(B measB model)2/Σ(B meas)2, where B meas are the measured signals, and B model the corresponding signals produced by the model; the sums are over the sensor channels. Once the locations of the dipoles were found, the dipole moment over time (source waveform) was determined for each dipole using the entire time period and all sensor channels. If a dipole was close to a focus of fMRI activity, the dipole was labeled anatomically and functionally according to its relative location in the distribution of fMRI foci. This improves the reliability of the identification of functional units because the accuracy of the locations of dipoles becomes worse when the number of simultaneously active sources increases (Supek and Aine 1993), whereas fMRI does not have this limitation.


The fMRI foci were used to evaluate the independent MEG results. Alternative inverse solutions were examined by placing dipoles at the fMRI foci and optimizing the orientation and magnitude of the dipoles for each time instant (a “rotating dipole” model). For example, if a single independently determined MEG dipole is located between two fMRI foci, one can evaluate the hypothesis of two sources by placing dipoles in the fMRI foci. If only one of the two fMRI sources contributes, then the single-dipole model appears appropriate. If both regions show activity in this model, then it is likely that there are two sources contributing to the MEG. This is especially assuring if the goodness-of-fit is prominently improved. One should consider, however, the fMRI-based model more likely also when the single-dipole model explains the measured data equally well. This situation is an example of the nonuniqueness of the MEG inverse problem, and here fMRI data can be particularly useful. If the two fMRI foci are very close or contiguous, then it may be difficult to resolve the activity into two MEG sources and they may have to be represented by a single dipole (Okada 1985). This is often necessary to, in effect, regularize unstable solutions consisting of quadrupole-like combinations of dipoles with large (nonphysiological) opposing amplitudes, the field patterns of which mostly cancel each other.

The fMRI-guided inverse solution also provided a basis for comparison and averaging of source waveforms across subjects. Following the procedures described in the preceding text, the MEG sources were found to lie in common anatomically and functionally (based on fMRI) defined regions across subjects. For each subject, an individual model was derived with nine rotating dipoles. Residual uncertainty in the depth of the dipoles can affect the dipole magnitude (Hari et al. 1988). Therefore the source waveforms were normalized by dividing them by their standard deviation over the prestimulus baseline period before averaging across subjects.

Note that there is a difference in the baselines or referents for MEG and fMRI measures. The result is that the fMRI activations correspond to areas responding to continuous motion only, continuous motion and reversals, or reversals only, whereas MEG activity contains only responses to reversals (with continuous motion as the baseline). Thus the difference in these measures does not eliminate, in fMRI, brain areas that would be MEG sources. In the MEG source analysis, fMRI-based dipoles placed at regions not corresponding to reversals (and thus not contributing to MEG signals) are not a problem, because they are expected to show little activity in the MEG inverse solution (Liu et al. 1998). Because some areas may respond much more to continuous motion than to reversals, the amplitude of fMRI signals in these areas can be large even when the amplitude of the MEG signals (reversal-related activity only) is relatively small.


MEG, independent source analysis

Figure 1 A shows visual evoked magnetic fields in response to the reversal of the direction of motion. Prominent deflections peaked at several latencies between 130 and 400 ms (Fig.1 B). In Fig.2 A , the corresponding spatial distribution of the measured magnetic field is depicted at three latencies. The varying spatiotemporal pattern indicated the presence of multiple underlying generators with different time courses.

Fig. 2.

Maps of the measured and modeled magnetic field at 135, 170, and 260 ms. A: isocontour lines of the extrapolated normal component of the magnetic field are depicted at 7.5-fT increments. Regions where B z is positive are indicated with shading. B: multidipole model of the brain sources underlying the MEG response. Maps depict the field generated by 3 equivalent current dipoles. Arrows represent the location, orientation, and magnitude of dipoles [middle temporal and neighboring areas (MT+), posterior superior temporal sulcus (pSTS), and frontal]. Dipoles were fitted independently to the MEG data only. Goodness-of-fit (g) is indicated for each case, based on a dipole fit to the measured signals in 55 right-hemisphere sensors. C:functional magnetic resonance imaging (fMRI)-constrained dipole model. Maps are based on a solution in which the dipoles were constrained to be at the locations of fMRI foci (V3A, MT+, and pSTS).D: combined fMRI-MEG model. This solution is based on 3 fMRI-constrained dipoles (V3A, MT+, pSTS) and 1 independently obtained MEG dipole (frontal).

An equivalent current dipole model of MEG sources derived on the basis of MEG alone is illustrated in Fig. 2 B (Independent MEG dipole model). The fMRI data (see following text) suggested focal activation limited to restricted regions of cortex, thus supporting the use of equivalent dipoles to model these MEG sources. For the right hemisphere MEG activity of this particular subject, a dipole was first fitted to the early deflection at 130 ms. The location of this dipole was in the lateral surface of the occipito-temporal cortex, presumably corresponding to the human homologue of monkey area MT and areas around it, thus called MT+. With the signal space projection method, the contribution from this dipole was removed and another dipole was fitted at 170 ms. The location of the second dipole was in the vicinity of the precentral sulcus; close to the frontal eye field and related areas. Using these dipoles as an initial estimate, a two-dipole fit was performed in the latency range 130–190 ms. A third dipole was fitted at 260 ms, resulting in a dipole located between the two previous ones. We refer to this dipole location as pSTS because it is in the vicinity of the posterior part of the superior temporal sulcus or the Sylvian fissure. Figure 2, C and D, will be discussed in the subsection on comparisons of independent and fMRI-guided analyses that follows.

Independent MEG multidipole solutions for each subject are shown in Fig. 3. Three to four sources were found in the vicinity of, or between, areas V3A, MT+, pSTS, and frontal cortex in the right hemisphere (as identified with fMRI, see following text). The variation in the number and location of sources across subjects is likely due to differences in cortical geometry and relative amplitudes of neighboring sources. In general, the measured signals were weaker over the left hemisphere, and fewer sources were identified. Dipoles were numbered according to common anatomic regions and waveform features: 1 and 2, occipital; 3, occipito-temporal; 4, parieto-temporal; 5, frontal. Two source regions were found in all subjects in both hemispheres: occipito-temporal (3 in the region of MT+) and frontal (5; Fig. 3 A). The activity in dipole 3 began at ∼130 ms and peaked at 150–180 ms (Fig. 3 B). Overlapping in time with dipole 3, activity in dipole 5 peaked at 170–190 ms. A parieto-temporal dipole (4) was found in the right hemisphere for all subjects, but in the left only for subject 3. For the other two subjects (S1 and S2), the measured magnetic field was smaller over the left than the right hemisphere, and the single left occipito-temporal dipole (3) explained most of the left posterior field patterns (without dipole 4). Activity in dipole 4 began later, ∼200 ms, and peaked at 230–260 ms. Dipole 4 showed long-duration, sustained-like activity in two subjects (S1 and S2), but in subject 3 the activation waveform returned to the baseline level at 350 ms. Forsubject 3, another posterior dipole (2, in the region of V3A) was found in the right hemisphere. In the left hemisphere ofsubject 1, two frontal dipoles (5 and 5a) were obtained. Thus there appears to be multiple frontal sources, exhibiting early and late activities.

Fig. 3.

Source activity for the MEG response to the reversal of motion for 3 subjects. These independent dipole models were derived from the MEG data only. A: locations of the equivalent current dipoles projected onto anatomic MRI slices. White lines indicate the relative locations of the oblique horizontal (top) and 2 sagittal (bottom) slices. Dipoles are labeled 2–5 from posterior to anterior in the left and right hemisphere; bilateral posterior occipital activity was modeled with a single dipole 1. Number of dipoles found was not equal across subjects and hemispheres; occipito-temporal and frontal dipoles were found consistently, however, and always were labeled 3 and 5, respectively. B:amplitude of the dipoles as a function of time. Waveforms of sources in the left (- - -) and right (—) hemispheres are superimposed. Scale bar (2 nAm) is indicated for each source. Vertical lines indicate the latencies 135, 170, and 260 ms. Goodness-of-fit (g) of the dipole model is indicated at the bottom.

Only one source was found in the mesial occipital region (1). It is likely that this dipole represents composite activity of multiple areas in the vicinity of the occipital pole, in particular V1 and V2 in each hemisphere. Our low-contrast stimulus was designed to generate very little activity in these areas, and the signal-to-noise ratio was insufficient for differentiating multiple sources within this region. Furthermore the anatomy is such that the cortical representations of the central visual field in these areas are close to each other with opposing orientations, in the upper and lower banks of the calcarine sulcus and the mesial wall of the left and right occipital lobes. Individual variations in the cortical folding (Belliveau et al. 1991; Brindley 1972) may cause asymmetric amounts of cancellation of the extracranial magnetic field; this may explain why the location of the net equivalent dipole was in the right hemisphere for S1 and S2 but in the left forS3. This may be also reflected in the relatively large variability of the source waveforms across subjects (peak latencies 170–260 ms).

Foci of fMRI activity

Figure 4 shows fMRI activation in response to the visual motion stimulation. A consistent pattern of three clusters of activity per hemisphere was seen. The most prominent activity was found in the occipito-temporal region corresponding to the MT+ complex. The second region was posterior to MT+, probably corresponding to area V3A. The third region of fMRI activity was anterior to MT+ near the posterior end of the Sylvian fissure or the superior temporal sulcus (pSTS). Talairach coordinates (Talairach and Tournoux 1988) of the voxels with largest significance level of activity within regional clusters of fMRI activation are listed in Table 1 (see also Fig. 5 A). In addition to MT+, V3A, and pSTS, less consistent fMRI activity was found in the V1/V2 region. No significant fMRI activity was found in the vicinity of the independent frontal dipoles. To identify this region in fMRI, we ran an additional experiment on the same subjects in which the visual motion stimulus was compared with fixation alone in the absence of a static pattern. Even under these conditions, no significant frontal fMRI activation was found. Incidental signal loss could be due to magnetic susceptibility effects in fMRI, but no such artifact was seen in the region of the frontal MEG dipoles. The locations of frontal activity, based on the independent MEG dipole model, are included in Table 1.

Fig. 4.

Depiction of the 3 consistent fMRI activation regions (subject 1). Locations of statistically significant (P < 0.01) differential responses to visual motion stimulation (vs. stationary pattern) are shown in white, superimposed on anatomic MR images. Lines on the schematic head depict the locations of the 2 horizontal slices (6 and 10); L, left; R, right. Three prominent foci in each hemisphere are labeled V3A, MT+, and pSTS.

View this table:
Table 1.

Talairach coordinates of activated brain regions

Fig. 5.

Combining data across subjects. A: locations of the fMRI foci of V3A, MT+, and pSTS (encircled) for 3 subjects, as well as frontal independent MEG dipoles, and mesial occipitally placed dipoles (V1/V2), all transformed to Talairach space. B: source amplitudes as a function of time, averaged over the 3 subjects. Source waveforms were obtained from a 9-rotating-dipole model in each subject, using the locations depicted in A. Horizontal lines indicate the 0 level of the dipole amplitudes. Vertical scale is in arbitrary units because before averaging, the source waveforms of each subject were scaled to equalize the prestimulus baseline level.

The distances between the locations of independently obtained MEG dipoles and the fMRI foci are given in Table2. In general, the independent MEG dipoles were near or in between the fMRI foci, with the exception of the frontal dipoles. The best correspondence between independent MEG dipoles and the fMRI foci was found for MT+. The distance from dipole 3 to MT+ in fMRI was 8–20 mm for all subjects. The distances from dipole 2 to V3A and from 4 to pSTS were 11–22 mm. These distances are similar to those in previous reports of MEG and fMRI comparisons (Beisteiner et al. 1995; Morioka et al. 1995; Sanders et al. 1996). The frontal dipoles 5 and 5a were clearly not associated with any fMRI foci (all distances >40 mm).

View this table:
Table 2.

Distances between independent MEG dipoles and fMRI foci

Comparisons of independent and fMRI-guided MEG source analyses

To overcome uncertainties in locating the sources of MEG data, we used fMRI from the same individual subjects to help in the interpretation of the MEG recordings. In the following, we will present two related analyses which differ in the way fMRI information was incorporated into the inverse solution. First, we will present an example in which fMRI foci were compared with two different independent MEG solutions to evaluate whether one was better in accordance with the fMRI data than the other. Second, in the fMRI-constrained analysis (next section), the fMRI foci themselves served as the locations of the equivalent dipoles.

The measured field pattern at 260 ms in Fig. 2 A provided an example of how the additional information from fMRI might prove helpful in weighting the feasibility of different, but almost equally good independent MEG solutions. This field pattern was explained by a single dipole in the independent MEG analysis (Fig. 2 B). In the fMRI-constrained solution, activity was distributed across dipoles placed at the fMRI foci of MT+ and pSTS (and also V3A), which were located around the single independent MEG dipole (Fig. 2 C). This division of source activity provides an example of different solutions of the MEG inverse problem both of which are reasonable. The independent solution explained the measured field with a simpler model (1 dipole, rather than the 2 or 3 in the fMRI-constrained model). On the other hand, assuming a correlation between the hemodynamic and electrophysiological activities, we tend to favor the fMRI-constrained model activity over the independent solution.

It was evident that the frontal activity modeled by a dipole around 170 ms in the independent solution had no correspondence in the fMRI-constrained solution (Fig. 2 C). Although MT+ dipole was similar for both solutions, the inadequacy of the fMRI-constrained solution was reflected in the lower goodness-of-fit (67 vs. 77%) despite the large amplitude assigned to the pSTS dipole. This missing (frontal) fMRI source illustrates the importance of making sure that the model is extensive enough to explain all the measured data. Figure2 D shows a model in which the set of fMRI-based dipoles is augmented with the frontal dipoles. Weak MEG signals over posterior regions are not illustrated (see next section).

Across-subject averaging of MEG source waveforms

For the purpose of comparing and averaging source waveforms across subjects, we performed a combined independent and fMRI-constrained dipole analysis. Nine rotating dipoles were used to model the MEG responses. For each individual subject, dipoles were placed in the fMRI foci of V3A, MT+, and pSTS in both hemispheres. Frontal dipoles from the independent MEG analysis were included bilaterally. In addition, one dipole was included mesially near the occipital pole to account for activity in the V1/V2 region. These source locations were compared in Talairach space across subjects and found to cluster into seven posterior sensory regions and three frontal areas that were more variable (Fig. 5 A).

Figure 5 B shows the MEG source activity averaged over the three subjects. Comparison of the source waveforms revealed characteristic temporal patterns of activity in these cortical regions. The activity appeared to start almost simultaneously at ∼130 ms bilaterally in MT+ and in the right hemisphere frontal region. There was a prominent transient deflection in MT+ peaking at ∼170 ms, and a second peak occurring at 260–280 ms. The frontal activity peaked at 180 ms in the right hemisphere, similar to MT+, and then again at 250–270 ms; in the left, the broad peak occurred at ∼300 ms. Note, however, that the independent MEG analysis shown in Fig. 3 suggests that these two peaks may originate from different neural populations. The activity in the V1/V2 and V3A dipoles peaked at 220–240 ms, with V3A peaking slightly later. Right-hemisphere pSTS peaked later, at 260 ms. Both pSTS and frontal sources exhibited more sustained activity beyond 300 ms. For V3A and pSTS, the activity was weaker in the left hemisphere than the right; this is consistent with independent MEG analysis, in which no dipoles could be determined for these regions forsubjects S1 and S2. These across-subject averaged waveforms from the fMRI guided dipole model emphasized the characteristic features observed in the single-subject independent MEG analysis (cf. Fig. 3). Different time courses in the electrophysiological response across cortical areas to the same stimulus suggest the possibility that fMRI activation patterns may vary across brain regions as well. Sunaert, Orban, and colleagues (unpublished results) have reported differing fMRI time courses in different cortical areas to visual motion stimuli.


Analyses of our combined MEG and fMRI experiments revealed a dynamic pattern of activity in a number of cortical areas (MT+, pSTS, frontal, V1/V2, and V3A), which represent a subset of regions known to be related to processing visual motion (Cheng et al. 1995; Dupont et al. 1994; Tootell et al. 1995b, 1997; Zeki et al. 1991). Our approach provides identification of areas that are likely to comprise a network for processing sudden changes in the direction of motion about which relatively little is known in humans or animals. The fMRI data suggested focal activation limited to restricted regions of cortex, thus supporting the use of equivalent dipoles for modeling MEG sources. The millisecond resolution of MEG indicated the temporal characteristics of activity in all the areas and the relative timing between the areas in this network. Our findings suggest two aspects of the processing in this system. First, the initial phasic process in MT, peaking at 150–180 ms, preceded activity in V3A, V1/V2, and pSTS but coincided with frontal activity. There was also a later activation in MT+ that followed the peak activity in V3a and V1/V2. Second, pSTS and frontal regions showed long-duration activity continuing over the 200- to 400-ms latency range, consistent with input from multiple areas and interactive processes over time (see Fig. 5).

Spatiotemporal distribution of activity


Although motion reversal evoked scalp potentials have been recorded for a long time (Clarke 1973, 1974; MacKay and Rietvelt 1968), the brain areas generating these responses have not been determined. In the present study, area MT+ showed a good correspondence between the locations of the independently fitted equivalent dipoles and the fMRI foci. The one previous PET study (Cornette et al. 1998) did not find significant responses in this occipito-temporal area specifically to changes in the direction of motion. However, MT+ has been established to be sensitive to a variety of visual motion stimuli in many previous studies using PET or fMRI (Cheng et al. 1995, Corbetta et al. 1991; Dupont et al. 1994, Tootell et al. 1995b; Watson et al. 1993; Zeki et al. 1991), EEG or MEG (Anderson et al. 1996;Kaneoke at al. 1997; Patzwahl et al. 1996; Probst et al. 1993; Uusitalo et al. 1997), transcranial magnetic stimulation (Beckers and Homberg 1992), and patients with lesions (Vaina 1994; Zihl et al. 1983). There is some evidence of multiple areas within this region (hence MT+) that respond to different aspects of visual motion (deJong et al. 1994;Tootell et al. 1996; Zeki et al. 1993), possibly including human homologues of areas MT and MST. The response evoked by a reversal of motion direction is thought to be mediated by direction-selective neurons (Clarke 1974;Cornette et al. 1998). In monkeys, MT neurons are direction selective, and MST cells show characteristic transient and sustained firing patterns to more complex motion stimuli, like optic flow (e.g., Duffy and Wurtz 1997). Thus MT and MST seem likely candidates for processing changes in the direction of motion. However, there have not been monkey studies to determine the nature of the mechanism for detecting sudden changes in the direction of motion. Our MEG data indicate, for the first time, transient activity in this MT+ region in response to changes in the direction of motion. In addition to this candidate area, other regions were found to respond to the motion reversal stimulus.

The onset latency of MT+ activation was ∼130 ms and the first peak was at 150–80 ms. Previous EEG and MEG studies of motion onset (not reversal) have revealed similar peak latencies in MT+ as well as an earlier response localized to V1 (Anderson et al. 1996;Probst et al. 1993; Uusitalo et al. 1997). Response to motion onset of faster moving stimuli has been shown to occur at ∼50 ms, presumably via a more direct pathway to MT+, bypassing V1 (ffytche et al. 1995). Responses to a low-contrast, slowly moving stimulus like ours is expected to take longer (ffytche et al. 1995). However, because the initial responses to visual stimuli can be small relative to later activity (e.g., ffytche et al. 1995), it is possible that earlier activity was not detected in our study against the baseline of continuous motion.


Activity in the parieto-temporal region (superior temporal sulcus or posterior end of the Sylvian fissure) also was observed in both MEG and fMRI. Previously, visual-motion-related activity in this region has been found in PET and fMRI studies (Bonda et al. 1996;Cheng et al. 1995; Dupont et al. 1994;Puce et al. 1998). The characteristic time behavior in pSTS was a broad response, peaking at 200–400 ms. To our knowledge, this is the first report of the electrophysiological response waveform of this area in humans. The long duration of the pSTS response found in this study suggests the possibility that pSTS is involved in integrating information from multiple input areas. In the monkey, this general region contains polysensory neurons (superior temporal polysensory area, STP), responding to visual, auditory, and somatosensory stimuli (Bruce et al. 1981). Thus pSTS could be responsive to motion in multiple sensory modalities. However, recent studies employing auditory motion stimuli have not found activation in this region (Griffiths et al. 1998;Howard et al. 1996; Mäkelä and McEvoy 1996).

Another hypothesis is that this area is related to processing vestibular information. In PET studies, a region similar to pSTS has been activated by vestibular stimulation (Bottini et al. 1994; Friberg et al. 1985). This region has been suggested to be a homologue of monkey parieto-insular vestibular cortex (PIVC) (Dupont et al. 1994), which contains cells responsive to visual motion as well (Grüsser et al. 1990). Previously, event-related scalp potential evidence of the interaction between visual motion and vestibular activity has been found in a study showing that the perception of self-motion reduces visual motion-onset-evoked potentials (Probst and Wist 1990). However, the source locations of those responses were not determined. The reversal between expanding versus contracting motion is essentially an optic flow stimulus, similar to visual input when an observer is moving forward and backward. Although our central-visual-field stimulus did not optimally evoke a perception of self-motion, it is reasonable to assume that this visual information is fed into other mechanisms related to the control of body position and movement, i.e., motor and vestibular systems.


A prominent transient MEG response was found to originate in the frontal lobe. Previously, MEG responses to visual motion onset, sustained motion, and speed modulation have been found in a similar frontal region (Lounasmaa et al. 1985; Uusitalo et al. 1997). We saw no fMRI activity in this region, suggesting that it may be activated equally well by the continuous motion (with reversals) and the stationary baseline condition. This is in accordance with PET studies that have indicated increased activity in FEF during mere fixation (Petit et al. 1995). The independent MEG analysis suggested the presence of more than one frontal source in the vicinity of precentral sulcus, some being close to the human frontal eye field (Paus 1996).

The long duration of the frontal (as well as pSTS) activation could reflect a combination of inputs from multiple regions, like MT+ and pSTS. In monkeys, there are projections from MT, MST, and STP to FEF (Boussaoud et al. 1990; Maioli et al. 1983; Seltzer and Pandya 1989; see alsoFelleman and VanEssen 1991; Schall et al. 1995), and visual responses have been recorded in FEF with a large range of peak latencies (86–300 ms) (Schall 1991).

A sudden change in the direction of motion is a salient visual event that might activate the frontal eye field even though the subject does not perform an eye movement. The onset latency of the frontal response (130–150 ms) is similar to the typical latency of making a saccade to a target stimulus. The passive nature of the experimental paradigm may explain why significant activity was not found in the parietal lobe. We did not use task factors that typically would involve parietal cortex (Colby et al. 1996; Corbetta 1998;Snyder et al. 1997). The experimental paradigm did not include eye or body movements and there was no attention task related to the stimuli (only maintaining fixation). Orienting and selective attention would be expected to modulate the activity in this network and involve other areas (e.g., parietal cortices).

V3A AND V1/V2.

Prominent fMRI activity was seen in the area corresponding to human V3A (Tootell et al. 1997) and posterior occipital regions (likely to include at least V1 and V2), but the independent MEG fit suggested relatively little MEG activity there. The posterior occipital areas are known to give large-amplitude MEG responses to pattern onset stimuli (e.g., Ahlfors et al. 1992; Aine et al. 1996). In our experimental design, we intended to selectively diminish responses in these areas by using a low-contrast motion reversal stimulus. The relative amplitudes of V1/V2 and MT+ activity may be partly specific to the single spatial frequency used in this study. The generality of this relationship needs to be explored with other spatial and temporal frequencies. The salient reversal of direction may have automatically affected the arousal or attentional state of the subject. This could further enhance the responses in MT+ compared with V1/V2 (O'Craven et al. 1997). However, the subjects were not asked to attend to any feature of the motion. The source waveforms of posterior occipital dipoles showed activity at 200–260 ms. The long latency of the responses in these areas, which are known to be “early” in the hierarchy of motion processing pathways, suggests that this activity reflects feedback input from other areas. More generally, activity patterns within and between these and the other visual motion areas identified in this study argue for reentrant processing over time.

Cortical network activity for processing visual motion

Multiple motion-sensitive areas demonstrated temporally overlapping but different characteristic patterns of activity in response to the sudden changes in visual motion. The data suggest two classes of response (see Fig. 5). Some regions (MT+, V1/V2) exhibited transient activity, whereas others (pSTS, frontal) displayed longer duration responses. These areas differed in their peak latencies and rise time, following a characteristic temporal order of MT+ and frontal, V1/V2, V3A, and pSTS.

The extant literature clearly suggests that the brain uses the same cortical areas in a wide variety of information processing. Modulation of temporal interactions between brain areas subserves mental function. Our data provide knowledge of the location and temporal patterns of human brain processing of changes in the direction of motion. The application of correlation techniques to the source waveforms holds promise for exploring the dynamics of network processing in the future; inclusion of temporal information will extend existing static computational models derived from PET or fMRI data (Buchel and Friston 1997; McIntosh et al. 1994). With the combined MEG-fMRI approach it is possible to obtain, in humans, information similar to multipass intracranial experiments in animals (Nowak and Bullier 1997; Schmolesky et al. 1998; Schroeder et al. 1998), that is, to identify activity in a network of areas, and to measure the relative timing of each.


We thank J. Foxe for useful discussions and B. Kennedy for technical assistance.

This study was supported by the Human Frontier Science Program; National Institutes of Health Grants NS-27900, NS-37462, MH-DA52176, MH-DA09972 (Human Brain Project), EY-07980, and RR-13609; the Whitaker Foundation; the Paavo Nurmi Foundation; Helsinki University Central Hospital Research Funds TYH 8102 and TYH 9102; and the Academy of Finland. This study was conducted during the tenure of an American Heart Association Established Investigator Award to J. W. Belliveau.


  • Address for reprint requests: S. P. Ahlfors, Rose F. Kennedy Center, Room 915, Albert Einstein College of Medicine, 1300 Morris Park Ave., Bronx, NY 10461.

  • The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.


View Abstract