When observing a particular movement a subset of movement-selective visual and visuomotor neurons are active in the observer's brain, forming a representation of the observed movement. Similarly, when executing a movement a subset of movement-selective motor and visuomotor neurons are active, forming a representation of the executed movement. In this study we used an fMRI-adaptation protocol to assess cortical response selectivity to observed and executed movements simultaneously. Subjects freely played the rock–paper–scissors game against a videotaped opponent, sometimes repeatedly observing or executing the same movement on subsequent trials. Numerous brain areas exhibited adaptation (repetition suppression) during either repeated observations or repeated executions of the same movement. A subset of areas exhibited an overlap of both effects, containing neurons with selective responses for both executed and observed movements. We describe the function of these movement representation areas in the context of the human mirror system, which is expected to respond selectively to both observed and executed movements.
The cortical motor system is commonly described as a hierarchically organized set of cortical areas where the task of planning and executing a movement is broken down into different components, which may range from sets of abstract motor goals to specific movement kinematics and dynamics variables. Particular executed movements are represented by specific subsets of cortical neurons that encode the movement's spatial goal (Graziano et al. 2002), directions in space (Georgopoulos et al. 1986), muscle forces (Kakei et al. 1999), velocities (Reina et al. 2001), and joint angles (Scott et al. 1997). Specific motor neurons are thus active during the execution of some movements and not others in a movement-selective manner, forming a unique neural representation of the executed movement.
In a somewhat analogous manner the cortical visual system is also composed of a set of hierarchically organized areas that respond to different components of visual stimuli ranging from simple edge orientations in early visual areas (Hubel and Wiesel 1968) to complex objects (Malach et al. 1995), body parts (Downing et al. 2001), biological motion (Grezes et al. 2001; Grossman et al. 2000), and observed movements (Hamilton and Grafton 2006; Kable and Chatterjee 2006; Shmuelof and Zohary 2005, 2006) in higher visual areas. Specific visual neurons are hypothesized to be active during the observation of some movements and not others in a movement-selective manner, forming a neural representation of the observed movement.
Mirror neurons are a unique class of visuomotor neurons that are active both during the execution of a movement and during the observation of the same movement (Gallese et al. 1996). Individual mirror neurons respond selectively during the observation and execution of specific movements and are believed to act as integration mechanisms for visual and motor representations of these movements (Rizzolatti et al. 2001). We thus expect mirror neurons to be localized only in cortical areas that exhibit selective responses to movements, whether observed or executed.
Functional magnetic resonance imaging (fMRI) has been used in conjunction with adaptation to assess the selectivity of neural responses in the human brain. This methodology is based on single-unit electrophysiology studies with nonhuman primates that have shown that sensory neurons selective for a particular attribute (e.g., movement direction, object identity) commonly reduce their firing rate when their preferred stimulus is presented repeatedly. fMRI adaptation identifies cortical areas that exhibit smaller responses during trials in which a certain stimulus attribute is repeated (“repeats”) compared with trials in which that attribute is changed (“nonrepeats”). Areas exhibiting smaller responses to repeats than to nonrepeats are interpreted as containing neurons selective for the stimulus attribute being repeated (Avidan et al. 2002; Boynton and Finney 2003; Desimone 1996; Engel and Furmanski 2001; Fang et al. 2005; Gardner et al. 2005; Grill-Spector 2006; Grill-Spector and Malach 2001; Henson and Rugg 2003; Huk and Heeger 2002; Huk et al. 2001; Kohn and Movshon 2003; Kourtzi and Kanwisher 2001; Kourtzi et al. 2003; Krekelberg et al. 2006; Larsson et al. 2006; Miller and Desimone 1994; Movshon and Lennie 1979; Neri et al. 2004; Suzuki et al. 1997). Although there are numerous reports of repetition suppression in studies of visual processing, only one previous study has reported it in the motor domain (Karni et al. 1995).
Here we report results from an fMRI experiment using the rock–paper–scissors (RPS) game in which subjects observed and executed long sequences of movements, without being instructed which movements to perform. Observed movements (performed by a virtual opponent) and executed movements (performed by the subject) were sometimes repeated on consecutive trials. We compared the responses to trials in which a movement was repeated (e.g., rock preceded by rock) with trials in which it was not (e.g., rock preceded by paper). Numerous cortical areas responded less to repeats, either observed or executed, than to nonrepeats. Six cortical areas showed an overlap or very close proximity of both effects: anterior inferior frontal sulcus (aIFS), ventral premotor (vPM), anterior intraparietal sulcus (aIPS), superior intraparietal sulcus (sIPS), posterior intraparietal sulcus (pIPS), and lateral occipital (LO). We propose that all six areas contain neurons that respond selectively to observed and executed movements and may contain mirror neurons. Furthermore, our results successfully demonstrate the efficacy of fMRI adaptation for assessing selectivity in motor and somatosensory cortex, which may prove useful for studying motor control in humans.
Thirteen healthy subjects (six males) between the ages of 21 and 35 yr participated in this study. All subjects had normal or corrected-to-normal vision, provided written informed consent, and were paid for their participation in the study. The Committee on Activities Involving Human Subjects at New York University approved the experimental procedures, which were in compliance with the safety guidelines for MRI research. Twelve subjects participated in the main experiment, which included a high-resolution anatomical volume, one imitation experiment, one movement observation experiment, and four runs of the RPS experiment. Eight of the original subjects and one new subject also participated in an additional experimental session, which included a high-resolution anatomical volume and four runs of the instructed-movement experiment.
Visual stimuli and motor response
Stimuli were presented by an LCD projector and custom optics onto a rear-projection screen in the bore of the MRI scanner. Subjects were supine and viewed the projected stimuli through an angled mirror, which also prevented them from seeing their own hands. A foamed rectangular tray was positioned above each subject's pelvis and attached to the patient bed. Subjects executed motor responses above the tray. The motor responses were videotaped through a window from the console room.
Rock–paper–scissors (RPS) experiment
The RPS experiment was based on the popular game rock–paper–scissors (http://en.wikipedia.org/wiki/Rock,_Paper,_Scissors). Subjects played the game against a videotaped opponent whose right arm and hand were visible in the frame in a first person view (Fig. 1). Subjects were asked to perform their own game movements simultaneously with the observed movements. Each game movement consisted of banging twice on the tray and then performing the rock, paper, or scissors action. Video clips of all three movements contained the same banging segments so that subjects would not be able to predict one movement from another. A trial consisted of 2 s of movement followed by a 1 s intertrial interval (blank screen). Each experimental run/game included 120 trials, 90 randomized game movements, and 30 randomly interleaved blank trials (Fig. 1A). We generated a different movie for each of the games by concatenating the same three movement video clips in different randomized sequences. Subjects played four runs/games in a single scanning session and were videotaped throughout the experiment to extract their movement choices. Each trial was categorized according to the movements observed and executed on the current and the preceding trial (Fig. 1B). Trials were categorized into eight trial type categories, which included four repeat categories and their complementary nonrepeat categories: executed repeats (repeated motor execution of a movement), observed repeats (repeated visual observation of a movement), executed-then-observed cross-modal repeats (execution of a movement followed by observation of the same movement), and observed-then-executed cross-modal repeats (observation of a movement followed by execution of the same movement).
Subjects were told in advance that they would receive extra monetary compensation if they won more trials than they lost over the whole scanning session; thus they were motivated to attend equally to every trial outcome regardless of whether a movement had been repeated. This unique feature of the game addresses an important concern regarding the source of response decreases in adaptation experiments because it has been shown that attention, arousal, and novelty can strongly affect fMRI responses (Huk et al. 2001) and subjects may naturally attend more to novel (nonrepeated) stimuli.
We wanted to compare the results from the RPS experiment described earlier with our own version of the imitation experiments used by others to define the human mirror system (Aziz-Zadeh et al. 2006; Buccino et al. 2004b; Carr et al. 2003; Dapretto et al. 2006; Grezes et al. 2003; Iacoboni et al. 1999; Leslie et al. 2004; Tanaka and Inui 2002; Williams et al. 2006). Subjects were instructed to either passively observe a movie clip of a specific movement (rock, paper, or scissors) being repeated six times, perform a specific movement six times in the dark, or imitate an observed movement six times (simultaneous observation and execution). The block alternation protocol contained 27 epochs in randomly shuffled order: three tasks (imitation, execution, and observation), on each of the three movements (rock, paper, scissors), each three times. Each 21-s-long epoch consisted of 12 s of stimuli and 9 s of blank screen. The final second of each epoch contained a written instruction informing the subject of the task in the following epoch.
Movement observation experiment
We also wanted to compare the RPS experiment with our own variant of movement observation experiments used by others to define the human mirror system (Blakemore et al. 2005; Buccino et al. 2001, 2004a; Haslinger et al. 2005; Iacoboni et al. 2005). The same protocol has also been used to define categorical visual areas (Hasson et al. 2003). Subjects viewed 32 epochs of category-specific movie clips (each 15 s long), which were shuffled randomly into an 8-min movie. Three distinct categories of movie clips were presented: faces under various natural situations (e.g., walking, socializing, eating, etc.), navigation of the camera through different urban areas, and object manipulation (e.g., playing guitar, cooking, answering phone, etc.).
This experiment was intended to test whether motor adaptation (“repetition suppression”) is a robust effect, not only in the complex situation of playing the RPS game, but also when movements were instructed. In this experiment subjects performed the RPS movements in the absence of visual stimulation according to auditory instructions, in a randomly shuffled order. Subjects were asked to close their eyes throughout the scan. An auditory sound track contained a rhythmic ping every second and a verbal instruction to perform a movement every other second. Subjects performed the rock, paper, or scissors movement on the ping after the auditory instruction. In this experiment RPS movements were not preceded by banging twice on the tray and thus enabled shorter scans that lasted only 4.5 min each. Each scan contained 120 trials, which consisted of 90 RPS movements and 30 blanks. Subjects were videotaped throughout the experiment to ensure that they were executing the instructed movements.
Functional and anatomical images of the brain were acquired with a Siemens (Erlangen, Germany) 3T Allegra MRI scanner equipped with a birdcage head coil used for RF transmit and a four-channel phased array of surface receive coils positioned to cover the sides and back of the subject's head (NM-011 transmit and NMSC-021 receive, NOVA Medical, Wakefield, MA). Blood oxygenation level–dependent (BOLD) contrast was obtained using a T2*-sensitive echo planar imaging (EPI) pulse sequence (TR = 1,500 ms for all experiments but instructed-movement experiment where TR = 2,000 ms, TE = 30 ms, flip angle = 75°, 24 slices, 3 × 3 × 3-mm3 voxels, field of view = 192 mm). High-resolution anatomical volumes were acquired with a T1-weighted three-dimensional (3D) MPRAGE pulse sequence (1 × 1 × 1 mm3).
Preprocessing, segmentation, and flattening
fMRI data were processed with the Brain Voyager software package (R. Goebel, Brain Innovation, Maastricht, The Netherlands) and with custom software written in Matlab (The MathWorks, Natick, MA). The first two images of each functional scan were discarded. Preprocessing of functional scans included 3D motion correction and temporal high-pass filtering with a cutoff frequency of six cycles per scan as well as 4-mm Gaussian spatial smoothing. Functional images were aligned with the high-resolution anatomical volume using trilinear interpolation. The anatomical and functional images were transformed to the Talairach coordinate system (Talairach and Tournoux 1988). The cortical surface was reconstructed from the high-resolution anatomical images, separately for each subject. The procedure included segmenting the gray and white matter, inflating the gray matter, cutting along several medial locations including the calcarine sulcus, unfolding the cortical surface, and flattening.
Statistical parameter mapping
We used the Brain Voyager software package to run a standard statistical parameter mapping (SPM) analysis (Friston et al. 1994). In short, we constructed a general linear model (GLM) for the underlying neural response to each experimental condition. For example, the model for our imitation experiment was a matrix that contained a row for each time point, where neural activity was modeled as either “on” = 1 or “off” = 0, and a column for each condition: observe, execute, and imitate. For the RPS experiment, the matrix consisted of 13 columns corresponding to observed repeats and nonrepeats; executed repeats and nonrepeats; observed-then-executed repeats and nonrepeats; executed-then-observed repeats and nonrepeats; win, lose, and draw trials; blank trials; and trials preceded by blanks. The expected neural activity model (each column of the model matrix) was convolved with a canonical HRF to create a model of the expected hemodynamic response (Boynton et al. 1996). We used linear regression to estimate response amplitudes (beta values) for each voxel and each condition, solving an equation of the form y = Ax, where the vector y is the measured time course during one scan, the vector x contains the estimated response amplitudes, and A is the linear model described earlier. A voxel-by-voxel t-test identified brain areas exhibiting significantly different responses to particular conditions. Data were combined across subjects using a random-effects analysis; response amplitudes were computed separately for each subject and then a paired t-test was used to determine significant voxel-by-voxel response differences across all subjects (Friston et al. 1999). Only voxel clusters exceeding 25 mm3 are displayed in the statistical maps. All cortical activation maps presented herein were generated by rendering the random-effects analysis results on a representative individual's cortical surface, except for Fig. 2 showing single-subject activation to illustrate how the regions of interest (ROIs) were defined.
ROI definition and analysis
We wanted to directly assess whether cortical areas exhibiting imitation responses also exhibited movement selectivity. We therefore used the imitation experiment results to define 10 ROIs individually for each subject. ROIs were defined with the Brain Voyager software package using a combination of anatomical and functional criteria. First, we identified single voxels that exhibited significant imitation responses using a maximal false discovery rate (FDR) of 0.01. FDR is a method of correcting for multiple comparisons by controlling for the expected proportion of false positives among suprathreshold voxels (Genovese et al. 2002) rather than for the rate of false positives among all comparisons, as done by the stricter Bonferroni method. Next, we applied a cluster threshold and further considered only voxels that were part of a cluster with a minimum volume of 25 mm3. Finally, we identified particular anatomical landmarks on each subject's anatomy and chose only functionally “active” voxels surrounding these locations while restricting the ROI size to a maximum diameter of 13 mm (Fig. 2).
Because many cortical areas responded during imitation (⇓Fig. 4) our choice of 10 particular ROIs (aIFS, vPM, aIPS, sIPS, pIPS, CMA, M+S, pSF, V, and LO) may seem arbitrary, although this was not the case. We wanted to relate our findings to the literature and thus chose five of these ROIs (aIFS, vPM, aIPS, sIPS, and LO) because they have been commonly reported as areas responding during imitation in mirror system studies (Table 1), although only two of them have been commonly referred to as the human mirror system (vPM and aIPS). We chose four more ROIs (CMA, M+S, PSF, and V) because they exhibited the strongest imitation activity. The final ROI (pIPS) was chosen because we noted that this area exhibited both observed and executed adaptation (Fig. 3). The definitions of all 10 ROIs, however, were statistically independent from the adaptation results (separate data sets). The importance of the ROI analysis was not only to show that some of the independently defined imitation areas exhibited movement selectivity, but also for systematically assessing and quantifying the physiological characteristics of the areas not otherwise evident in parameter mapping analyses. These characteristics included the amplitudes of visual and motor adaptation (Fig. 5), the lack of cross-modal adaptation (Fig. 5), the similarity of responses to all trial outcomes (⇓Fig. 7), the similarity of adaptation effects regardless of trial outcome (Fig. 8), and the similarity of adaptation effects regardless of the hemodynamic response function used to model the data (Supplementary Fig. 2).1
All 10 ROIs were defined in all subjects except for area aIFS, which was defined in only nine of the 12 subjects due to a lack of imitation activity in this area in three of the subjects. Note that the precise boundaries of these ROIs were arbitrary and depended on the choice of anatomical landmarks, functional statistical threshold, and spatial restriction used to define them. We repeated the analysis with ROIs defined using the same anatomical landmarks, but with different statistical thresholds whose FDR values ranged from 0.1 to 0.005, individually for each subject as well as in multisubject averaged maps, and always obtained results comparable to those reported here.
ROI analyses were carried out by averaging across voxels to yield a single response time course for each ROI in each individual subject. We used standard event-related analysis methods to estimate fMRI response amplitudes, as described earlier, separately for each ROI in each subject individually. We then performed paired t-tests to determine which ROIs showed significant response differences across subjects for selected pairs of conditions (e.g., observed repeat vs. nonrepeat) while correcting for multiple comparisons using the Bonferroni method.
When performing this ROI analysis, we assumed a canonical hemodynamic response function (HRF). Because individual subjects may exhibit differences in their HRFs, this approach could lead to a poor fit to the data such that response amplitudes would be misestimated. To evaluate whether this was the case, we estimated the HRF for each subject individually (Supplementary Fig. 1) and recomputed the ROI analysis with these individual subject HRFs instead of the canonical HRF (Supplementary Fig. 2). See supplementary materials on-line for details. We found that the individually defined HRFs were very similar to the canonical HRF (mean r = 0.87 averaged across subjects and ROIs, SE = 0.03; see Supplementary Fig. 1). Results of the ROI analysis were almost identical using the canonical and individually defined HRFs, demonstrating that the visual and motor adaptation effects were robust in all but one ROI. The only exception was LO where the visual adaptation effect was lost (Supplementary Fig. 2).
Rock–paper–scissors adaptation experiment
In the main experiment subjects played the rock–paper–scissors (RPS) game against a virtual opponent (Fig. 1). Subjects observed a video of the opponent's hand and performed their own movements simultaneously while lying in the scanner (Fig. 1A). Because subjects performed the game movements with their right hand, we have focused our analysis on the left hemisphere. Each trial was categorized according to the executed and observed movement on that trial and the one preceding it (Fig. 1B). There were four possible types of repeat trials (Fig. 1B): executed (repeated motor execution of a movement), observed (repeated visual observation of a movement), observed-then-executed (observation of a movement followed by execution of the same movement), and executed-then-observed (execution of a movement followed by observation of the same movement). Every trial was labeled as either a repeat or nonrepeat in each of the categories. The irregular ordering and large number of trials collected from each subject enabled us to tease apart the fMRI responses associated with each trial type.
Some cortical regions exhibited smaller responses to observed movement repeats than to nonrepeats, and other cortical areas exhibited smaller responses to executed movement repeats than to nonrepeats (Fig. 3, green and orange, respectively). Five areas exhibited an overlap of both effects (Fig. 3, yellow): they were located in anterior inferior frontal sulcus (aIFS), ventral premotor (vPM) cortex, anterior intraparietal sulcus (aIPS), superior intraparietal sulcus (sIPS), and posterior intraparietal sulcus (pIPS). In a sixth area, the lateral occipital (LO) cortex, the effects were adjacent but did not overlap. We did not find any cortical areas exhibiting smaller or larger responses to cross-modal repeats than to nonrepeats, neither for observed-then-executed nor executed-then-observed repeats even at very low statistical thresholds.
These findings of repetition suppression are consistent with three recent studies that used fMRI adaptation protocols to assess selective neural responses to repeatedly observed movements (Hamilton and Grafton 2006; Shmuelof and Zohary 2005, 2006). All of these reported movement-selective repetition suppression in aIPS, whereas some also reported repetition suppression in other locations throughout the intraparietal sulcus (IPS) as well as weak (but not statistically significant) repetition suppression in vPM (Table 1). In this study areas aIPS, sIPS, pIPS, and vPM exhibited repetition suppression not only to repeatedly observed movements, but also to repeatedly executed movements. These results are also consistent with a large body of literature elucidating the roles of these areas in movement representation and planning (including the movement's goal, speed, and spatial endpoint), visuomotor transformations, and coordination of grasping movements (Castiello 2005; Culham et al. 2006; Grefkes and Fink 2005; Orban et al. 2006; Tunik et al. 2005).
Perhaps surprisingly, we did not find any repetition suppression to repeatedly observed movements in the superior temporal sulcus (STS), an area commonly considered as important for the perception of movement (Puce and Perrett 2003) and often considered the “visual component” of the human mirror system (Iacoboni 2005). In agreement with our results, three previous studies failed to find visual adaptation to repeated movement observation in the STS (Hamilton and Grafton 2006; Shmuelof and Zohary 2005, 2006).
The finding of movement-specific repetition suppression in lateral occipital cortex (LO) may also seem surprising because this is a cortical area not commonly thought of as representing either observed or executed movements. The area we labeled as LO in our figures may include the adjacent extrastriate body area (EBA) (Downing et al. 2001). Both LO and EBA respond robustly in observations of body parts (Saxe et al. 2006), yet this is the first report of LO exhibiting observed movement selectivity (Fig. 3, green). A closely adjacent region also exhibited selectivity for executed movements (Fig. 3, orange), as might be expected from a somatosensory or motor area, but not from a purely visual area. This result, however, is consistent with several studies showing that similar lateral occipital areas respond during motor tasks (Table 2) (Astafiev et al. 2004; Peelen and Downing 2005) perhaps reflecting a mechanism of motor imagery (Astafiev et al. 2004). The LO area we identified also responded more to movement execution than rest in two further experiments (see following text, Figs. 4 B and 9). Because of intersubject anatomical variability, it is difficult to be certain whether the visual and motor adaptation effects were truly segregated into two neighboring areas within LO/EBA or were somewhat overlapping. Segregated yet adjacent movement-selective responses may be interpreted as evidence supporting a recently proposed functional role of this area in separation of self from others (Jeannerod 2004). Overlapping movement selectivity may be interpreted as supporting evidence for the existence of mirror neurons in this area.
Finally, area aIFS is located in lateral prefrontal cortex, which is commonly active in tasks involving working memory (Curtis 2006; Petrides 2005). One might therefore postulate that movement-selective activity in this area during our experiments was due to the working-memory demands of the tasks (e.g., loading observed and executed movements into working memory for comparison during the RPS game).
Imitation and movement observation experiments
To compare our results with those of previous human mirror system studies, we performed a variant of the imitation protocol commonly used to localize candidate human mirror areas (Aziz-Zadeh et al. 2006; Buccino et al. 2004b; Carr et al. 2003; Dapretto et al. 2006; Grezes et al. 2003; Iacoboni et al. 1999; Jackson et al. 2006; Leslie et al. 2004; Tanaka and Inui 2002; Williams et al. 2006). The rationale of the imitation protocol is that mirror neurons respond more strongly during imitation (simultaneous observation and execution) than during observation or execution alone. In this experiment subjects were instructed to observe, execute, or imitate (simultaneously observe and execute) the rock, paper, or scissors movements repeatedly. Many cortical areas responded more strongly during imitation blocks than during observation or execution blocks (Fig. 4A), including all six of the movement-selective areas described earlier (aIFS, vPM, aIPS, sIPS, pIPS, and LO). Note, however, that other cortical areas including the cingulate motor area (CMA), early visual areas (V), primary motor and somatosensory areas (M+S), and posterior sylvian fissure (pSF) also exhibited stronger responses during imitation blocks (Fig. 4A).
Another commonly reported expectation of cortical areas containing mirror neurons is that they respond both when executing movements without any visual stimulation and when observing movements without executing any movements. Using the same imitation protocol we identified areas responding during movement execution (Fig. 4B, orange) and movement observation (Fig. 4B, cyan) compared with blank (rest) blocks. Four of the six cortical areas described earlier (vPM, aIPS, sIPS, and LO) responded during both observation and execution.
We also performed a variant of the passive movement observation protocol used in previous studies with the rationale that mirror neurons respond more strongly to visual stimuli containing movements than to visual stimuli that do not (Blakemore et al. 2005; Buccino et al. 2001, 2004a; Hamilton and Grafton 2006; Haslinger et al. 2005; Iacoboni et al. 2005; Shmuelof and Zohary 2006). In this experiment subjects passively viewed three types of movie clips containing object manipulation movements (e.g., picking up a phone), houses (e.g., downtown urban areas), and faces (e.g., people talking). Many cortical areas exhibited stronger responses to the object manipulation movements than to the other two categories of movies including all six of the movement-selective areas described earlier (Fig. 4C, blue).
Region of interest analysis
We reanalyzed data from the RPS experiment in 10 ROIs: aIFS, vPM, aIPS, sIPS, pIPS, LO, M+S, V, CMA, and pSF (Table 3), which were defined using the imitation experiment. Cortical activity during imitation was widespread, making it difficult for us to isolate ROIs even at high statistical thresholds (e.g., defining where aIPS activity ends and sIPS activity begins). This is a common problem in imitation experiments; thus we selected ROIs individually for each subject using a combination of anatomical landmarks and volume constraints, which limited their location and size (see methods, Fig. 2). The ROIs defined in this way were similar in their location to previously reported foci of activation during imitation and movement observation (compare Table 3 with Tables 1 and 2). We averaged the RPS responses across voxels in each ROI, estimated the response amplitude to each type (observed, execute, cross-model) of repeat and nonrepeat, and then computed the mean and SE across subjects.
The RPS response amplitudes were significantly smaller during both observed repeats (P < 0.05, paired t-test, Bonferroni corrected) and executed repeats (P < 0.01) in five ROIs: aIFS, vPM, aIPS, sIPS, and pIPS (Fig. 5, Supplementary Fig. 2). LO also exhibited smaller response amplitudes during observed and executed repeats, both of which were significant when assessed using a canonical HRF (Fig. 5), whereas only executed repeats were significant when assessed using individually defined HRFs (Supplementary Fig. 2). Responses in areas M+S and CMA were significantly smaller only for executed repeats (P < 0.001), whereas responses in early visual cortex (V) were significantly smaller only for observed repeats (P < 0.005). Area pSF responses were not significantly smaller for either executed repeats (P > 0.2, uncorrected) or observed repeats (P > 0.6, uncorrected). We also compared responses in the same ROIs to cross-modal repeats and nonrepeats for both observed-then-executed and executed-then-observed trials (Fig. 5B). There were no significant differences between cross-modal repeats and nonrepeats in any of the ROIs (P > 0.08, uncorrected).
In a similar manner we also reanalyzed data from the RPS experiment in six ROIs defined using the passive movement observation experiment. We used the same anatomical landmarks, but now used the functional activity during movement observation (compared with observation of houses and faces) to define areas aIFS, vPM, aIPS, sIPS, pIPS, and LO. We found similarly smaller responses during executed and observed movement repeats than during nonrepeats in all six ROIs (P < 0.05, corrected).
Behavioral and outcome analyses
In the RPS game subjects freely chose which movement to perform on every trial and were motivated to attend the trial outcomes (subjects received extra monetary compensation if they won more trials than they lost). This was done to preserve the visuomotor context of the game and keep subjects engaged with the experiment. However, it raises the concern that subjects may have adopted a behavioral strategy while playing the game that might confound the interpretation of the results in either of two ways. First, subjects might have selected their movements based on the movements that they or their opponent performed on the previous trial (e.g., copying the movements performed by their opponent on the previous trial). If so, the resulting repetition suppression might have been generated by adaptation of neural populations involved in making strategic decisions (whether to repeat). Second, subjects may have selected their movements based on the outcome of the previous trial (win, lose, or draw). If so, the resulting repetition suppression might have been generated by neural populations involved in reward representation rather than by neural populations involved in the representation of observed and executed movements.
The distributions of subjects' executed repeats and nonrepeats, however, did not differ significantly from the distributions expected by random movement choices (Table 4). The same was true for executed-then-observed repeats and nonrepeats. In the case of observed-then-executed repeats and nonrepeats, however, we found a slight yet significant deviation from the expected distributions. Subjects tended to avoid executing the previously observed movement (avoided “copying” their opponent) in a way that slightly skewed the actual distributions (Table 4). Although this behavioral strategy does not confound the interpretation of observed and executed repeats and nonrepeats where we found repetition suppression effects, it may confound the interpretation of the cross-modal trials where we did not find any repetition effects. Because we focus our attention on the former rather than the latter we will not further discuss this apparent behavioral strategy.
Moreover, subjects did not significantly alter the frequency of executed movement repeats and nonrepeats based on the outcome of the previous trials. The distributions of trials where movements were repeated or not following a win, lose, or draw trial did not differ significantly from chance (Table 5). Thus there was no evidence that subjects chose their executed movements based on whether they won, lost, or drew on the previous trial (e.g., they did not systematically choose to repeat winning movements).
To determine whether cortical responses to trial outcome (lose, draw, or win) confounded the interpretation of our results we compared responses to winning trials with losing trials. A number of cortical areas responded more to winning trials than to losing trials (none responded more to the opposite comparison), including motor, somatosensory, and early visual areas (Fig. 6A). The largest effect of trial outcome, however, was in subcortical areas including the bilateral striatum (Fig. 6B), which was evident at much higher statistical thresholds (Fig. 6C) and is consistent with previous reports concerning neural representations of reward (O'Doherty 2004). We also performed an ROI analysis to compare responses of all three outcomes within each of the 10 ROIs described earlier (Fig. 7). The effect of outcome on response amplitude was not significant in any of the six movement-selective ROIs [F(2,35) < 1.8, P > 0.18, ANOVA run independently for each ROI]. Finally, to make absolutely certain there was no influence of trial outcome on the response amplitude of repeat trials (preceded by the same movement) and nonrepeat trials (preceded by a different movement) we split the trials according to outcome (win, lose, or draw) on the current trial and compared repeats and nonrepeats within each of the ROIs for each of the three outcomes (Fig. 8). We then performed an ANOVA, independently for each of the ROIs, and separately for observed and executed repeats and nonrepeats. Each ANOVA had two factors: outcome (win, lose, and draw) and repetition (repeat and nonrepeat). Repetition suppression was statistically significant for both observed and executed repeats in all six of the movement-selective ROIs [F(3,144) > 4.8, P < 0.03], but not in any of the control ROIs. Outcome effects were statistically significant in sIPS (observed movements only, P < 0.04) and in pIPS (both observed, P < 0.01 and executed, P < 0.02), but note that there were also clear repetition-suppression effects in these ROIs for all three outcomes. There were no significant interactions between repetition and outcome in any of the ROIs (P > 0.1).
To assess the robustness of motor adaptation we ran a fourth experiment in which subjects performed the RPS movements in randomly shuffled order, according to auditory instructions in the absence of any visual stimulus. Large regions including motor and somatosensory cortex responded during movement execution (Fig. 9A, blue) some of which (Fig. 9A, orange) exhibited motor adaptation (smaller responses to repeated than to nonrepeated movements). Only three ROIs (aIPS, M+S, and vPM) exhibited motor adaptation in both this experiment (Fig. 9B) and the RPS experiment. In fact, compared with area M+S, areas aIFS, vPM, sIPS, pIPS, and LO exhibited rather weak responses during this experiment in contrast to the RPS experiment (compare Figs. 5 and 9B). Note that in this experiment the subjects did not freely choose their movements, they did not compare their executed movements to observed movements, nor did they have any motivation to perform the movements (other than to cooperate with the experimenter's instructions). We attribute the robust difference between the results of the RPS and instructed-movement experiments to these differences in behavioral context, although there were other differences between the two experimental protocols (e.g., number of subjects, timing of each trial).
Adaptation and movement representations
During the RPS game fMRI responses were smaller in several cortical areas when subjects observed or executed the same movement repeatedly (Fig. 3). These smaller responses provide evidence for underlying neural adaptation (repetition suppression) in neurons selective for aspects of the movements being observed or executed. Specifically, the results suggest that there are separate subpopulations of neurons that respond selectively to each of the RPS movements such that each subpopulation adapts when its preferred movement is observed or executed repeatedly. Six cortical areas exhibited overlapping or closely adjacent visual and motor adaptation: aIFS, vPM, aIPS, sIPS, pIPS, and LO (Fig. 3). We propose that these areas contain visual, motor, and visuomotor neurons, which are involved in the neural representation of both observed and executed movements.
Adaptation and mirror neurons
Approximately 20–30% of the neurons in areas F5 and IPL of the monkey have been described as mirror neurons, which are selective for particular movements, whether executed or observed (Rizzolatti and Craighero 2004). The human mirror system areas, however, have been defined using the imitation protocol (Aziz-Zadeh et al. 2006; Buccino et al. 2004b; Carr et al. 2003; Dapretto et al. 2006; Grezes et al. 2003; Iacoboni et al. 1999; Jackson et al. 2006; Leslie et al. 2004; Tanaka and Inui 2002; Williams et al. 2006) and the passive movement observation protocol (Blakemore et al. 2005; Buccino et al. 2001, 2004a; Haslinger et al. 2005; Iacoboni et al. 2005), neither of which assesses the same criteria used in the monkey.
There are two concerns with both of these experimental protocols. The first concern is that large swaths of the cortex including areas not expected to contain mirror neurons (e.g., early visual areas) exhibit strong fMRI responses during imitation and movement observation both in our results (Fig. 4) and in previously published studies (Aziz-Zadeh et al. 2006; Buccino et al. 2001, 2004a,b; Carr et al. 2003; Dapretto et al. 2006; Grezes et al. 2003; Iacoboni et al. 1999, 2005; Leslie et al. 2004; Tanaka and Inui 2002; Williams et al. 2006). This means that the utilized tasks and comparisons are unable to selectively activate only mirror neurons, but rather activate a host of other neurons. In spite of this, previous work has focused on imitation and movement observation responses in only two of the many active areas (vPM and aIPS) because these are assumed by some to be homologous to monkey areas F5 and IPL, respectively.
The second concern is that neither of these protocols assesses whether neurons within the identified cortical areas respond selectively to particular movements, a defining physiological signature of mirror neurons in the monkey (Gallese et al. 1996). Movement selectivity is a crucial feature for mirror neurons' proposed role in action understanding (Rizzolatti et al. 2001) and imitation learning (Buccino et al. 2004b). Mirror neurons are believed to act as a visual to motor integration mechanism where an observed movement representation is mapped onto the motor neurons used by the observer to execute the same movement. Successful mapping must thus be accomplished in a movement-selective manner.
Using the rock–paper–scissors adaptation protocol we were able to address both of these concerns. Our results clearly demonstrate that a small number of cortical areas (aIFS, vPM, aIPS, sIPS, pIPS, and LO) contain neurons selective for both observed and executed movements (Fig. 3). These movement-selective areas overlap well with areas exhibiting increased activity during imitation and movement observation as can be seen in the multisubject averaged maps (Fig. 4), in the ROI analysis results (Fig. 5), and in a comparison with the literature (Table 1). The overlapping adaptation effects, however, were isolated to these particular cortical areas, unlike the widely distributed effects associated with imitation and movement observation. Furthermore, we propose that an overlap of observed and executed movement selectivity is a superior criterion for identifying candidate mirror system areas because it assesses a fundamental feature of mirror neuron physiology as originally described in the monkey: movement selectivity. In contrast, mirror neuron responses to imitation have never been assessed in the monkey. It is possible that mirror neurons do not respond more strongly during imitation than during observation or execution, and that the fMRI responses observed during imitation arise instead from the activity of neurons that are involved in other cognitive processes engaged during imitation (e.g., working memory or attention). Note also that we found some cortical areas (the control ROIs in Fig. 5) that exhibited no movement selectivity but responded strongly during imitation. Our results are thus an important validation that movement-selective neurons are contained in some (but not all) of the cortical areas active during imitation and movement observation.
Despite our claim that the adaptation protocol is a superior way of identifying mirror system areas, we do not claim that it is an exclusive measure of mirror neuron activity. It is possible that the overlapping visual and motor adaptation effects were generated by two separate (possibly intermingled) subpopulations of visual and motor neurons that adapted independently during repeated observation and execution of the movements. Cross-modal adaptation, in trials where the same movement was observed and then executed or executed and then observed, would have provided strong evidence that the visual and motor adaptation was taking place in a single subpopulation of visuomotor mirror neurons. Unfortunately, we did not find any cross-modal adaptation using our protocol (Fig. 5B, Supplemental Fig. 2B). There are two possible reasons for the lack of cross-modal adaptation. First, it is possible that mirror neurons simply do not adapt to cross-modal repeats. The term “neural adaptation” is used very broadly to refer to numerous different physiological mechanisms including presynaptic neurotransmitter depletion, postsynaptic receptor trafficking, postsynaptic receptor desensitization, and hyperpolarization leading to spike frequency adaptation (Zucker and Regehr 2002). One can draw a distinction between adaptation mechanisms where the stimulated neuron becomes less excitable regardless of the source of input (e.g., due to hyperpolarization) and mechanisms where the adaptation results from a reduced efficacy of the synapse(s) relaying the stimulation. The difference between these two classes of mechanisms is that in the former case a neuron would show adaptation on repeated stimulation regardless of the source, whereas in the latter case the neuron would show adaptation only when stimulated by the same set of synapses repeatedly. Thus the involvement of different adaptation mechanisms would lead to different predictions about whether cross-modal adaptation would be expected—it is entirely plausible that mirror neurons exhibit within-modality adaptation, but not cross-modal adaptation. A second possibility is that our protocol is not suitable for capturing the cross-modal adaptation effect. Different forms of neural adaptation take place at different timescales lasting anywhere from a few hundred milliseconds to hours and days. As described earlier within- and across-modality adaptation might be expected to rely on different cellular mechanisms, which may act at different timescales. It is possible that although the rate of movement observation and execution in our RPS experiment was suitable for generating within-modality adaptation, it could have been too rapid or too slow to generate cross-modal adaptation. In any case, because this is a negative result we would not want to base any strong conclusions regarding the physiology of mirror neurons on the lack of cross-modal adaptation.
While keeping this caveat in mind note that three independent protocols expected to elicit mirror neuron activity (imitation, movement observation, and selectivity for observed and executed movements) suggest that mirror neurons may be localized in three new candidate areas (aIFS, sIPS, pIPS) on top of the two cortical areas (vPM and aIPS) commonly proposed as the human mirror system. LO might also be considered a candidate mirror area, although it exhibited closely adjacent rather than overlapping adaptation (Fig. 3), which was somewhat weaker in comparison to the other movement-selective ROIs (Figs. 5, 8, and Supplementary Fig. 2). Regardless of whether any of these areas contains mirror neurons, we have shown that they contain movement-selective neurons, which is by itself a notable feature of their physiology.
Movement representation and context
A movement may be represented at different levels of “abstraction,” ranging from the trajectories and velocities of participating effectors to the goal or the intention of the movement, which may contain a symbolic meaning or emotional information depending on its context. An identical movement may thus be represented by different neural populations in different contexts (Fogassi et al. 2005; Iacoboni et al. 2005; Umiltà et al. 2001).
The visuomotor context of the game and the symbolic meaning of the movements may have modulated the activity of movement-selective neurons, resulting in stronger responses than what would have been observed without the game. In support of this conjecture, we found that only three areas (aIPS, M+S, and vPM) exhibited motor adaptation in both the RPS experiment and the instructed-movement experiment (during which the same movements were executed in a different context).
Similar adaptation protocols may enable the dissociation of selectivity for different movement representation dimensions, in different contexts, such as comparing adaptation to symbolic and nonsymbolic hand movements or to movements with identical kinematics and different dynamics.
In conclusion, using the rock–paper–scissors fMRI adaptation protocol we were able to successfully assess movement selectivity in the motor and visual domains simultaneously and identify six cortical areas containing neurons selective for both observed and executed movements: aIFS, vPM, aIPS, sIPS, pIPS, and LO. The RPS adaptation protocol provides a means for assessing human mirror system activity by movement selectivity, which is closer in rationale to the original description of mirror neuron responses in monkeys and offers better isolation of movement-selective cortical areas. We thus propose that future assessment of the human mirror system, for example, in autistic or schizophrenic individuals (Arbib and Mundhenk 2005; Dapretto et al. 2006; Williams et al. 2006), would benefit by taking advantage of this protocol. Finally, mirror neurons aside, we suggest that assessing movement selectivity using motor repetition suppression may prove useful for the study of motor control in humans.
This work was supported by National Institutes of Health Grants R01-MH-69880 to D. J. Heeger and R01-EY-14030 to N. Rubin, by Human Frontier Science Program grant to U. Hasson, and by the Seaver Foundation.
We thank C. Curtis, B. Knappmeyer, I. Levy, R. Malach, L. Noy, B. Pesaran, and E. Yang for useful discussions and for help in implementing the experiments.
↵1 The online version of this article contains supplemental data.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2007 by the American Physiological Society