Previous neurophysiological studies of the frontal eye field (FEF) in monkeys have focused on its role in saccade target selection and gaze shift control. It has been argued that FEF neurons indicate the locations of behaviorally significant visual stimuli and are not inherently sensitive to specific features of the visual stimuli per se. Here, for the first time, we directly examined single cell responses to simple, two-dimensional shapes and found that shape selectivity exists in a substantial number of FEF cells during a passive fixation task or during the sample, delay (memory), and eye movement periods in a delayed match to sample (DMTS) task. Our data demonstrate that FEF neurons show sensory and mnemonic selectivity for stimulus shape features whether or not they are behaviorally significant for the task at hand. We also investigated the extent and localization of activation in the FEF using a variety of shape stimuli defined by static or dynamic cues employing functional magentic resonance imaging (fMRI) in anesthetized and paralyzed monkeys. Our fMRI results support the electrophysiological findings by showing significant FEF activation for a variety of shape stimuli and cues in the absence of attentional and motor processing. This shape selectivity in FEF is comparable to previous reports in the ventral pathway, inviting a reconsideration of the functional organization of the visual system.
In the distributed, interconnected, and hierarchical organization of the cortical visual system, different areas exhibit selectivity for different aspects, dimensions, or features of objects. Many studies have demonstrated the existence of two general processing streams, a “what” ventral stream more associated with object recognition and a “where” dorsal stream more associated with spatial relations and action execution (for reviews, see Felleman and Van Essen 1991; Hubel and Livingstone 1987; Lennie 1980; Livingstone and Hubel 1988; Ungerleider and Haxby 1994). Whereas the shared early visual areas are tuned for simple, low-level object properties such as orientation and color, the anterior inferior temporal cortex (AIT), the highest level of the ventral pathway, contains neurons selective for more complex object features such as faces or three-dimensional (3-D) objects (Desimone et al. 1984; Tanaka 1996). In contrast, areas in parietal cortex, such as the lateral intraparietal area (LIP), considered late stages of the dorsal pathway, have neurons that are modulated by spatial attention, spatial working memory, and motor planning (Andersen et al. 1997; Chafee and Goldman-Rakic 1998; Colby and Goldberg 1999; Sereno and Amador 2006).
Both visual pathways project to distinct but overlapping regions of frontal cortex (Barbas and Pandya 1991; Goldman-Rakic 1987; Pandya and Barnes 1987; Fuster 1989; Levy and Goldman-Rakic 2000). One contentious subject of debate is the degree to which there is functional specificity for object and spatial processing in different areas of prefrontal cortex. Early lesion work suggested functional parceling in prefrontal cortex in relation to the spatial or nonspatial nature of visual information (Bachevalier and Mishkin 1986; Mishkin and Manning 1978; Passingham 1975). However, several early physiological studies found activity during the delay period in prefrontal cortex when the animals performed feature selective or spatial selective delayed response tasks (Fuster 1973; Fuster et al. 1982; Kubota and Niki 1971; Kubota et al. 1980; Niki 1974; Rosenkilde et al. 1981). The report by Fuster et al. (1982) appears to be the first study of prefrontal cortex that investigated the effects of both feature and spatial delayed match-to-sample (DMTS) tasks in the same population of cells. They reported that they did not see regional segregation of neurons exhibiting spatial selective delay activity and those exhibiting color selective delay activity. More recently, some studies have found specialized zones in prefrontal cortex that are more sensitive to object properties (Wallis et al. 2001) or more sensitive to object properties (shapes or faces) versus spatial properties (e.g., Hoshi et al. 2000; Ó Scalaidhe et al. 1999; Wilson et al. 1993). However, other studies have not found regional dissociations between prefrontal neurons that are sensitive to object properties (shapes or colors) and those sensitive to spatial properties (e.g., Fuster et al. 1982; Rainer et al. 1998a,b; Rao et al. 1997).
The frontal eye field (FEF), located in the prefrontal cortex, is a critical stage of cortical processing for voluntary saccadic eye movements (Guitton et al. 1985). It receives inputs from multiple visual cortical areas (Schall et al. 1995b), including extensive projections from areas in the dorsal pathway such as LIP, as well as MT and 7a (Barbas and Mesulam 1981; Schall et al. 1995b). Although typically considered a continuation of the dorsal pathway, FEF also receives, to a lesser degree, projections from V4 and areas in inferotemporal cortex, including AIT (Schall et al. 1995b). These projections from the ventral pathway are restricted to lateral FEF, the region responsible for generating short saccades. Thus Schall et al. (1995b) noted that convergence from the dorsal and ventral processing streams occurred in lateral FEF but not in medial FEF, the region responsible for generating longer saccades.
Many previous neurophysiological studies of FEF have focused on its role in target selection and gaze shift control (Ferrera et al. 1999; Sato and Schall 2003; Tehovnik et al. 2000) rather than feature sensitivity. Central to this line of research has been the idea that FEF is of major importance in transforming the outcome of visual processing into a command to orient (Schall 2004). Several more recent investigations have shown that although the initial activity of visually responsive neurons in FEF does not discriminate whether a target or distractor stimulus appears in their receptive fields, over time (>100 ms), the activity of FEF neurons evolves to signal the location of the task-defined target (Schall 2002; Schall and Hanes 1993; Schall et al. 1995a). Based on these findings, it has been proposed that FEF neurons form a visual salience map for target selection (Bichot and Schall 1999; Thompson and Bichot 2005; Thompson et al. 2005). In such a map, responses are not sensitive to the specific nature of visual features themselves but rather mark the locations of features that are behaviorally significant. In two studies where FEF neurons appeared to be selective for color and shape, it was argued that such apparent feature selectivity was induced through the formation of arbitrary learned associations between features and their significance within the task for subserving spatial guidance of attention and oculomotor movements (Bichot and Schall 1999; Bichot et al. 1996). Contrary to the general viewpoint outlined in the preceding text, we present here evidence that some FEF neurons are in fact intrinsically sensitive to visual features independent of behavioral significance.
An early qualitative study found little visual feature sensitivity in FEF, reporting no orientation selectivity and only traces (6%) of “color- or motion”-responsive cells (Mohler et al. 1973). More recently, some investigators have found more widespread FEF selectivity to stimulus properties. Xiao, Barborica, and Ferrera (2006) reported evidence suggesting that many FEF neurons (81%) are modulated by stimulus motion, and Ferraina et al. (2000) reported that many FEF visual and visuo-movement neurons (76%) were selective for binocular disparity during either early or late responses. Motion and disparity are both properties widely associated with the dorsal visual stream (Felleman and Van Essen 1991). Given the strong anatomic connections between FEF and dorsal areas, it is therefore not surprising to find FEF neurons selective for these properties. The current study examines FEF selectivity for a property associated with the ventral pathway, namely shape, which has not previously received consideration in a quantitatively rigorous manner.
The fact that FEF neurons carry information relevant to target selection (Thompson and Bichot 2005) need not preclude the possibility of shape feature selectivity in FEF. Across decades of research within different disciplines, including neuropsychology, animal lesion studies, neuroanatomy, neurophysiology, and computational modeling, a consensus has emerged suggesting that shape and other aspects of object processing are largely segregated in the brain from spatial processing. This conceptual framework, which we challenge here, has limited investigators' experimental approaches. There has been no straightforward neurophysiological investigation of shape selectivity in FEF, probably as a consequence of its known dense connections with dorsal stream areas. A variety of recent reports, on the other hand, have shown that several areas in the parietal cortex, considered to be late stages in the dorsal pathway, contain neurons that exhibit sensory, attentional, mnemonic, and intentional 2-D shape and 3-D object selectivities (Konen and Kastner 2008; Murata et al. 2000; Nakamura et al. 2001; Sereno and Amador 2006; Sereno and Maunsell 1998; Sereno et al. 2002; Shikata et al. 1996), suggesting that there are indeed representations of object properties in the dorsal pathway.
In sum, several lines of evidence suggest that FEF may carry shape selectivities typically thought to be associated with the ventral processing stream. First, neurons in LIP, an area with the heaviest projections to FEF, have been shown to have sensory and mnemonic 2-D shape selectivities (Sereno and Maunsell 1998). Second, FEF does also receive projections from ventral stream areas including AIT, a high-level visual area in the “what” pathway specialized for form and object processing (Schall et al. 1995b). Finally, a recent imaging study in anesthetized monkeys has demonstrated 3-D shape encoding in primate FEF (Sereno et al. 2002). Employing a conventional, straightforward experimental design used to study single cell responses in visual areas of the “what” pathway, we directly test, for the first time, sensitivity of FEF neurons to simple, 2-D shapes. We also present fMRI evidence using a variety of shape stimuli to provide information about the localization of shape representations in the FEF.
Single-unit recordings were obtained from two male rhesus monkeys (Macaca mulatta). Eye position was monitored by an infrared system (ISCAN). After implantation of a head post, animals were trained first on DMTS tasks and then on a passive fixation (PF) task. In each animal, before recording, a chamber centered 28 mm anterior to the ear bars and 18 mm lateral to the midline was mounted on the skull. All surgical and experimental procedures were conducted in accordance with the National Institutes of Health Guidelines and reviewed and approved in advance by the University of Texas Medical School at Houston Animal Welfare Committee.
Localization of FEF
Coronal slice scans 1 mm apart were obtained from a 1.5 T MRI scanner with the animal aligned in a stereotaxic frame (Fig. 1). The MRI images (T1 weighted, 3DFSPGR) were compared with atlases of the rhesus monkey brain to identify the location of the FEF (Paxinos et al. 1999; Saleem and Logothetis 2007). In addition, we verified the recording sites as FEF using microstimulation to evoke eye movements (Bruce et al. 1985). Biphasic pulse trains (100 ms in duration, pulse duration: 200 μs, 200 Hz) were delivered through a microelectrode. Sites where stimulation of ≤50 μA elicited substantial eye movements ≥50% of the time, plus regions within 2–3 mm of these locations were considered to be FEF (Bruce et al. 1985; Ferrera et al. 1999). All cells recorded and reported herein met these criteria.
Visual stimuli and behavioral tasks
Stimuli were displayed on a 20-in, 75-Hz CRT monitor with a resolution of 1,152 × 864 pixels, placed 65 cm in front of the animal. For both tasks, each shape stimulus was selected from a set of eight simple geometric forms presented against a black background (see Sereno and Maunsell 1998; see also Fig. 2, top row), all consisting of black (minimal luminance) and white (highest luminance) pixels in equal numbers. The stimulus size ranged from 0.8 to 1.0°, varying with eccentricity that ranged from 2 to 12°.
For each trial, following fixation at a central spot for 280–340 ms within a 1.6° window (full-width), one of the eight shape stimuli was flashed four times in the receptive field of the cell (see Recording procedures for isolation criteria and receptive field mapping). Stimulus duration for each flash was 400 ms with an ISI of 150 ms. The monkey performed no behavioral task other than passive fixation during stimulus presentation and was rewarded at the end of the trial when the fixation spot extinguished if the animal had maintained central fixation throughout the trial. There was one trial for each of the eight possible shapes presented per block (i.e., 8 trials per block). Order of these trials was random within a block; for each new block, the order of shapes was again randomly selected (random block mode). Each shape trial type was repeated at least six times (i.e., 6 blocks of trials).
For this task, three shapes were selected from among the eight possible shapes (2 shapes that elicited the strongest responses and 1 that elicited the weakest response). This selection of three shapes was based on a qualitative estimation from on-line spike histograms of preliminary trials that presented all eight possible shapes in the RF. In each trial, one of these three possible sample stimulus shapes appeared at one of three peripheral locations (RF plus 2 additional locations ∼120° apart). Sample duration was 800 ms, followed by a delay period ranging from 730 to 770 ms in which only the fixation spot remained on the display. After the delay period, the three shape stimuli were presented simultaneously, each at one of the three locations. The animal was required to make an immediate saccade to the target stimulus that matched the sample stimulus (performance level 91% correct). There were two matching subtasks that the animal performed, a shape matching and a location matching subtask (for additional details, see Sereno and Amador 2006). The animal performed each subtask in alternating short blocks of about nine trials. A small cue was briefly presented around the fixation point at the trial onset indicating which DMTS subtask the animal was to perform. For most cells, each shape and location condition was repeated 12 times (median) for each DMTS subtask and presented in a random block mode. Very few neurons showed a significant interaction between shape and subtask (4, 4, and 6% across all neurons recorded during sample, delay, and eye movement periods, respectively, 2-way ANOVA, P < 0.05). That is, although the subtask that the animal performed significantly influenced the responses of FEF neurons, it did not significantly interact with the shape selectivity of the neuron (for an example neuron in LIP, see ⇓⇓⇓⇓⇓⇓⇓⇓Figs. 11 and 13 in Sereno and Amador 2006). We therefore collapsed the data across the two subtasks (shape-matching and location-matching) for our analysis of shape selectivities. We have limited the discussion of subtask effects as they do not alter any findings we report in this paper (see Sereno and Amador 2006 for discussion of DMTS subtask effects in LIP).
Data collection and statistics analysis
All encountered single units in area FEF that were well isolated and stable were recorded extracellularly with a tungsten microelectrode (1–2 MΩ, Frederick Haer). After isolation of each unit, the receptive field was qualitatively mapped, and the most and least effective shape stimuli from among the eight possible shapes were also qualitatively determined. We recorded data from 65 neurons with both tasks, typically DMTS first. For 94 neurons, only data from the PF task were collected, and for 116 neurons, only data from the DMTS task were collected.
We calculated the visual latency of the population of neurons we recorded from as the time of average half-maximal response after stimulus onset. For the 181 neurons recorded in the DMTS task, the visual latency was 61 ms. For the 159 neurons recorded in the PF task, the visual latency was 64 ms. For sake of consistency across analyses and tasks, we used a value of 60 ms to shift the beginning and ending time point for spike windows when calculating average firing rates for sample and delay periods in the DMTS task and sample period in the PF task. For the calculation of average firing rate during the eye movement period, the spike window was from 100 ms before to 100 ms after saccade onset time.
To determine which neurons were selective for shape, we performed an F-test (ANOVA) on the average rate of firing for the eight different shapes in the PF task and for the three different shapes in the DMTS task. To quantify the magnitude of shape selectivity for each neuron, we calculated a shape selectivity index (shape-SI), defined as the Michaelson contrast of the firing rates: (FRmaxShape –FRminShape)/ (FRmaxShape + FRminShape). Similarly, to quantify the spatial selectivity of shape-selective neurons during the sample and eye movement period, we used a spatial SI (spatial-SI), defined as: (FRmaxLocation –FRminLocation)/(FRmaxLocation + FRminLocation). In all statistical tests, a significance criterion level of P < 0.05 was used.
To further examine the time course of shape selectivity and quantify the time point when the visual neural response first started to distinguish between shapes, we calculated a “discrimination latency.” To determine when responses to different shapes differed significantly, we performed a one-way ANOVA on the average firing rates for the different shapes presented at the receptive field across a short-duration, sliding time window. The duration of the sliding window was 40 ms, and it was incremented with a 2-ms step. The time point when statistical significance was first reached (P < 0.05) was defined as the discrimination latency. In this analysis, trials from all shape-selective cells recorded in a given task were combined into one pool to increase the statistical power of the analysis. This increase in power allowed the use of narrower time windows, thereby increasing the temporal resolution of this analysis. Solely for a more strict comparison with discrimination latency, we also used a sliding window method to calculate a second measure of the visual response latency of shape-selective neurons using windows of 40-ms width and 2-ms step. For this calculation of visual latency, we performed a one-way ANOVA between evoked responses to shape stimuli and spontaneous activity (the baseline firing rate after the animals fixated the fixation spot at the center of a black screen but before first stimulus onset in the PF and DMTS task). As in the calculation of discrimination latency, visual latency by this sliding window measure was considered the time point when the P value first dropped <0.05.
The functional imaging study, performed at the Max Plank Institute for Biological Cybernetics in Tübingen, Germany, presents 13 experiments in seven healthy M. mulatta monkeys (monkeys 3–9) weighing 5–13 kg, approved by the local authorities (Regierungspraesidium) in full compliance with the guidelines of the European Community (EUVD 86/609/EEC) for the care and use of laboratory animals. Surgical procedures for custom-made plastic head holders, the protocol for anesthesia during fMRI, and procedures for visual stimulus generation and positioning have been described previously (Logothetis et al. 1999; Sereno et al. 2002).
Three-dimensional objects. A variety of shape stimuli were presented, including 3-D objects, 3-D surfaces, and monkey faces (see Fig. 10). Example objects (rendered with shading) are shown in Fig. 10A. Object surfaces were rendered with white-colored random dots or square texture elements. Objects were ∼8.5° of visual angle. Random dot objects were rotated in depth about the vertical axis while textured objects were translated with small displacements in the x-y plane (mimicking microsaccades that keep the image from fading on the retina). Control stimuli for rotating random dot objects were constructed by repositioning object dots in 3-D space, such that the scrambled stimuli appeared as rotating, ill-defined volumes of independently moving dots rather than rigidly rotating objects with clearly defined surfaces (see Fig. 3A from Sereno et al. 2002 for stimulus construction). Textured object control stimuli were constructed by scrambling the texture gradient (i.e., swapping texture element positions; see Fig. 3B from Sereno et al. 2002 for stimulus construction). Textured control stimuli were also jittered in the x-y plane.
Three-dimensional surface stimuli were defined by dynamic (random dots with motion parallax) and static (shading and contour) shape cues (see Fig. 10B, left, for an example surface defined by shading and contour). Each shape defined by a particular cue was paired with a control stimulus consisting of a scrambled or disrupted cue gradient to diminish or abolish an impression of depth. Surface and control stimuli subtended 23° vertical × 30° horizontal of visual angle. Two-dimensional contour control stimuli were created by randomly swapping contour positions (see Fig. 10B, bottom right). Two different 2-D controls were used for the shaded surfaces. In the first, the luminance gradient was turned into discrete levels, and resulting dark and light regions were shifted slightly relative to each other. These stimuli contain the same local information (luminance range and pattern) as the original surfaces but the pattern of light and dark regions appear as flat regions of pigment rather than shading due to changes in surface depth. The second control was a Fourier phase-scrambled version of the original image (see Fig. 10B, top right). Random dot versions of the surfaces, from a bird's eye view, were rocked back and forth about a vertical axis (125 frames for each direction of rotation, or ∼860 ms). The control stimulus consisted of an extended field of random dots moving in opposite directions every 125 frames with dot speed incrementing or decrementing with each change in direction. The speed of dots in each control stimulus ranged from near 0 to the maximum 2-D speed present in the corresponding surface stimulus (see Fig. 8C from Sereno et al. 2002 for an illustration).
Monkey face stimuli were gray-scale photos (distributed by the Yerkes National Primate Research Center at Emory University) of adult and juvenile monkeys with neutral or emotional expressions (see Fig. 10C). Control stimuli were phase-scrambled versions of the original image. Face and control stimuli subtended 9 × 9° of visual angle.
PRESENTATION CONDITIONS: PROCEDURE AND DESIGN.
Some stimulus types (moving random dot objects, textured objects, and monkey faces) were run repeatedly within a single scan session (day) while others (moving random dot, shaded, and contour surfaces) were alternated with each other in pseudorandom order within and across several scan sessions (days). Ten to 20 scan repetitions were obtained for each scan type. Each scan typically consisted of a block design with 48-s stimulus and control epochs alternating four times. The same eight objects or surfaces were presented in each scan. In the texture, shading, contour, and face scans, 24 stimuli (3 tilts of 8 objects/surfaces or 24 different monkey images) were presented sequentially during an epoch, followed in similar fashion by 24 matched control stimuli. All static shape stimuli (defined by texture, shading, or contour) moved with small displacements in the x-y plane. In the random dot scans, 8 rotating objects or rocking surfaces were presented one after the other during each epoch, followed in similar fashion by 8 motion controls.
MRI DATA COLLECTION/ACQUISITION.
Measurements were made on a vertical 4.7 T scanner having a 40-cm-diam bore (Biospec 47/40v, Bruker Medical, Ettlingen, Germany). The system was equipped with a 50 mT/m actively shielded gradient coil (Bruker, B-GA 26) of 26 cm ID. A custom primate chair and a special transport system were designed for positioning the monkey within the magnet (see Logothetis et al. 1999). A Helmholtz coil allowed homogeneous excitation of most of the brain volume. The signal-to-noise ratio of this system was typically between 80:1 and 120:1. All images were acquired with a 128 × 128 mm field of view. T1-weighted, high-resolution (0.5 mm isotropic) anatomical images were obtained in 8 segments using the 3D-MDEFT (3-D modified driven equilibrium Fourier transform) pulse sequence. Multi-slice fMRI was performed with multi-shot (8 segments), gradient-recalled echo planar imaging (GE-EPI; TR typically 750 ms; TE, 20 ms; FA, 40°). Volumes of 18 slices were collected (1 × 1 × 2 mm voxel size). See Sereno et al. 2002 for more detail on scanning parameters. Horizontal sections were oriented parallel to the Frankfurt zero plane. Navigator scans were used to correct frequency, phase, and intensity fluctuations. For each scan, an autoshim algorithm was used for tuning the linear shim coils.
MRI DATA ANALYSIS.
Data were analyzed using a general linear model (Friston et al. 1995) as implemented in SPM2 and executed in MATLAB. The multi-slice data were first converted into time points, then smoothed (with a 2-mm Gaussian kernel), normalized, and high-pass filtered. Functional data acquired over multiple sessions (days) were co-registered (using the anatomical images obtained from each session) and re-sliced. Linear contrasts were used to identify voxels that responded significantly more to intact objects/surfaces/faces compared with associated scrambled controls (P < 0.001).
Data were collected from 275 cells in FEF. FEF was anatomically localized with an MRI scan prior to the start of recording (Fig. 1) and verified by using microstimulation (<50 μA) to evoke eye movements. We recorded from 159 FEF neurons in two rhesus monkeys during a PF task and 181 neurons during a DMTS task. Sixty-five of these FEF neurons were recorded in both tasks. The average visual latency for the 159 neurons recorded in the PF task was 64 ms and for the 181 neurons recorded in the DMTS task, 61 ms.
Shape selectivity in PF task
Eight different shape stimuli (Fig. 2A, top row) were briefly presented (400-ms duration) within the receptive field while the animal maintained passive fixation. Figure 2A illustrates the spike histogram of an example cell that responded strongly to a pound sign (#), with much weaker response to an upside-down Y, resulting in a shape selectivity index of 0.46 (shape-SI, see methods). About a quarter of the neurons recorded in the PF task (42 of 159, 26%) exhibited significant modulation depending on the shape of the stimulus. The mean shape-SI of the 42 units with significant shape selectivity (Fig. 2B, shown in solid black) was 0.37.
Time course of shape selectivity (PF task)
For the 42 neurons with significant shape selectivity, Fig. 2C shows the average time course of their responses to the most preferred (solid line) and least preferred (dashed line) shape. The shape selectivity of the visual response commenced early and remained present throughout the sample period. We performed a sliding window ANOVA analysis (see methods) to determine when the responses of these shape-selective neurons began to significantly differentiate the eight shapes. As illustrated in Fig. 3 (black curve), the time at which the sliding ANOVA P value curve crossed the P < .05 criterion was 64 ms (black arrow head) following stimulus onset. This discrimination latency for shape-selective neurons was at most 12 ms slower than our measurement of visual latency (64 ms using a half-maximal response time measure across all 159 units or 52 ms using a sliding window measure across shape-selective neurons, see methods).
Shape preferences across the population (PF task)
The group of eight shapes employed in the current study has been previously used in studies exploring shape selectivity of LIP and AIT neurons (Lehky and Sereno 2007; Sereno and Maunsell 1998; Sereno and Amador 2006). These 2-D shapes contained different shape features but were controlled for luminance and size. We were interested in the question of whether one particular shape (perhaps because of some feature it contains) was better at driving all FEF neurons in general or if different shapes served as the most effective stimulus for different neurons. If the former occurred, FEF neurons would appear to be shape selective, but the FEF population would not really be encoding shape. To examine this, we plotted in Fig. 4 the frequency distribution of the eight shapes going from the most preferred rank (top row) to the least preferred rank (bottom row) for all 159 neurons (Fig. 4A; 159 neurons over 8 shapes, i.e., on average 159/8 = 19.9 neurons per shape). As illustrated in the plot, no shape was particularly preferred or disfavored across all FEF neurons. We compared the distributions of eight shapes between all eight response ranks, and only 1 of the 28 (3.6%) comparisons (ranki and rankj, 1 ≤ i ≤ 8, 1 ≤ j ≤ 8, i ≠ j, 28 possible pairs) showed a significant difference (ranks 2 and 6, Kolmogorov-Smirnov goodness-of-fit test). Figure 4B plots the distribution of shapes including only the shape-selective population of 42 neurons. Note that the small number of neurons results in additional noise in the distribution of shape preference (42 neurons over 8 shapes, i.e., on average 42/8 = 5.2 neurons per shape). However, across shape-selective FEF neurons again no shape was statistically preferred. When comparing the distribution of eight shapes between all the response-ranks, only 1 of 28 possible pairs (3.6%) showed a significant difference (ranks 6 and 8, Kolmogorov-Smirnov test). The relatively uniform distribution of each shape across all ranks is what one might expect from a shape-encoding neural population: each shape is represented with approximately equivalent rankings by a comparable number of neurons.
Stability of shape selectivity across blocks (PF task)
Throughout data collection for the PF task, stimuli were presented in a conventional random block mode: each block contained a complete set of eight different shapes with each shape repeated for one trial per block (presented 4 times during each trial). The order of the shapes within a block was randomized from block to block. This strategy guaranteed that no shape could accidentally dominate a particular time period during the recording session as a consequence of randomization across the recording session. Of 159 neurons recorded, the responses of 58 units showed significant changes in response as a function of block number (2-way ANOVA, block main effect) with this change likely due to adaptation effects during blocks subsequent to the first one (cf. Lehky and Sereno 2007). However, of these 58 neurons, only 2 (3%) were shape selective (significant main effect for shape in ANOVA) and also exhibited a significant interaction between shape and block number. This suggests that despite a reduction in response, the shape preferences in shape-selective neurons remained stable across blocks.
Shape selectivity during sample period of DMTS task
Animals were also trained on a DMTS task. A single shape (of 3 possible shapes, at 1 of 3 possible peripheral locations) was presented during a sample period. Following a variable memory delay period during which the animal maintained central fixation, a set of three test shapes at three locations was displayed. The animal was required to make a saccade to the test stimulus that matched the sample stimulus. We recorded from 181 FEF neurons in two rhesus monkeys during the DMTS task. Consistent with findings in the PF task, we found units that were selective for the shape of the stimulus during the sample period. The example cell shown in Fig. 5A responded maximally to an upside-down Y and minimally to a diamond, with a shape-SI of 0.31. Among the 181 neurons recorded during the DMTS task, 12% (21 of 181) showed a significant shape effect with an average shape-SI of 0.20 (Fig. 5B).
Although there was a smaller percentage of significant shape-selective units and smaller mean shape-SI in the DMTS task versus PF task, these differences may be attributed to the reduced number of shapes used (3 shapes in the DMTS task in 3 locations vs. 8 shapes in the PF task in the receptive field). When choosing the three shapes used in the DMTS task, although we tried to pick the shapes that elicited the strongest and weakest responses, the choices were made from qualitative estimation from on-line histograms of a few test trials per shape and not quantitative calculations across many trials. This less than optimal selection of stimuli combined with a smaller number of shape stimuli and less than optimal presentations with respect to the receptive field on 2/3 of the trials (2 of the 3 locations) would have reduced the probability of finding statistically significant differences in responses to different shapes in the DMTS task.
Time course of shape selectivity (DMTS task, sample period)
For neurons with significant shape selectivity during the sample period, Fig. 5C shows the time course of their responses to the most preferred (solid line) and least preferred (dashed line) shape. Similar to the findings in the PF task, the shape selectivity of the visual response commenced early and remained present throughout the sample period. As in the PF data analysis, we performed a sliding window analysis to determine when the responses of these shape-selective neurons began to significantly differentiate the three shapes. As illustrated in Fig. 3 (gray curve), the time at which the sliding ANOVA P value curve crossed the P < 0.05 criterion was 82 ms (gray arrow head) following stimulus onset. This discrimination latency for shape-selective neurons in the DMTS task is only ∼20 ms slower than our calculation of visual latency (61 ms using a half-maximal response time measure across all 159 neurons or 62 ms using a sliding window measure across shape-selective neurons, see methods).
Shape preferences across the population (DMTS task, sample period)
Similar to our analysis of shape preferences in the PF task, we examined the relative response sensitivity to the eight different shapes across the population of 181 neurons recorded during the DMTS task. Figure 6A illustrates the frequency distribution of the 181 FEF neurons for the eight shapes going from the most preferred rank (top row) to the least preferred rank (bottom row). Similar to the results from PF task, no shape was particularly preferred or disfavored by FEF neurons. We compared the distributions of eight shapes between all three ranks (3 pairs), and none showed a significant difference (Kolmogorov-Smirnov goodness-of-fit test). Given that only three shapes were selected for each particular neuron in this task and, thus the number of selections for each shape across the population were unlikely to be exactly equal due to purely random factors, we normalized distributions for each rank by dividing the occurrence of each shape by the corresponding total number of selections and plotted the results in Fig. 6B (selection frequencies of the 8 shapes, from left to right, were 71, 60, 85, 78, 59, 63, 71, and 56). Again, no shape stands out in the distributions. In addition, among the small population of 21 neurons with significant shape selectivity (only 21 neurons across eight shapes, i.e., on average 21/8 = 2.6 neurons per shape), Kolmogorov-Smirnov tests show that the distributions of eight shapes across all three response-ranks were statistically the same. Thus, similar to the results in the PF task, the relatively uniform distribution of each shape across all ranks suggests that the shape selectivity we observed was not due to some particular preference for (or disfavor of) one specific shape across all FEF neurons.
Consistency of shape selectivity across PF and DMTS tasks
Of 159 neurons recorded during the PF task and 181 neurons recorded during the DMTS task, 65 were recorded during both tasks. Of these neurons, five showed significant shape selectivity in both tasks. For all of these shape-selective neurons, the shape selectivity remained consistent across tasks. That is, for each neuron, the response in the PF task to the two preferred shapes of the DMTS task was greater than the response in the PF task to the least preferred shape of the DMTS task.
There were also several neurons that showed significant shape selectivity in one task but were not significantly shape selective in the other task (11 significant in PF task but not in DMTS task and 8 significant in DMTS task but not PF task). Nevertheless the majority of these units again showed consistent shape preferences across the PF and the DMTS tasks (6 of 11 for PF shape-selective units and 4 of 8 for DMTS shape-selective units). The preference order across tasks of the remaining neurons was less consistent. However, the responses of these remaining 9 neurons showed very little changes in firing rate during the task that failed to show significant shape differences. Average firing rates differed by <3 spike/s with a mean difference of 2.0 spike/s across the different shapes. Thus their shape preference order in the task that failed to show significant shape effects was not clear and therefore not meaningful.
It is important to note that differences in the significance of shape-selective responses between the PF and DMTS tasks for individual neurons were likely caused by a number of differences between the tasks including factors that would affect statistical power and which could vary from neuron to neuron (e.g., number of shapes, number of repetitions, placement with respect to the receptive field, correct selection of best and worst shape). In addition, there were important behavioral differences between the PF and DMTS tasks, including differences in covert and overt orienting. Careful manipulation and control of orienting was achieved by the two subtasks in the DMTS task (beyond the scope of this paper). Whereas the subtask manipulation did not significantly affect the shape selectivity of the neurons recorded (see description of DMTS task in methods), factors influencing statistical power and behavioral differences remain a potential confound in direct comparisons across the PF and DMTS tasks.
Shape selectivity during delay period of DMTS task
We also found shape selectivity in the sustained activity during the delay period of the DMTS task. Figure 7A shows the response histogram of an example cell that exhibited significantly different responses during the delay period dependent on the preceding sample shape. Note that although this particular cell responded poorly to the sample stimulus, it developed an increased neural response as well as selectivity for shape (shape-SI = 0.19) during the delay period when no stimulus was present and the animal was required to remember the sample stimulus. Among all 181 units recorded in DMTS task, 25 (14%) showed significant shape-selective responses during the delay period with an average shape-SI of 0.21 (Fig. 7B). For neurons with significant shape selectivity during the delay period, Fig. 7C shows the average time course of their responses to the most preferred (solid line) and least preferred (dashed line) shape.
Shape selectivity during eye movement period of DMTS task
As FEF plays an important role in eye movement, we also examined shape selectivity of the 181 neurons at the time of the eye movement in the DMTS task (±100 ms, see methods). Figure 8, A and B, shows the response histograms of two example cells that exhibited significantly different responses during the eye movement that were dependent on the target shape. Although both neurons responded selectively for shape at the time of the eye movement (shape-SI = 0.32 and 0.41 for 8A and 8B, respectively), they differed in their responsiveness to other aspects of the task (visual and mnemonic). In particular, the neuron in Fig. 8B had very little activity during the other periods of the task except at the time of the eye movement. Among all 181 units recorded in DMTS task, 33 (18%) showed significant shape-selective responses during the eye movement period with an average shape-SI of 0.14 (Fig. 8C). For neurons with significant shape selectivity during the eye movement period, Fig. 8D shows the average time course of their responses to the most preferred (solid line) and least preferred (dashed line) shape.
Shape selectivity across different periods of the DMTS task
SEPARATE POPULATIONS OF SHAPE-SELECTIVE NEURONS ACROSS DIFFERENT TRIAL PERIODS.
Neurons with sample period shape selectivity were generally different neurons than those showing shape selectivity during the delay period. Only 10% of neurons (4 of 42) with significant shape selectivity during either the sample or delay period showed shape selectivity during both periods. This nonoverlap of shape-selective neurons between the sample and delay period was also true for comparisons between other task periods: only 15% of neurons (7 of 47) with significant shape selectivity during either the sample or eye movement period showed shape selectivity during both periods. Similarly, 21% of neurons (10 of 48) with significant shape selectivity during either the delay or eye movement period showed shape selectivity during both periods. And last, only 5% of neurons (3 of 65) with significant shape selectivity during any of the three periods, were shape selective across all three periods. Combining neurons that were shape selective during either the sample, delay, or eye movement periods resulted in 36% (65 of 181) of recorded FEF neurons that showed shape selectivity during the DMTS task.
CONSISTENCY OF SHAPE SELECTIVITY ACROSS PERIODS OF THE DMTS TASK.
As mentioned in the preceding text, only four neurons showed significant shape-selective responses during both the sample and delay periods of the DMTS task. Interestingly, for three of the four neurons, the preference order of the three shapes remained identical in the two periods. For the fourth neuron, the most preferred shape during the sample period also elicited the strongest response during the delay period. Similarly, four of seven neurons showing significant shape selectivity during both sample and eye movement periods had completely matching preference order for the shapes during the two periods, and for the remaining three neurons the best two shapes during the sample remained the best two shapes during the eye movement period. For the 10 neurons that were shape selective during both delay and eye movement periods, 4 preserved the exact preference order, and 3 had the same best two shapes over the two periods. Last, for all three neurons that showed significant shape selectivities across all three periods, the preference orders of the three shapes during the sample, delay, and eye movement periods were exactly the same. In sum, whereas there were only a small number of neurons that showed significant shape selectivity across different periods of the DMTS task, the shape preference orders of these neurons typically remained consistent at different time periods.
SENSORY VERSUS MOTOR DIFFERENCES ACROSS PERIODS.
An early study by Bruce and Goldberg (1985) classified FEF neurons into three categories based on the relative strengths of their visual and movement presaccade activities: visual, movement, or visuomovement. Visual cells had strong visual responses to stimuli in the RF in the absence of saccades and no presaccadic activity associated with an eye movement to the RF in the absence of visual stimulation (movement response). In contrast, movement cells showed strong movement responses but weak or no visual responses. Visuomovement cells exhibited both visual and movement activity. This classification forms a continuum rather than a clear-cut trichotomy with visual and movement cells at the extremes.
We examined whether shape-selective cells differed in the relative strengths of their sensory and motor responsiveness during different periods of the DMTS task. For example, cells showing shape selectivity during the sample period (n = 21) might be more visually dominated, while the largely nonoverlapping population of cells showing shape selectivity during the delay period (n = 25) might have a stronger motor component. Our experiments did not measure neural activity for saccades without a visual target (motor alone) as Bruce and Goldberg (1985) did, hence we used the ratio of average firing rate at the time of the eye movement to the average firing rate during the sample period to assess relative sensory-motor responsiveness in both sample and delay shape-selective neurons. According to the original definition by Bruce and Goldberg (1985), one should expect the ratio of neural response during a visually targeted saccade (eye movement period in our experiment) to neural response during a visual stimulus presentation without saccade (sample period in our experiment) to be highest for movement cells.
Figure 9A illustrates the frequency distribution of the ratios of average firing rates for neurons with significant shape selectivity during the sample period (magenta) compared with neurons with significant shape selectivity during the delay period (green). The responses of neurons represented to the right were more movement-related, showing greater responsiveness during the eye movement period, whereas neurons represented on the left side contained a stronger visual component, showing greater responsiveness during the sample period. Shape-selective neurons from both the sample (magenta) and delay (green) periods showed greater responsiveness at the time of the eye movement. The magenta (median = 1.4) and green (median = 1.5) arrowheads denote the median value of the sample shape-selective and delay shape-selective population, respectively. A Kolmogorov-Smirnov test shows that the two ratio distributions were not significantly different from each other (P = 0.86), indicating that the two populations of shape-selective FEF cells (those during sample period and those during delay period) do not differ substantially in their sensory versus visuomotor responsiveness.
In addition, although most FEF neurons respond more vigorously before a visually guided saccade than to a visual stimulus without eye movement (illustrated in Fig. 9A), their spatial selectivity indices during these two periods remain similar (illustrated in Fig. 9B). Figure 9B plots the spatial-SI during the sample period against the spatial-SI at the time of the eye movement. The spatial-SIs during the sample period (median = 0.09) and those during the eye movement period (median = 0.10) of the 21 neurons with significant shape selectivity in the sample period (magenta dots) did not differ significantly (Kolmogorov-Smirnov test, P = 0.53). In addition, the 25 neurons with significant shape selectivity during the delay period (green dots) did not differ significantly in their spatial selectivity during the sample (median = 0.09) and eye movement periods (median = 0.12) (Kolmogorov-Smirnov test, P = 0.88). Finally, there was no significant difference in the distributions of spatial selectivity ratios (sample period spatial-SI divided by eye movement period spatial-SI) when comparing neurons that were shape selective during the sample period (magenta dots) with neurons that were shape selective during the delay period (green dots; Kolmogorov-Smirnov test, P = 0.94).
fMRI shape responses in FEF
SHAPE RESPONSES WITHOUT ATTENTION OR EYE MOVEMENTS.
A variety of shape stimuli were presented to examine whether shape was represented in the FEF under conditions which excluded attention and eye movement effects in monkeys that were anesthetized and paralyzed, using methods previously described (Sereno et al. 2002). Shape stimuli included 3-D objects defined by structure-from-motion or texture elements (see Fig. 10A), 3-D surfaces defined by structure-from-motion, shading, or surface contour (see Fig. 10B, left), and monkey faces (see Fig. 10C). Control stimuli (e.g., Fig. 10B, right) were used to abolish 2- and 3-D shape and depth.
FMRI ACTIVATION FOR SHAPE IN FEF ACROSS A VARIETY OF SHAPE STIMULI.
Results show bilateral activation in the FEF in at least two monkey subjects for each experimental condition (random-dot rotating objects; textured objects; surfaces defined by dynamic random dots, shading, or contour; and monkey faces). Figure 11 shows six coronal sections through the arcuate sulcus (arranged from left to right, matching posterior to anterior positions in the brain) corresponding to those shown in Fig. 1 depicting recording sites. Significant activation can be seen in the FEF primarily in the inferior limb of the arcuate sulcus where neuronal selectivity for 2-D shape was reported. Figure 11A shows significant activation in monkey 3 for rotating random dot objects versus scrambled controls; Fig. 11B shows activation in monkey 4 for 3-D surfaces defined by motion or shading versus 2-D controls; Fig. 11C shows activation in monkey 5 for 3-D surfaces defined by motion or contour versus 2-D controls; and Fig. 11D shows activation in monkey 5 for monkey faces versus scrambled controls. Activation is bilateral in all cases except in Fig. 11D where a left hemisphere surface coil was used for that experiment (a 2nd monkey presented with face stimuli showed bilateral FEF activation).
EXTENT AND LOCALIZATION OF FMRI ACTIVATION TO SHAPE IN FEF.
Significant activation across different shape stimuli, cues, and animals clustered at the approximate dorsal/ventral location of the FEF near the end of the principal sulcus. Figure 12 presents saggital, coronal, and horizontal views of FEF activation in two monkey subjects. Figure 12A shows significant activation in monkey 3 for rotating random dot objects, whereas Fig. 12B shows activation in monkey 5 for 3-D surfaces defined by motion or contour. The white cross-hairs pinpoint activation in the FEF on the anterior bank of the arcuate sulcus. Each white line specifies the position of one of the two other sections (e.g., the vertical line in the saggital section, which passes through the FEF, indicates the position of the coronal section). FEF activation is present in at least three contiguous, 2 mm horizontal sections in both monkeys. The anterior portion of horizontal slices labeled 1–3 (the location of the arcuate sulcus) roughly corresponds to the same portion of the horizontal slices shown in Figs. 27, 29, and 31 of the combined MRI/histology atlas of the monkey brain by Saleem and Logothetis (2007). Activation in upper slices 2 and 3 overlap region 8A of the FEF, whereas lower slice 1 overlaps region 45/8A. Other regions of activation are seen in the horizontal slices including areas MT, V4, V3A, V3d, LOP (the lateral occipital parietal area), LIP, and some prefrontal activation. The saggital slices show activation extending down the length of the STS (superior temporal sulcus) and also in the IOS (inferior occipital sulcus).
We found that visual responses in a substantial number of FEF neurons showed significant shape selectivity (26–36%, depending on task) when tested with a set of eight, simple, 2-D, high-contrast forms. This shape selectivity was observed under conditions indicating that responses of FEF neurons are tuned directly to shape features in visual stimuli, not just influenced by their behavioral significance. Further, we found reliable fMRI activation in the FEF for a variety of shape conditions under conditions of anesthesia and paralysis, also precluding attention and motor effects as the basis for these activations.
The number of shapes tested in the electrophysiological aspects of our study was limited to eight, mainly due to our goals, design plan, and recording time constraints. Our shape sample pool was not designed to fully characterize the encoding of shape in FEF neurons but rather to establish for the first time the existence of shape selectivity in this area with a single-cell recording approach. First, these eight shapes clearly carried distinct features such as edges, corners, curves, and holes, qualifying themselves as reasonable probes to explore the existence of 2-D shape selectivity in FEF using an electrophysiological approach. Second, these same eight shapes were used in a prior investigation uncovering shape selectivity in LIP (Sereno and Maunsell 1998). Third, these shapes were also used to make a first comparison of shape selectivity across ventral and dorsal stream areas and, even with such a restricted set of stimuli, to show striking differences in the organization and encoding of shape across the two pathways (Lehky and Sereno 2007). And, finally, many early studies important in establishing the role of ventral areas in object processing have used restricted sets of stimuli (e.g., Fuster and Jervey 1981; Moran and Desimone 1985; Schwartz et al. 1983). Hence to be fair and consistent with prior literature, we also describe our physiological findings as shape selective. It is possible that during training and recording sessions in our single-cell recording experiments, neuronal responses of FEF neurons to these particular eight shapes may have been altered or tunings refined. However, the fMRI results in anesthetized animals with novel shape stimuli clearly demonstrate that task training with stimuli is not necessary to elicit shape-selective FEF responses. In further support, we also found consistent fMRI activation for novel shapes in the FEF using a broad range of shape stimuli and cues (objects, surfaces, and faces defined by cues such as motion, texture, and shading). Future studies will be critical both to examine whether and how training may affect shape encoding in FEF neurons and to better understand how these shape selectivities compare with shape selectivities in other cortical regions (cf. Lehky and Sereno 2007; see also Konen and Kastner 2008; Sereno et al. 2002).
Shape selectivity in FEF
Previous electrophysiological investigations have suggested that area FEF shows all the characteristics of a visual salience map (Bichot and Schall 1999; Schall 2004; Thompson and Bichot 2005; Thompson et al. 2005), which indicates the locations of behaviorally significant features (but not the object features per se) to enable motor planning and guidance of orienting behavior. When feature selectivity has been observed (Bichot and Schall 1999; Bichot et al. 1996), it has been interpreted as due to the behavioral significance of the target and target selection strategies acquired through training. In other words, the conventional wisdom has been that FEF is only feature selective if a particular feature correlated consistently with the solution to a behavioral task.
However, the shape-specific modulations we demonstrate cannot be attributed to the behavioral significance of the stimuli or to a target selection strategy. In the PF task, each shape was presented equally often and in random order in the cell's receptive field. There were no differences in reflexive or voluntary orienting across conditions (overt eye movements or covert attentional shifts), and the animal was rewarded equivalently across conditions for maintaining fixation. In the DMTS task, all chosen shapes during the sample period had an equal chance of being the target shape—in a typical recording session, many different target sets were chosen. Nor can the modulations be attributed to historical factors in stimulus presentation, such as priming or perceptual learning, because presentation of all shapes in the stimulus set were equally likely and balanced both in training and within each data collection session. That is, all possible sample shapes were equally relevant across all trials in the DMTS task, yet some FEF neurons showed a greater response during the sample presentation, memory delay, or even eye movement to the same target location for a particular shape.
In sum, in the context of both passive (PF) and active (DMTS) tasks, our electrophysiology studies showed shape selectivity in sensory, delay period, and eye movement period responses of FEF neurons. Likewise, fMRI data from the FEF of anesthetized and paralyzed monkeys also showed shape selectivity and served to reinforce the exclusion of many of the same factors discussed in the preceding text as the basis for the selectivity, including behavioral significance of the stimuli, target selection strategy, eye movement or attentional effects, priming, and perceptual learning.
Using fMRI, we found significant responses in the FEF to a variety of shape stimuli defined by static or dynamic cues in seven different monkeys. These activations were localized to patches in the anterior bank of the arcuate sulcus at a dorsal-ventral position approximately at the level of the caudal end of the principal sulcus. The region of activation extended from the junction of the superior and inferior limbs of the arcuate into the inferior limb (see Figs. 11 and 12). This region overlapped with the area (lateral FEF) reported to receive converging inputs from both dorsal and ventral processing streams (Schall et al. 1995b), including many other regions activated by our 3-D shape stimuli (areas V4, TEO, MT, FST, the ventral bank and fundus of the STS, and LIP). A comparison of Figs. 1 and 11 supports the finding that the localization of shape-selective neurons in the physiology is consistent with the functional imaging results.
Percentage of shape-selective neurons
It is possible that the percentage of shape-selective neurons we observed is an underestimation due to sampling issues. First, while we recorded across all of FEF (45B, 8Ad, 8Av), patterns of afferent projections (Schall et al. 1995b) as well as patterns of fMRI activation (Sereno et al. 2002) suggest that shape selectivity may be localized in certain subregions. We indeed observed some clustering of shape-selective neurons in both physiology and fMRI. Specifically, our results suggest that the location of FEF shape selectivity lies at the dorsal/ventral position of the arcuate that is congruent with the posterior end of the principal sulcus (see Figs. 1, 11, and 12). A second possible factor that could lead to an underestimate of shape selectivity is that, at later stages of visual processing, shape-selective neurons may have preferences tuned along more complex dimensions than the simple patterns used here (Hegde and Van Essen 2007; Tsao et al. 2006). The excellent response to the complex object, surface, and face stimuli used in the fMRI portion of this study supports this notion.
BIAS DUE TO STIMULUS SELECTION AND NUMBER OF CONDITIONS.
The smaller percentage of shape-selective neurons during any individual period (sample, delay, or eye movement) of the DMTS task compared with that in the PF task may arise from the following facts. First, as stated previously, the three shapes for the DMTS task were chosen from among the eight available shapes via qualitative estimation during the experiment without quantitative analysis, increasing the risk of failing to find the most and least preferred stimuli of the set. In addition, the larger number of repetitions per condition in the PF task (4 × 6 = 24 vs. 12) as well as the fact that the stimuli were all presented in the receptive field in the PF task (for DMTS, three locations, only one centered in the receptive field) may have provided additional statistical advantage even if the shapes were all properly selected. Nevertheless given the reduced percentage of shape-selective neurons in the DMTS task in any given trial period, it is also possible that a more demanding task (memory vs. passive fixation) reduced the innate shape sensitivity of FEF neurons.
SHAPE-SELECTIVE UNITS DURING THE SAMPLE AND DELAY PERIODS IN DMTS TASK.
To our knowledge, there has been no study that has examined the overlap or consistency of selectivities across periods of a task. In an early study using a color DMTS task in IT cortex, Fuster and Jervey (1982) commented on the consistency of differential responses to color across the sample and delay, and sample and test periods, but did not present any quantitative analysis of IT cells. In particular, they noted that although some units carried over into the delay period the differential firing to color that was exhibited during the sample period, others did not. With respect to the sample and test, they pointed out that although some cells had a similar response at sample and test, reactions of most cells to the two events often differed considerably. Another tangentially relevant study (Chafee and Goldman-Rakic 1998) examined the percentage overlap of task-related neurons in LIP and FEF with significant spatial selectivity during cue (37/68, 54% for LIP and 48/82, 58% in FEF, respectively) and delay (23/40, 58% in LIP and 25/58, 43% in FEF, respectively) periods, and hinted that there was a much smaller proportion of neurons in either area with significant spatial selectivity during both cue and delay periods. In FEF, we found very few neurons that showed significant shape selectivity in multiple time periods of the trial. For those cells that did, however, the shape selectivities were largely consistent across different time periods.
COMPARISON OF PERCENTAGES ACROSS OTHER VENTRAL AND DORSAL STREAM AREAS.
Previous studies in inferotemporal cortex (IT), the highest stage of the ventral processing pathway have reported varied proportions of feature- and face-selective neurons, ranging from a few percent up to 97%. These discrepancies in percentages have varied for a number of reasons including differences in the characteristics of the visual stimulus probe (ranging from simple colored oriented bars, to more complex objects or scenes or parts of objects or scenes, to faces) and differences in the locations of the recording sites (Desimone et al. 1984; Gross et al. 1972; Perrett et al. 1982; Tanaka et al. 1991). For example, face selective neurons in IT were found to be clustered in localized regions of the superior temporal sulcus (Tanaka et al. 1991; Tsao et al. 2006), and early studies that did not target these regions reported very small percentages of neurons showing face selectivity, as low as 2–3% (Desimone et al. 1984; Gross et al. 1972). In addition to these differences between studies, the criteria for selecting neurons to record also varied. Some studies restrict selected neurons at a more general level, such as selection of visual responsive neurons (Desimone et al. 1984), or neurons with delay activity, whereas others are very specific and highly restrictive, selecting and recording from only those neurons with selectivity for a feature dimension to be studied (e.g., face-selective neurons) (Leopold et al. 2006). Because of these differences in recording procedures, it is not straightforward to make direct comparisons across studies.
However, we have recorded in both LIP (Sereno and Amador 2006; Sereno and Maunsell 1998) and AIT (Lehky and Sereno 2007) with identical task conditions and, to the extent possible, identical recording procedures. We did not notice any obvious clustering of shape-selective neurons in LIP or AIT. We also did not see any difference in the percentage of shape-selective units in AIT (60%) and LIP (57%) in the PF task (Lehky and Sereno 2007). We do see a significantly smaller percentage of shape-selective neurons in FEF (26%) compared with either area. It is likely that there is a weaker representation of shape-selective cells in FEF. It is also possible that there is greater clustering or localization of shape-selective cells in FEF. Both factors would lead to a reduced percentage of shape-selective cells. Additional comparisons of the shape selectivity across FEF, LIP, and AIT (cf. Lehky and Sereno 2007) will be important to see how the encoding of shape in FEF compares to LIP and AIT.
Time course of shape selectivity in FEF
Using a sliding ANOVA procedure, we demonstrated that these shape selectivity effects were occurring at the same time (64 ms, PF task) or shortly after (82 ms, DMTS task) the visual latency of FEF neurons (Fig. 3). This timing is also consistent with the idea that these shape-selective neurons were tuned directly to shape features in visual stimuli, not just influenced by their behavioral significance. Interestingly, although both tasks reached significance (P < 0.05) at this early time period, there was a slower rise in reaching even higher levels of significance in the DMTS task, raising the possibility that there was a second process influencing shape selectivity in this task.
Shape selectivity in FEF: role of attention and task order
For some neurons (n = 65), the animal performed the DMTS task first, then the PF task. Given that only a subset of stimuli were used and attended to in the DMTS task (3 of 8), it is conceivable that performance of the DMTS task increased the responsiveness of the neuron during the subsequent PF task to those three stimuli. If such was the case, one might expect to find some neurons in the PF task for which the three most preferred shapes were the three shapes used in the DMTS. We found no such neuron. Further, for 45 of the neurons (among which 35 neurons were not shape selective for the PF task), the best shape in the PF task was not one of the three chosen stimuli for the DMTS.
There were a few neurons that were not shape selective in the DMTS task that were subsequently shape selective in the PF task (n = 11). For 10 of these 11 units, the most preferred of the subset of shapes used in the DMTS task was not the most preferred shape in the PF task. This suggests that the change in statistical significance of shape selectivity between tasks was due in part to inaccuracies in selection, for each cell, of best (and/or worst) shape when picking the three shapes used in the DMTS task. Thus the change in significance would be attributable to decreasing statistical power to detect shape-related response changes in the DMTS task rather than the DMTS task itself influencing or inducing shape-selective responses in the PF task. Furthermore, if performance of the DMTS task increased the responsiveness of the neuron to the three chosen stimuli, one might expect that there would be a higher percentage of shape-selective neurons in the PF task for sessions when the animal completed the DMTS task before the PF task compared with sessions where the animal performed only the PF task. However, the percentage of shape-selective neurons was the same (26%) in both cases.
Of importance, but not presented in detail here, the animal performed two interleaved DMTS subtasks that varied in whether the animal matched, hence, attended to, the shape or the position of the sample stimulus. Under these carefully controlled attentional conditions, across the whole population of 181 neurons, we found that the shape selectivities of very few neurons were significantly altered by the subtask (4% for both sample and delay periods, 6% for eye movement period). This is similar to what has been previously reported in LIP (see Figs. 11 and 13, Sereno and Amador 2006). That is, although the DMTS subtask may have significantly altered the activity of many FEF neurons (e.g., increased the activity of the neuron in the shape DMTS subtask relative to the location DMTS during the sample period), these attentional subtask effects did not significantly interact with the shape selectivity of the neuron. Given that when we manipulated attention we did not see interactions with the shape selectivity of these neurons, these findings suggest that shape selectivity would not have been significantly altered during the PF task (a task that required only attention to the fixation spot) regardless of whether or not the animal attended to the (peripheral) stimulus.
Significance of shape selectivity in FEF
We have demonstrated that stimulus shape information is represented in the visual responses of many cells in the FEF. Under the conventional bifurcation of visual processing into a ventral “what” stream and a dorsal “where” stream, such shape representation in a dorsal structure such as FEF is not expected, although it is consistent with a number of previous reports of shape selectivity in other dorsal stream structures (Murata et al. 2000; Nakamura et al. 2001; Sereno and Amador 2006; Sereno and Maunsell 1998; Sereno et al. 2002; Shikata et al. 1996). The shape representation we observed in FEF, like that previously reported in LIP (Sereno and Maunsell 1998), may seem redundant, as many neurons in the so-called “what” pathway are known to be specialized in object feature processing (Hegde and Van Essen 2007; Tanaka 1996). However, re-representation of properties is a pervasive feature of the primate nervous system (>30 distinct visual areas are recognized) and the extraction of shape information in different areas may be distinct and presumably serving different goals (Lehky and Sereno 2007; in contrast, see Konen and Kastner 2008).
Behaviorally, our DMTS task demonstrated that the monkeys were capable of making saccades to peripherally located targets based on target shape. This ability to saccade directly to a shape-defined target indicates that shape information must be available in some manner to the machinery computing voluntary saccade trajectories. We suggest that the shape responsive neurons observed in FEF may contribute to such computations. Shape information in FEF could be used in a direct and detailed manner. For example, Moore (1999) reported behavioral results showing that saccade endpoints demonstrate small deviations that align along the long axis of an elongated bar. That would require information about the positions and orientations of the target's edge contours during saccade planning. However, the shape information in FEF need not necessarily be used directly. Rather it could also serve as input for the construction of a salience map, which some previous reports have hypothesized to be located in FEF (Bichot and Schall 1999; Schall 2002; Schall and Hanes 1993; Schall et al. 1995a; Thompson and Bichot 2005; Thompson et al. 2005). If so, our findings would indicate that FEF salience maps could be generated internally within FEF based on relatively low-level perceptual inputs rather than transmitted to FEF from elsewhere.
Different subpopulations of neurons in FEF may be engaged in different functions. Only a limited percentage of FEF cells were found to be shape selective. Another subpopulation may form a more abstract representation of the sensory input in the form of a salience map as has already been alluded to. Still others may provide orienting signals as output to subcortical oculomotor nuclei (Segraves 1992; Segraves and Goldberg 1987; Sommer and Wurtz 2000) or attentional modulation of visual cortex (Awh et al. 2006; Moore and Fallah 2003; Wardak et al. 2006).
Shape: perceptual similarity versus motor interactions
Previously we have shown that the shape spaces within the ventral and dorsal streams are different (Lehky and Sereno 2007), perhaps reflecting different requirements for visual tasks focused on perceptual similarity, such as object recognition, and those focused on guiding motor sequences. These findings suggest that what differentiates the two visual streams is not a dichotomy between shape versus spatial processing but rather different implementations of both those attributes in each stream to accomplish different goals. The representation of shape in FEF and other dorsal stream areas therefore need not relate to perceptual similarity per se but rather shape as it relates to our motor interactions with the world, such as eye-hand coordination. For example, during visually directed hand/arm movements, an eye saccade to the grasp point of an object typically precedes the actual grasp of the object (Ballard et al. 1992; Johansson et al. 2001; Land et al. 1999). However, the grasp point is not always predictable from the object's form or visual center of gravity. For instance, two objects such as a sword or axe may share a visually similar form (i.e., an elongated object with one end a bit larger) and thus have a similar center of gravity yet be quite dissimilar with respect to a reach or grasp point. That is, one would typically grasp a sword at the visually “heavy” end, an axe at the “light” end. Thus the shape representation used in planning saccades within a behavioral context that affords or demands a particular reach or grasp may be embedded within a shape space that relates to object manipulation as opposed to perceptual similarity (Murata et al. 2000; see also Riggio et al. 2006 for grasp effects on orienting responses). Further, the shape space may be influenced by the goals of the grasping behavior (a grasp point is different when sharpening the axe vs. using the axe). In the present task, the shapes were neither graspable nor clearly associated with a graspable 3-D object, and these factors may have reduced observed shape-related selectivity in FEF. Even under these conditions, we demonstrate unequivocal shape selectivity in FEF.
In conclusion, by focusing directly on simple 2-D shape selectivities rather than target selection, we have uncovered with single-cell recordings a novel aspect of FEF function— shape selectivity that is independent of covert and overt orienting. The fMRI activation of FEF in anesthetized monkeys in response to 3-D shapes further supports our findings. Additional investigations and comparisons between the shape encoding strategies in FEF and other areas will be necessary to address questions about the origin, mechanism, and functional significance of FEF shape selectivity. The observation of shape selectivity in FEF, a property typically associated with the ventral “what” pathway, encourages a general reconsideration of the essential differences between dorsal and ventral visual processing.
This work was supported by National Institute of Mental Health Grant R01 MH-63340 to A. B. Sereno and grants from the National Alliance for Research in Schizophrenia and Depression to A. B. Sereno, the McDonnell Foundation to A. B. Sereno and M. E. Sereno, and the Alexander von Humboldt Foundation to M. E. Sereno.
We thank Dr. N. K. Logothetis for inclusion of fMRI data collected in his laboratory at the Max Planck Institute for Biological Cybernetics, Tübingen Germany, Dr. J. D. Schall for critiques on a previous version of the manuscript, and Dr. K. S. Saleem for useful feedback on the fMRI data.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2008 by the American Physiological Society