INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

The primate posterior parietal cortex (PPC) contains several cortical areas that are crucial for the coordination of spatial perception and action (Ungerleider and Mishkin 1982). A key to understanding the functioning of the PPC is to determine the representations employed by these cortical areas in transforming visual information into motor plans. In particular, in this paper we are concerned with the extent to which parietal representations are influenced by the environment in which the movement is made, here the presence of an object to which an animal is looking. We focus on the lateral intraparietal area (LIP) that lies on the lateral bank of the intraparietal sulcus and is involved in the control of saccadic eye movements (Andersen et al. 1987; Asanuma et al. 1985; Colby et al. 1996; Gnadt and Andersen 1988).

LIP activity has been the focus of many studies, and some basic properties are well established. Cells in LIP show elevated activity both in response to visual stimuli and before spatially guided saccadic eye movements (Andersen et al. 1987; Goldberg et al. 1990). Yet as shown by the memory saccade paradigm, this activity requires neither the immediate presence of a visual stimulus nor the immediate execution of an eye movement: when a visual cue is extinguished prior to the "go" signal for the saccade, increased neural activity is maintained during the delay (Barash et al. 1991; Gnadt and Andersen 1988). Furthermore, LIP uses a predominately retinotopic representation, i.e., activity depends on the vector from the current eye position to the cue or movement end point location, although the gain of the response can vary as a function of eye position (Andersen et al. 1990; Barash et al. 1991).

One caveat to the preceding description is that LIP has often been studied with movements to points of light on a homogeneous background. Human psychophysical experiments have shown that the objects in a scene have a profound influence on the location of saccade targets. For example, when viewing images such as photographs and drawings, the saccadic scan paths follow certain patterns (Noton and Stark 1971; Yarbus 1967). At a finer level, when subjects are asked to look at small objects, saccadic end points are influenced by the shape of the object (McGowan et al. 1998).

Several authors have recorded neural activity in the eye movement areas of the parietal and frontal lobes in monkeys trained to make saccades to objects that possess particular features among fields of distractor objects (Bichot and Schall 1999; Bichot et al. 1996; Constantinidis and Steinmetz 2001; Gottlieb et al. 1998; Hasegawa et al. 2000; Schall and Hanes 1993). In general, the activity correlates with the retinotopic location of the salient or target object. While these experiments examined target selection between objects, Olson and Gettner (1995) studied target selection within an object. In this case, neurons were observed in the supplementary eye fields (SEF) of the frontal lobe with object-fixed receptive fields. The goal of this study is to investigate whether neurons in area LIP make use of object-fixed or other object-based representations when the saccade target is specified by its location on an object.

Object-based representations in the parietal lobe are suggested by the existence of patients exhibiting object-fixed unilateral spatial neglect following lesions to the PPC (Behrmann and Moscovitch 1994; Driver and Halligan 1991; Driver et al. 1994; Hillis and Caramazza 1995). In these cases, subjects tend to ignore parts of an object that are on its contralesional side, with respect to some canonical orientation, regardless of the position or orientation of the object in the visual field. In addition, two groups have shown that the shape or pattern of visual objects is reflected in the patterns of neural activity in LIP (Goldberg and Gottlieb 1997; Sereno and Maunsell 1998).

Together, these facts raise the possibility that cells in LIP may represent the object-fixed location of a target or encode other information about the object itself during an object-based saccade task. To test whether such a task-dependent representation exists, we have studied the activity of cells in LIP while monkeys made saccadic eye movements to object-based target locations. Our object-based saccade task is based on the standard memory saccade paradigm. However, while in the standard memory task the target location is indicated by a briefly flashed cue on an otherwise dark and homogeneous background, in the object-based task, the target is a previously cued part of an abstract object that was rotated between the cue presentation and the execution of the saccade. We analyzed the neural activity recorded in LIP during these object-based saccades to determine which spatial parameters were represented, and these results were compared to similar analyses of two control tasks, a standard memory saccade task, and passive viewing of the object.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Two juvenile male Rhesus monkeys (Maccaca mulatta) were used in this study. All the protocols were approved by the Caltech Institutional Animal Care and Use Committee. During the experiments the animals were on a water-restricted schedule. In each session they were seated in a primate chair with their head restrained, facing a projection screen 66 cm ahead in an otherwise completely dark room. Visual stimuli were created on a PC at VGA resolution (640 × 480 pixels) and projected on the screen using a NEC Imager410 CRT projector. Animals were rewarded with a drop of water or juice for each successful trial. An experimental session usually lasted for 3-5 h and was terminated when the animals' performance dropped below a variable threshold.

Animal preparation

A sterile surgical procedure under sodium pentabarbitol was performed to prepare the animals for the experimental procedures. Animals were fitted with a chronic stainless steel head holder embedded in methylmethacrylate to allow immobilization of the head. A monocular scleral search coil was implanted between the conjuctiva and sclera to monitor the eye position (Judge et al. 1980). In a subsequent surgery a 2-cm diam craniotomy was performed at the following stereotaxic coordinates: posterior 12 mm and lateral 6 mm. The placement was chosen so as to be centered at the intraparietal sulcus and to gain access to the LIP. A Lucite cylinder was placed on top of the craniotomy and was fixed in methylmethacrylate.

The placement of the craniotomy was verified by structural magnetic resonance imaging (MRI). In the first monkey, the MRI images were taken shortly after the completion of the recordings, which was approximately 2 yr after the surgery. The images revealed local damage in area LIP, probably caused by microvascular bleeding from repeated electrode penetrations. This served as a confirmation of the placement of the recording electrodes in the experiment. It should be noted however, that although the local damage in LIP was clearly visible in MRI, behavioral tests in the monkey (extinction paradigm; Li and Andersen 1997) did not show any deficits. In the second monkey, MRI scans were available before the surgery and served as verification of the stereotaxic coordinates for the subsequent craniotomy.

Electrophysiological recordings

Extracellular recordings were made using tungsten electrodes (Frederick Haer Inc.) with approximately 1-1.5 M impedance measured at 1 kHz. A stainless steel guide tube was first manually lowered until it penetrated the dura. Electrodes were advanced through the guide tube with a hydraulic micropositioner (Narishige). Signals from the electrodes were amplified and band-pass filtered (200 Hz-3 kHz). Single neuron action potentials were manually isolated using dual window discriminators. If a waveform passed through both windows, the acceptance signal from the second discriminator was time stamped and recorded by custom software on a PC.

During a recording session, the electrode was slowly advanced while the monkey performed the memory saccade task, and the signal was monitored for signs of oculomotor-related activity. When such activity was found, a single cell was isolated and spike times were recorded and stored for further data analysis (see Data analysis). Cells were recorded at depths ranging between 1 and 7 mm below dura. In all, 261 cells were recorded: 171 from monkey 1 and 90 from monkey 2.

Behavioral tasks

Three behavioral tasks were used in this study: a memory saccade task (MEM-SACC), an object-based saccade task (OBJ-SACC), and an object-fixation task (OBJ-FIX) (Fig. 1). Activity was recorded for all 261 cells during both the MEM-SACC and the OBJ-SACC tasks. For 44 of those cells, the OBJ-FIX task was conducted as well. Typically the tasks were run in separate blocks. However when only the MEM-SACC and OBJ-SACC tasks were performed, trials for the two tasks were sometimes randomly interleaved. For most cells, five repetitions of each trial condition were performed.

View larger version (16K): [in this window] [in a new window]   Fig. 1. Behavioral tasks. Time lines show the appearance (high) and extinction (low) of the object, the cue, and the fixation light for each of the 3 tasks. Labels on the abscissa show the time (milliseconds) between events. The dotted lines in the MEM-SACC and OBJ-SACC time lines represent the variable interval between the "go" signal (extinction of fixation point) and the saccade (filled triangle). The saccade target had to be acquired within 1,000 ms of the "go" signal to receive a reward. Behavioral periods used in subsequent analyses are shown above each time line. OBJ-SACC: figures on top show the visual scene during each of the behavioral periods for a sample trial. Arrow in the saccade figure represents the saccade and was not displayed during the experiment.

In all three tasks, the central fixation point and any saccade target locations were displayed as circles of approximate diameter 1° of visual angle. The central fixation point was located directly in front of the monkey at eye level.

MEMORY SACCADE TASK. Trials began with the appearance of a red central fixation point. After the monkey acquired fixation, a peripheral target briefly (500 ms) appeared 10° of visual angle from the fixation point. The target cue was identical in appearance to the fixation point. After a 500-ms delay, the fixation point was extinguished, which was the signal to make a saccade towards the remembered location of the cue. The trial was successful if the saccade landed within an 8° rectangular window centered on the target. For monkey 2 only, the target was redisplayed 150 ms after a successful saccade (shown on Fig. 1). This feedback was necessary to maintain accurate performance, and the neural data used in all our analyses were recorded prior to redisplaying the target. Three MEM-SACC trial periods were used for analysis of neural activity (see Fig. 1): the 500-ms duration of the cue (Cue), the 400-ms window starting 100 ms after cue offset and ending at fixation point offset (Delay), and the 300-ms window beginning 200 ms before saccade initiation (Saccade). The only visual stimuli presented during MEM-SACC trials were the fixation and target lights; no object appeared. There were eight trial conditions corresponding to eight potential target locations, equally arrayed around the central fixation point.

OBJECT-BASED SACCADE TASK. The object used in this task was a filled green polygon with the distinctive shape depicted in Fig. 1. This shape was designed to meet several criteria. 1) The shape should have a clear principal axis. Such elongated objects have a clearly definable object-fixed reference frame (Marr and Nishihara 1978). 2) The shape should be non-symmetrical. Since there are reports of object-based neglect for non-symmetrical but not for symmetrical objects (Behrmann and Moscovitch 1994), this choice is more conservative. 3) The shape should contain several potential target regions that are devoid of distinct local features. This condition was chosen to force the monkeys to use the overall shape of the object to locate a previously cued region. 4) The shape should be abstract, to avoid any previously learned associations and biases. The resulting design has four nearly identical "fingers" emanating from an elongated "handle." The fingers extend from the center of the object at angles of 0°, 45°, 90°, and 135°.

View larger version (12K): [in this window] [in a new window]   Fig. 2. Three theoretically plausible coordinate frames. R, retinotopic direction; O, object orientation; F, object-fixed location ("finger").

OBJECT FIXATION TASK. In this task the monkey was required only to fixate the central point while the object was displayed and rotated in the manner of the OBJ-SACC task. No saccade cues were displayed. If the fixation was broken at any point during the trial, the trial was aborted. The nature of the task was signaled to the monkey by a different color for the central fixation point (yellow instead of red). Two OBJ-FIX trial periods were used for analysis of neural activity (see Fig. 1): the final 600 ms of the first object appearance (Pre-Rotation) and the final 600 ms of the object reappearance (Post-Rotation).

Data analysis

SPATIAL VARIABLES. The main purpose of this study was to investigate the coding of object-based saccades in LIP. As such, we will consider the various spatial variables that describe the OBJ-SACC task and that could potentially be used as reference frames for describing the visual stimulus or the saccadic eye movement. Note that the central fixation point was constant across trials, the object was always centered at this point, and the amplitude of saccades was fixed (10°). Thus there are only three remaining spatial variables across trials, as shown in Fig. 2. Consider the cue period of the task. The direction of the cue with respect to the central fixation point will be called the retinotopic direction of the cue, or R. Note that while we use the designation retinotopic, the monkey's head and body were fixed throughout the trial, so this variable could equally well be called head-centered or ego-centric direction of the movement. Next, the angle of the visual cue with respect to the axis of principal elongation of the object will be called the object-fixed direction, or F (for "finger"). This variable depends only on the part of the object that had been cued, and thus represents the location of the cue in the object-fixed reference frame. Finally, the orientation of the object on the projection screen will be denoted O. Note that although each of these angles specifies different information about the display and the task requirements, the three variables are not independent, since R = O + F.

View larger version (27K): [in this window] [in a new window]   Fig. 3. Idealized cells "tuned" for each task angle. Each column shows imaginary firing rates for a different idealized cell. Top row: tuning curves represent the average firing rate during a trial interval as a function of the retinotopic direction, R, of the cue or target. Firing rates are plotted separately for targets lying on each of the 4 object "fingers," F (see object icon at bottom for key). Bottom row: Tuning as a function of object orientation, O, during a trial interval. As above, trials were grouped by object-fixed target locations, F, and separate tuning curves were made for each group.

GENERAL LINEAR MODELS FOR THE OBJECT-BASED SACCADE TASK. To interpret the response of cells with more complex firing patterns than of Fig. 3, we made use of a hierarchical set of general linear models (GLM). Each model attempts to fit the observed firing rates r_i (for trial i) in a particular temporal period of the experiment, i.e., the total number of spikes that occurred in that period divided by the duration of the period. The GLM assumption is that the firing rate is influenced by each spatial variable (R, O, and F) in an additive manner, i.e., the effects sum to define the overall response of the cell during the OBJ-SACC task. The hierarchical scheme is depicted in Fig. 4. The simplest GLM is one in which all trial classes have the same mean firing rate, . This model is denoted by the symbol , since the set of variables on which the firing rate depends is the empty set. A one-way ANOVA on the response of an  cell as a function of any of the three task angles would yield no significant effect. Responses in this category should thus be considered "untuned."

View larger version (13K): [in this window] [in a new window]   Fig. 4. The hierarchy of general linear models (GLM). Each box represents a potential model for the firing rate of a cell in a particular experimental period, as a function of the retinotopic direction R, the object orientation O, and the object-fixed location F. Hierarchy level denotes the number of independent variables included in the model. is the model prediction, is the overall mean response of the cell, and f(·) is the "tuning curve" or additive contribution due to the subscripted variable. Lines between models denote a hierarchical relationship: the top model is a superset of the bottom.

MODEL SELECTION. We wanted to identify the model of Fig. 4 that "best" accounts for the response of each cell during each of the four OBJ-SACC trial periods (Pre-Cue, Cue, Delay, and Saccade; see Fig. 1 for definitions). While it is a simple matter to fit each of the models, it is less straightforward to choose among them. We used a variant of the standard stepwise regression method (Draper and Smith 1998). For each cell and task period, we started by adopting the  model, and iteratively added input variables that significantly improved the fit (step up the hierarchy) and pruned those which no longer do (step down). This continued until a model was reached that was a significant improvement over all those beneath it and for which all models above were not a significant improvement. At every step a comparison was also made to the  model, ensuring that when the iteration settled on a model, it was not only a significant improvement over the models directly below it on the hierarchy, but it was also significantly better than the  model, i.e., it was significant in the absolute sense.

GLMS FOR THE MEM-SACC AND OBJ-FIX TASKS. We analyzed the neural responses for the other two tasks in a manner similar to that described above for the OBJ-SACC task. In the MEM-SACC task, there is only one spatial variable, the retinotopic direction of movement. Thus the hierarchy contains only two models,  and R. In the object-fixation task, there is also only one spatial variable, the orientation of the object. However, the value of that variable changes during the course of the trial. We thus included two input variables, O_pre and O_post, giving a total of five models, , O_pre, O_post, O_preO_post, and Int. Note that we could have allowed the models in the OBJ-SACC task to depend on both the pre- and post-rotation values of the variables, but that would have led to an unmanageable number of models. And as will be seen later, for the object fixation task at least, cells only appear to code the current value of the variable.

CELL SELECTION. We chose to use neural responses in the MEM-SACC task to select cells for further analysis. In the following, we only consider cells that showed significant tuning in R in at least one of the three MEM-SACC trial periods (Cue, Delay, and Saccade). Significance was assessed by the means described above, i.e., using the non-parametric permutation test (Efron and Tishirani 1993).

COMPARISON OF TUNING ACROSS TASKS. We will want to compare a cell's retinotopic tuning between the OBJ-SACC and the MEM-SACC tasks and its object orientation tuning between the OBJ-SACC and OBJ-FIX tasks. We will do this using a shifted cross-correlation analysis. The cell's response from a particular period of the OBJ-SACC task was fit with the RO model to get retinotopic and object orientation tuning curves, f_R(R) and f_O(O). The retinotopic tuning curve was also obtained for the analogous trial period in the MEM-SACC task, and similarly for the object orientation tuning in the OBJ-FIX task. Consider first the retinotopic tuning. A set of eight shifted tuning curves was obtained, f_R(R  ) with   0°, 45°, ... , 315°. Each of these eight tuning curves was cross-correlated with the MEM-SACC tuning, and the value of the shift  that resulted in the best cross-correlation was noted. The same computation was carried out for the orientation tuning comparison. If the greatest similarity in tuning occurs at  = 0°, we say that the two tuning curves are aligned.

ANALYSIS OF OBJECT ORIENTATION TUNING. We will show that the firing rate of many cells is influenced by the object orientation, O. We will investigate the relationship between this effect and cells' retinotopic tuning properties by considering three hypotheses. First, the retinotopic and object orientation tuning curves could be derived independently from a cell's input. This "Independent Coding" hypothesis predicts no systematic relationship between the two tuning curves. Alternatively, cells could be responding to the orientation dependent presence of potential targets, i.e., fingers, in the retinotopic receptive field. This is the "Potential Targets" hypothesis. Cells could also be responding to variations in the overlap between their sensory receptive fields and the object: the "Sensory Stimulation" hypothesis. These two later models both predict increased activity when a finger of the object is in a cell's retinotopic receptive field. The Sensory Stimulation hypothesis predicts an equal response to the object handle. As will be seen, the animals almost never erred by looking at the handle of the object. Thus the Potential Targets hypothesis would predict no increase in activity due to the presence of the handle in a cell's receptive field, giving us a means of comparing these two models.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Behavioral results

Before considering the neural recordings, we present results on the monkeys' performance in the OBJ-SACC task. Figure 5 shows the saccade end point location for all trials with a retinotopic target of R = 0°. The figure includes every such trial used in the neural analysis below. Two kinds of trials are displayed: successful trials, or hits, and trials in which saccades were executed at the proper time but towards an incorrect location, errors. In both cases, location of the mark in Fig. 5 represents the end point of the first saccade after the go signal. The timing of the OBJ-SACC task was designed to prevent corrective saccades after an error. Figure 5 shows that this effort was moderately successful: in over 4,000 trials displayed, there were 18 corrected saccades (square markers). In the whole data set, 1.7% of trials contained successful corrective saccades. To ensure that these trials did not effect our conclusions, they were removed from the data set along with all error trials, prior to the analysis of the neural data.

View larger version (20K): [in this window] [in a new window]   Fig. 5. Sample saccade end points. Movement end points for the first saccade recorded in every trial with target at R = 0°. Each column contains data for a single monkey. Each row contains trials with a different target finger, F. Hits, black circles; errors, gray triangles; fixation point, black +; target, white +. Trails with successful corrective saccades (see text) are indicated with a large box around the end point of the 1st saccade. Monkey 1, N(hit) = 3,062, N(err) = 394; monkey 2, N(hit) = 1,373, N(err) = 367, all target locations combined. Note that 1 panel is missing in the right column since monkey 2 only performed movements to 3 of the object fingers.

The errors in Fig. 5 show a distinct pattern, clustering at locations on the object which are potential targets. Furthermore, the density of errors on a particular finger depends primarily on the proximity to the target finger. This observation is confirmed for the whole data set in Fig. 6. For each target finger, a histogram shows the rate of errors as a function of the object-fixed angle of the movement, F_movement. It was rare for either monkey to make a saccade to a finger more than 45° away from the target.

View larger version (16K): [in this window] [in a new window]   Fig. 6. End point angle of error trials. Histogram of Fmovement for all error trials, including corrected errors (dark bars). Ordinate represents the rate of errors within a bin of movement angles, as a percentage of the total number of trials with that target finger. Each column shows data for a single monkey. Each row shows trials with a different target finger, Ftarget. The solid vertical line represents Ftarget; dotted lines represent the 3 other finger locations. Monkey 1, N = 28,546; monkey 2, N = 13,166, all target locations combined. Note that 1 panel is missing in the right column since monkey 2 only performed movements to 3 of the object fingers.

Hit rates for the OBJ-SACC task are shown in Fig. 7. There is a clear dependence of the error on the identity of the target finger, F. For example, both monkeys made the least errors when looking to the first finger, F = 0°. There is also a significant dependence on R.

View larger version (16K): [in this window] [in a new window]   Fig. 7. Hit rate. Each data point is the average (SD) hit rate in the OBJ-SACC task across individual recording sessions. Abscissa is the retinotopic target location, R. Each line is a single target finger, F (for key see Fig. 3). Two-way ANOVAs, R × F, show significant main effects due to R and F (P < 0.01) and significant interaction (P < 0.01) separately for both monkeys. Monkey 1, N = 171 sessions; monkey 2, N = 90 sessions.

Neural recording

We recorded from 261 cells in two monkeys, 171 from the right hemisphere of monkey 1, and 90 from the left hemisphere of monkey 2. Of these, 165 were selected for further analysis, 81 and 84 from each monkey, respectively (see METHODS).

We begin by asking whether the representation used by neurons in LIP during the OBJ-SACC task corresponds to any of the three ideal reference frames: retinotopic direction, object-fixed direction, or object orientation. We plotted cells' responses as in Fig. 3, and attempted to informally classify them. Many cells, such as the one shown in Fig. 8, appear to code the task in a purely retinotopic coordinate frame. Others, such as the cell depicted in Fig. 9, might be best described as coding the orientation of the object. On the other hand, this informal inspection did not reveal any cells with responses approximating the object-fixed reference frame of Fig. 3. Also, many of the cells we recorded displayed complex response patterns that were difficult to interpret. For example, Fig. 10 shows a cell whose response does not lie squarely in any of the three postulated coordinate frames. Therefore to better characterize the population, we employed a GLM analysis based on these three task variables.

View larger version (28K): [in this window] [in a new window]   Fig. 8. A sample retinotopic direction cell. Monkey 1. Each column shows mean firing rates for a different trial period. Data points represent average firing rates for each trial condition. Conditions with targets lying on each of the four object "fingers" are plotted as separate lines (see object icon at bottom for key). The single error bar on the right is the average SE across all trial conditions. Top row: firing rate as a function of retinotopic direction. Bottom row: firing rate a function of object orientation.

View larger version (26K): [in this window] [in a new window]   Fig. 9. A sample object orientation cell. Monkey 1. See legend of Fig. 8 for details.

View larger version (25K): [in this window] [in a new window]   Fig. 10. A sample cell with a complex response pattern. Monkey 2. See legend of Fig. 8 for details.

GLM analysis

For each cell, task, and period, we fit a hierarchical array of GLMs incorporating an increasing number of independent variables and then selected the best among these models using a stepwise regression algorithm and permutation hypothesis tests (see Fig. 4 and METHODS). This procedure allowed us to classify cells based on the spatial variables that significantly influenced the cell's firing rate, even in the case of cells with complex response patterns. Some cells showed no significant tuning to any of the spatial variables in some trial periods, and the model selection criterion classified these cells as  during those periods. Figure 11 summarizes the results for the population of cells recorded during the OBJ-SACC task. Each histogram displays the distribution of models selected to fit the 165 cells during each of four trial intervals: Pre-Cue, Cue, Delay, and Saccade (see Fig. 1).

View larger version (17K): [in this window] [in a new window]   Fig. 11. Summary of GLM results for the OBJ-SACC task. Cell-count histograms of best-fit GLM model for each trial period.

PRE-CUE. In this period the cue location is not yet known, so the R and F variables should not be able to influence firing rate. As expected, all but a few cells are either untuned, , or show tuning only in the object orientation, O.

CUE. This is the interval when the cue is illuminated at one of the four object locations. Most of the cells' firing rates are best fit using either the retinotopic location R (30), the object orientation O (31), or both (41). No cells were classified with purely object-based reference frame (F), although a number show finger modulation effects (4 RF, 3 OF, and 16 ROF). A smaller number of cells (13) were best described as having non-linear interactions between the variables (Int).

DELAY. This interval begins 100 ms after the reappearance of the object in its new orientation. The population response pattern is similar to that of the Cue interval, with the biggest difference being a drop in the number of untuned cells (12 in Delay compared to 24 in Cue) and a commensurate increase in the number of cells exhibiting significant non-linear interactions (26 cells in Delay).

SACCADE. In this interval, the number of cells classified as orientation cells, O, decreased dramatically (11 in Saccade compared to 37 in Delay), while those classified as retinotopic, R, increased by nearly as much (45 in Saccade, 26 in Delay). As the task proceeds toward the motor response, more cells are showing tuning in the retinotopic coordinate frame.

Sample cells

We now attempt to give an intuitive description of the GLM classification by examining a few sample cells. For the retinotopically tuned cell shown in Fig. 8, the R model was selected for all three trial periods. Similarly, for the object orientation cell shown in Fig. 9, the O category was selected for the Cue and Delay periods (the cell was classified as R for the Saccade period). In general, when the best model contains only a single input variable, we will say that the cell represents the task in the corresponding reference frame.

The "difficult to interpret" cell from Fig. 10 was classified in the RO category. Its response is re-plotted in Fig. 12A, along with the RO model fit (B) and the corresponding turning curves (C). This seemingly complex response pattern is well described by the simple summation of retinotopic and object orientation tuning curves [R² = (0.48, 0.53, 0.61); R = (0.26, 0.33, 0.44); in Cue, Delay, and Saccade periods, respectively]. Comparison of the retinotopic tuning from the RO model, f_R(R) in Fig. 12C, with the retinotopic tuning from the MEM-SACC task (Fig. 12D) reveals a very similar response pattern. This correspondence suggests that the retinotopic effects could be task independent and lends some support to the additive effects model of the GLM. (For this cell, the OBJ-FIX task was not performed, so we can't verify the object orientation component, but this issue will be addressed further below).

View larger version (24K): [in this window] [in a new window]   Fig. 12. A sample cell with best model RO. This is the same cell from monkey 2 shown in Fig. 10. A: mean firing rates. See legend of Fig. 8 for details. B: predictions of model RO, which was the best fit for this cell in each of the 3 trial periods. C: retinotopic and object orientation tuning curves that comprise the best-fit model. D: retinotopic tuning curves from the MEM-SACC task.

Figure 13 shows a different cell that is classified as RO in all three trial periods. Here too the model fits the data well and has good predictive power [R² = (0.74, 0.71, 0.79); R = (0.66, 0.63, 0.73); in Cue, Delay, and Saccade periods, respectively]. There is also good agreement between the retinotopic tuning as defined by the RO model (Fig. 13C) and that measured in the MEM-SACC task (Fig. 13D). Finally, Figs. 12 and 13 both exhibit a trend that is typical of the whole population: while the magnitude of the object orientation effect remains constant or decreases over the course of the trial, the retinotopic effect tends to increase from the Cue to the Saccade period. This effect will be examined more quantitatively below.

View larger version (27K): [in this window] [in a new window]   Fig. 13. A sample cell with best model RO. Monkey 1. A: mean firing rates. See legend of Fig. 8 for details. B: predictions of model RO, which was the best fit for this cell in each of the 3 trial periods. C: retinotopic and object orientation tuning curves that comprise the best-fit model. D: retinotopic tuning curves from the MEM-SACC task.

The least intuitive of the model categories is Int. These are cells that showed significant non-additive dependencies on the three spatial variables. Figure 14 shows OBJ-SACC activity for two cells that were classified as Int for two or three trial periods. The cell in Fig. 14A was classified as R for the Cue period, but as Int for the Delay and Saccade periods. The R² value is of course higher in all three periods for the more complex model [R = (0.27, 0.40, 0.67), R = (0.37, 0.67, 0.80), for the three successive periods], but the cross-validation measure agrees with the model selection procedure [R = (0.19, 0.33, 0.63), while R = (0.03, 0.47, 0.67)]. These differences are not large, and it would be difficult to say by eye whether the R or Int models are better descriptions of the data in the Delay and Saccade periods. We return to this issue below.

View larger version (36K): [in this window] [in a new window]   Fig. 14. Two sample cells with best model Int. A: cell from monkey 1 with best model Int in Delay and Saccade periods, R in Cue period. B: cell from monkey 2 with best model Int in all 3 periods. See legend of Fig. 8 for details. Note that since the Int model contains a free parameter for each trial condition, the model prediction is the same as the class averages shown here.

Figure 14B shows a second example of a cell that was classified as Int in all three periods. As is usually the case, the Int model had a fairly high [R² (0.78, 0.56, 0.63) for the Cue, Delay, and Saccade periods]. The model also had better predictive value than either of the two closest models, ROF and RO [R = (0.65, 0.40, 0.42), R = (0.55, 0.10, 0.24), and R = (0.59, 0.07, 0.21)].

Overall, 13% of the cell-by-period classifications were assigned the Int model. One possible explanation for this finding is that the interaction between the input variables could be multiplicative, i.e., the cells might have "gain fields" (Andersen et al. 1997). To test whether this explanation is valid, we re-ran the full GLM analysis using the logarithm of the firing rate in each period, turning multiplicative interactions into additive interactions. Overall, the results did not show a substantially different picture from the one presented above, except that over one-half (37 of 67) of the cell-by-period Int classifications made using the additive model were now categorized otherwise, almost all as R, RO, or ROF. This suggests that for at least some portion of the cells recorded, a multiplicative interaction would be a better model. Still, it must be noted that of the 165 cells recorded, only 5 were classified as Int in all three trial periods.

Finally, Fig. 15 shows data from a cell with a small but significant and consistent effect due to F. The ROF model was selected in the Cue period and Saccade periods, and the Int model in the Delay period. In all three periods, the predictive power of the model improved with the addition of the F factor [R = (0.08, 0.34, 0.43), R = (0.11, 0.40, 0.52), and R = (0.14, 0.46, 0.51)]. Note that the basic pattern of activity is quite similar in the three trial periods and neither the ROF nor Int models are clearly preferable in any period. Another thing to note about this cell is that the retinotopic and object orientation tuning curves are stable across the trial and resemble the tuning from the MEM-SACC and OBJ-FIX tasks, respectively. The correspondence between tasks is typical, as will be assessed directly below.

View larger version (21K): [in this window] [in a new window]   Fig. 15. A sample cell with best model ROF. Monkey 2. A: mean firing rates. See legend of Fig. 8 for details. B: predictions of the best-fit model for each period: ROF in Cue and Saccade, Int in Delay. C: retinotopic and object orientation tuning curves that comprise the best-fit model. Note that the Int model has a free parameter for each trial condition and so it doesn't have tuning curves like the other model. To facilitate comparison with D and E, the Delay period tuning curves for the ROF model are plotted here. D: retinotopic tuning curves from the MEM-SACC task. E: object orientation tuning curves from the OBJ-FIX task.

The example of Fig. 15 helps illustrate an important point: the model selection procedure forces a hard categorization even when several models may be almost equally suitable. This can be seen in previous examples as well; it would be hard to decide by eye whether the R or O model better describes the Cue period activity in Fig. 8 or the Saccade period activity in Fig. 9. This problem is inherent in any categorization scheme, and so we would like to confirm our findings using a graded measure of the effect on firing rate of the three spatial variables. Such an analysis is presented in the next section.

Predictive power: R

In the following, we will consider the percentage of the total variance in a cell's firing rate predicted by each of the three spatial variables. However, to establish a reference point, we first examine the predictive power of the best-fit GLM models. Figure 16 shows the distribution of R for the best-fit model for each cell and trial period, broken down by model complexity. As the number of independent variables in the best-fit model increases, the goodness of fit improves. This is not simply due to the increase in the number of parameters, since the R measure penalizes over-fitting. Rather, more predictable firing rates, i.e., lower variability within a trial class, will generally yield greater statistical significance for more complex models.

View larger version (14K): [in this window] [in a new window]   Fig. 16. Box-plot of the best-fit R as a function of model complexity. Each column shows the median (middle horizontal line), upper, and lower quartile values (box limits), and ranges (T-bars, with outliers shown as single data points) of the R values for cell periods with best-fit models of a given complexity. Right-most box is a summary of all tuned cells, regardless of model complexity.

Figure 17 shows population histograms of R for each of the three single variable models in each of three trial periods. The first point to note is that the median value for R is near zero in each of the trial periods [median over significant fits, (0.037, 0.040, 0.085), for the Cue, Delay, and Saccade periods, respectively]. Furthermore, the F model is significant in about one-tenth of the cells (9% Cue, 12% Delay, and 15% Saccade). Together, these two facts suggest that object-fixed coding plays a small role in the neural activity of LIP.

View larger version (22K): [in this window] [in a new window]   Fig. 17. Summary of predictive power of the R, O, and F models in the object-based saccade task. Each column contains histograms of R values for a particular period of the task, each row for the effect of a particular variable. Black bars signify cells where the model was a significant improvement over the  model, white bars are for cases which did not reach significance. (Significance is tested with a permutation test, see METHODS). Gray triangle, median R for all cells; white triangle, median R for significant cells only.

The second main point is that while the magnitudes of R and R are similar during the Cue and Delay periods, during the Saccade period the effect of R increases, while that of O decreases [median R over significant fits, (0.11, 0.17, 0.19); median R over significant fits, (0.14, 0.16, 0.095), for the three periods]. A parallel shift is seen in the number of cells for which each single variable model yields a significant fit (R: 68% Cue, 72% Delay, 85% Saccade; R: 70% Cue, 82% Delay, 66% Saccade). This confirms the trend seen in Fig. 11 that in the saccadic period the retinotopic tuning of the cells is more pronounced than in the previous two intervals, where the R and O variables had comparable effects.

MEM-SACC and OBJ-FIX tasks

We now consider the classification of cells' responses in the two other behavioral tasks. For the MEM-SACC task, there are only two models to consider, the  and R models, representing untuned and tuned responses, respectively. Of the 165 cells analyzed, 104 were tuned in the Cue period, 97 in the Delay period, and 111 in the Saccade period. Note that by our cell selection criterion, all cells were tuned in one of the three periods.

In the OBJ-FIX task, both the pre- and post-rotation values of O were used as independent variables. Of the 33 cells analyzed for which we have recordings during the OBJ-FIX task, 23 were tuned during the Pre-Rotation period, and 29 during the Post-Rotation periods. With only two exceptions in the Post-Rotation period, the best-fit model was the single variable model containing the current object orientation. Of the two exceptions, one was classified as O_pre and one as O_preO_post.

Since the three behavioral tasks are all quite distinct, it is possible that the neural representations observed in LIP would change across tasks. On other hand, if cells code the basic spatial variables in a constant manner, then the tuning curves should be similar across tasks. The sample cell in Fig. 15 supports the latter case: the retinotopic tuning curve in the OBJ-SACC task (C, top) has nearly the same shape as the that seen in the MEM-SACC task (D), and the orientation tuning (C, middle) is the same as that seen in the OBJ-FIX task (E). To test for such task-independent tuning across the cell population, we employed a cross-correlation analysis (see METHODS). Figure 18 shows the comparison of the retinotopic tuning between the OBJ-SACC and MEM-SACC tasks, and the comparison of OBJ-SACC and OBJ-FIX object orientation tuning. In both cases, the histograms of alignment angles have a clear peak at 0°, the point at which the tuning curves in the two tasks are aligned. Furthermore, few cells show an absolute shift of greater than 45°. We conclude that cells' retinotopic and object orientation tuning patterns are largely task independent.

View larger version (19K): [in this window] [in a new window]   Fig. 18. Alignment of tuning across tasks. A: best angle of alignment between retinotopic tuning in OBJ-SACC and MEM-SACC tasks. Trial periods in panel titles refer to the period from which retinotopic tuning was estimated in the OBJ-SACC/MEM-SACC tasks. B: best angle of alignment between object orientation tuning in OBJ-SACC and OBJ-FIX tasks. Trial periods in panel titles refer to the period from which object orientation tuning was estimated in the OBJ-SACC/OBJ-FIX tasks. A and B: best alignment angle is the angle by which to shift the OBJ-SACC tuning to achieve maximum correlation with the tuning curve from the reference task (see METHODS). A best angle of 0° means that tuning in the 2 tasks is aligned. Histograms only include cells that displayed significant tuning in both tasks in the relevant variable and period.

Orientation tuning

Last, we investigated the nature of the object orientation tuning observed in many cells. As discussed in METHODS, there are three models of the relationship between retinotopic tuning and tuning in O. The two tuning characteristics could be independent. Cells could be responding to the orientation dependent presence of potential targets in a retinotopic receptive field. Or the activity might be due to the overlap between the object and the cell's retinotopic sensory receptive field. To distinguish between these possibilities, we examined the mean orientation tuning curve, averaging across the population of cells after aligning the curves to the preferred retinotopic direction of the cell. The resulting curves, shown in Fig. 19, represent the average change in firing rate when a particular part of the object, indexed by F, is oriented toward a cell's preferred retinotopic direction.

View larger version (22K): [in this window] [in a new window]   Fig. 19. Orientation tuning curves aligned to retinotopic preferred direction. Average aligned tuning curves represent the change in cells' activity when a particular part of the object, F, is aligned with the retinotopic preferred direction. Each cell's orientation tuning curve was normalized to O f(O) = 1 before averaging, which determined the scale of the ordinate. A: orientation tuning estimated by fitting the ROF model to the OBJ-SACC data from each of the 4 labeled trial periods. B: orientation tuning estimated from the OBJ-FIX data in the 2 behavioral periods. Predictions of the Potential Targets (solid) and Sensory Stimulation (dashed) models are also shown, in bold lines if the model was a significant improvement over the null hypothesis of no main effect in F (P < 0.05, contrast analysis, Rosenthal and Rosnow 1985).

The Independent Coding model would predict an arbitrary relationship between the orientation tuning curve and the retinotopic preferred direction of a cell, resulting in a flat average, aligned orientation turning curve. In contrast, the data in Fig. 19 show a systematic dependence of f_O(O) on the cell's preferred direction in every period of both tasks (P < 0.05, one-way ANOVA). Furthermore, the parts of the object that preferentially excite the population of cells are consistent across tasks and trial periods. As can be seen in Fig. 19, this pattern is consistent with the predictions of both the Potential Target (solid lines) and Sensory Stimulation (dashed lines) models. The two models make largely similar predictions, and it is difficult to distinguish between them. However, we note that the the Sensory Stimulation model predicts a comparable response when either the finger at F = 0° or the handle at F = 180° is centered in a cells receptive field, while the Potential Targets model predicts higher activity due to the finger. In fact, in every period in both tasks, the average response to the alignment of the 0° finger with the retinotopic preferred direction was greater than the average response due to the handle. The differences were significant in every case except the Cue period in the OBJ-SACC task (one-tailed t-test, P < 0.05).

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

We have shown that cells in LIP exhibit spatial tuning in the object-based saccade task. The activity patterns are best accounted for using a combination of effects due to the retinotopic direction of the movement and the orientation of the object on the projection screen. The influence of the object-fixed location of the target was relatively minor. Roughly one-tenth of the cells we analyzed showed significant object-fixed effects, but even in those cases this modulation was a relatively small contribution to the overall variation in firing rate.

The GLM analysis

Since our method of classifying cells is based on a linear model, it is possible that significant nonlinear effects were missed. In fact, there was evidence for multiplicative effects in our data. While the GLM analysis assigned the Int model to 13% of the total number of cell-by-period classifications, that number was cut in half when the analysis was performed on the logarithm of the firing rate. However, despite this difference, the rest of the GLM classifications were largely unaffected by the log transformation. This emphasizes that while the GLM assumes additive effects, the framework can be useful even when this assumption doesn't hold. It provides a convenient first-order analysis of the responsibilities of a set of independent variables for even complex patterns of neural activity.

Our conclusions also depend critically on our choice of independent variables. One obvious candidate that wasn't included was the direction of the object rotation. This variable, along with the original retinotopic direction of the cue, would be sufficient to solve the task. We performed a GLM analysis that included this variable, and across the population, it was found to have almost no effect on the firing rate.

Finally, we consider whether our results depend on the cell selection criterion employed. For example, since cells were chosen for analysis based on their activity during the memory saccade task, it is possible that we excluded the cells with greater object-fixed effects. We repeated the GLM analysis on the 100 cells that did not pass the selection criterion. The distribution of best-fit models was quite similar to those seen for the selected cells, but there were more units at the bottom of the hierarchy: in any particular period of the object-based saccade task approximately one-half the cells were untuned, and about three-quarters of the remainder were classified as either R or O. Still, there was no evidence of a population of cells with object-fixed coding.

Alternative solutions of the task

One of the main conclusions we draw from this study is that cells in LIP do not use an object-fixed reference frame in performing object-based movements. This conclusion relies on our assumption that the animals employed an object-based strategy in performing the OBJ-SACC task. It is important to note that the design of the OBJ-SACC task does not absolutely require the monkeys to use an object-based strategy, and so we can not be certain that the monkeys did indeed solve the task in this way. Nonetheless, the behavioral data are consistent with this approach and appear to be inconsistent with the two alternative strategies that we could devise.

First, the existence of local features near the target locations on the object would obviate the need for an object-based strategy. This was a primary consideration in our design of the object, and we believe that no such local features existed. Of course two of the fingers were in unique contexts: the finger at 0° had no finger clockwise to it, and the finger at 135° had the handle counter-clockwise to it. However these cues depend on the relationship between the various object parts. Such cues are very different from, say, differently colored fingers. In the latter case, the monkey could scan the scene for path of the target color. Here, he has to parse the object and look at the finger which bears the correct relationship to the other parts. In other words, he has to use the known structure of the object to solve the task, which was our intention.

The behavioral data also suggests that the animals solved the task by using object-based cues not local features. Consider the case when the target was the finger at 90°. There are two neighboring potential targets: the finger at 45° that has an identical local context and the finger at 135° that lies next to the handle of the object. Thus relying on local features would cause many more errors to the 45° finger. However Fig. 6 shows that monkey 2 almost never looked in error at the 45° finger (which was never cued for him) but made many errors toward the finger at 135°. Monkey 1 had identical error rates to the fingers at 45° and 135°. Similarly, when monkey 1 was presented with a target on the 45° finger, he looked at the 0° finger about 350% more often than the 135° finger, even though the local context of the latter is more similar to the target finger.

A second alternative solution relies on the fact that the object always rotates either +90° or 90°. If the animal were to remember only the cue location, then the saccade target would be constrained to one of two possible locations. Since no two potential targets on the object lie 180° apart, it is always the case that only one of these two possible locations will be occupied by a finger, and this is the correct target location. There is one wrinkle in this strategy. When the 0° finger is cued, one possible target location is occupied by it, and the other is occupied by the object handle. Thus if the animals used this strategy, we would expect that the error rate would be highest when F = 0°. On the contrary, Fig. 7 shows that both animals had a significantly higher hit rate for saccades to the 0° finger than to any other finger. Furthermore, Figs. 5 and 6 show that the monkeys almost never looked at the object handle. In fact, on trials when F = 0°, the monkeys looked to within 22.5° of the handle on less than 1 in a 1,000 trials. And although the rate of such errors was miniscule in all conditions, it was lowest for F = 0° in both monkeys.

Based on these arguments, we believe it is unlikely that the monkeys employed either of these alternative strategies. Of course we can not exclude the possibility the monkeys devised other non-object-based strategies. However observations such as the fact that monkey 2 almost never made saccades to the one finger that was never a target suggest that the monkeys were analyzing the targets in terms of their locations on the object, i.e., that they were using object-based information in deciding where to look.

Object-fixed coding

Our finding that cells in LIP do not use an object-fixed representation in the object-based saccade task appears to be at odds with data from SEF reported by Olson et al. (Olson and Gettner 1995, 1999; Olson and Tremblay 2000). They found that when monkeys make a saccade to a cued edge of a horizontal bar, some saccade related cells show a preference for a particular end of the bar, regardless of the position of the bar on the screen. In other words, cells appear to utilize an object-fixed reference frame. The striking difference between these studies and our own could simply be due to the difference in coding employed in LIP and SEF. The difference could also be due to differences in the details of the two tasks, such as the the size and the shape of the objects.

One important difference between our task and that of Olson et al. is that, while they studied translations of an object on the screen, we used a rotation around a central fixation point. Observing object-fixed effects in a translation task rules out a purely retinotopic reference frame. Similarly, a lack of object-fixed effects in a rotation task rules out a purely object-centered reference frame. However both of these findings are consistent with a "stimulus-centered" reference frame, which remains centered on the stimulus region but does not rotate with objects within that region. The existence of stimulus-centered coding is suggested by stimulus-centered effects in hemi-field neglect following parietal lesions in humans (Arguin and Bub 1993; Hillis and Caramazza 1991). We also note that while object-based neglect is present for conditions of rotation (Behrmann and Tipper 1999), it may be less robust than other forms of neglect (Farah et al. 1990). In any case, fully distinguishing between these three frames of reference will require a study which combines both object rotation and translation.

Finally, Deneve and Pouget (1998) argue that the object-fixed effects observed in SEF may not be due to a purely object-centered frame of reference. They show that the neural responses in (Olson and Gettner 1995) are consistent with retinotopic tuning curves which are modulated by an object orientation gain field. Under this model, the difference between the representation of object-based saccades in LIP and SEF may be just one of degree: the object related signals in LIP being relatively weak while those in SEF are relatively strong.

Object orientation effects

Firing rates of the majority of neurons in our population during the OBJ-SACC and OBJ-FIX tasks appear to be modulated by the orientation of the object on the projection screen. The presence of these object orientation effects in our data might suggest that LIP codes for a visual variable that can not be derived solely from the retinotopic response fields of its cells. Object orientation is an important variable in the OBJ-SACC task, and so such a pattern of activity could reflect a novel, task-relevant visual signal in LIP. Such a finding would be reminiscent of the responses in area AIP that code for the three-dimensional shape of manipulable objects (Sakata et al. 1995, 1998). Non-retinotopic visual activity in area LIP was also reported by Sereno and Maunsell (1998), who found shape selective responses when monkeys were required to make saccades towards abstract two-dimensional objects. However, our experiment was not designed to identify true object orientation effects: the orientation of the object was confounded with both the portion of the visual field occupied by the target and the location of the potential targets on the screen. Furthermore, this interpretation of the object orientation effect is inconsistent with the systematic dependence of object orientation tuning curves on retinotopic tuning, as observed in Fig. 19. Rather, these results suggest that the dependence of firing rate on the orientation of the object is due the orientation-dependent location of the various parts of the object.

We have investigated whether this orientation dependent activity depends on the behavioral relevance of the object parts that overlap with cells' receptive fields. In the Sensory Stimulation model, object orientation effects are due to the overlap of any part of the object with a cell's receptive field. In the Potential Targets model, neural activity is affected only by the location of potential saccade targets. Our results favor the latter model, as the increase in activity due to the presence of the first object finger in the receptive field is greater than that due to the object handle, which was never a saccade target. In addition, the Potential Targets model is consistent with a decrease in the influence of non-target fingers just before the onset of the saccade, since by that point these other locations have been excluded as saccade end points. Prior studies using a visual search task have shown a similar temporal shift in activity from distractor objects to target objects in Macaque FEF (Bichot and Schall 1999; Bichot et al. 1996; Schall and Hanes 1993). In our task, such a transition would result in a diminution of the magnitude of the object orientation effects and a commensurate increase in the retinotopic effects, both of which were observed between the Delay and Saccade periods of the task. Our results therefore suggest that the object orientation effects result from the confound between the orientation of the object and the location of the potential saccade targets in the visual field.

Retinotopic coding

Prior studies of LIP have shown a retinotopic coding of saccades to simple targets such as spots of light on a homogeneous background (Barash et al. 1991). Our findings suggest that the neural activity during an object-based saccade task is still largely retinotopic. Furthermore, the alignment of the tuning curves in the standard memory saccade and object-based saccade tasks shows that this retinotopic coding is invariant across tasks. In summary, while the neural activity of some cells in LIP do show a small but significant modulation of activity due to the location of the saccade target in an object-fixed reference frame, LIP appears to use largely the same representations for our object-based saccade task as it does for making saccades to simple point targets.