INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Primates use vision to guide their interactions with the environment. In wakefulness, the brain generates a steady stream of decisions to shift the gaze, to position the body, and to grasp, avoid, or classify objects, often with the guidance of data from the visual cortex. Unless an action is purely reflexive or purely capricious, a higher level of information processing must link sensation to action. Sensory data must be interpreted to execute, revise, or delay pending action. The goal of this study is to investigate the neural underpinnings of one such interpretive mechanism: a simple decision process in a two-alternative, forced-choice psychophysical paradigm.

We trained monkeys to discriminate opposed directions of motion in a stochastic random dot display and to report the perceived direction with a saccadic eye movement to one of two visual targets. At least three processing stages must be engaged during each trial the monkey performs (Fig. 1). First, a sensory process must extract motion information from the visual image and represent the outcome within the visual cortex. For our task, the relevant representation of motion resides largely in areas MT and MST of extrastriate cortex (Britten et al. 1992, 1996; Celebrini and Newsome 1995; Croner and Albright 1999; Newsome and Paré 1988; Salzman et al. 1992; Shadlen et al. 1996). Neurons in MT and MST generate smoothly varying responses that reflect the amount of motion energy within a specific band of velocities (direction and speed) to which they are tuned (Albright 1984; Maunsell and Van Essen 1983; Simoncelli and Heeger 1998; Zeki 1974). Second, the map of motion direction in MT and MST must be interpreted, or read out, to form a categorical decision: is the net motion flow in direction A or direction B? Third, after a decision is formed, it may need to be stored in working memory until an operant response is programmed and executed. In our task, neural signals for guiding the operant response must ultimately emerge from eye movement-related structures such as the superior colliculus, the frontal eye field, and the lateral intraparietal area (LIP) of the inferior parietal lobe, areas that have been studied extensively over the past few decades (for reviews, see Andersen et al. 1992; Colby and Goldberg 1999; Schall 1995). We therefore have a reasonable base of knowledge concerning the sensory and motor processing stages that must be engaged during performance of the task, but we know virtually nothing concerning the key cognitive stage of decision formation (see also Romo and Salinas 2001).

View larger version (32K): [in this window] [in a new window]   Fig. 1. Stages of processing in the motion-discrimination task and their putatative neural correlates. Information about visual motion is represented in the extrastriate visual cortex. These neural signals inform a decision process, constrained by the demands of the task to 1 of 2 possible judgments. The judgment, once made, persists during the delay period that follows motion offset, ultimately informing the behavioral response. A neural correlate of the decision formation is not known, but several brain structures contain neurons that would be expected to sustain a representation of the animal's commitment to one of the possible behavioral alternatives. The central hypothesis of the present study, symbolized by the dashed arrow, is that such neurons might also lend insight into the computation of the decision itself.

As an initial step toward analysis of the decision process, we have studied the activity of neurons in LIP that carry high-level signals appropriate for identifying salient visual targets and ultimately for guiding saccadic eye movements. Many neurons in LIP modulate their level of activity when there is sufficient information to plan a saccade, even when execution of the saccade may be delayed by several seconds (Colby and Goldberg 1999; Mazzoni et al. 1996; Snyder et al. 2000). Our central question is whether the activity of these neurons can provide insight into the process of decision formation during performance of our psychophysical task. Differentiating decision-related activity from strictly sensory activity is reasonably easy. By requiring the monkey to discriminate weak, noisy motion signals near psychophysical threshold, we create a situation in which the decision varies from trial to trial for repeated presentations of the same motion stimulus (i.e., the monkey decides correctly on some trials and incorrectly on others). To a first approximation, sensory activity will reflect the motion in the stimulus irrespective of what the monkey decides, whereas activity in higher level circuits that interpret the motion signals should vary strongly with the monkey's decision.

Differentiating decision-related activity from strictly motor activity, however, is not so straightforward. In a trivial sense, all motor signals are decision-related in that they reflect the outcome of the decision process. The key problem is to differentiate processing stages in which the decision is actually formed and represented from stages that simply represent a movement to be executed. We have adopted two tactics to gain experimental leverage on this issue. First, we have introduced an instructed delay period between presentation of the motion stimulus and the "go" signal to execute the saccadic eye movement. This tactic delays overt motor activity until the end of the trial, thereby separating the period of motion viewing (hence the decision) from motor execution. Second, we have taken advantage of a fact that all psychophysical subjects know well: all decisions are not created equal. Subjects are certain of decisions made on the basis of strong sensory information but are quite doubtful of decisions made on the basis of ambiguous evidence. We assume that neural circuits intimately linked to the process of decision formation should reflect this level of certainty either in the amplitude or timing of decision-related activity (Basso and Wurtz 1998). In other words, decision-related activity should bear some signature of the intensity of the sensory stimulus.

We have found that some neurons in LIP are plausible candidates for participating in the decision process. These neurons generate sustained activity that predicts the impending saccade, and thus the monkey's decision. Both the amplitude and timing of this activity reflect the certainty of the decision and cannot be accounted for by any parameter of the eye movement itself that we have investigated.

We have briefly described some of these results elsewhere (Shadlen and Newsome 1996; Shadlen et al. 1994).

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Subjects, surgery, and daily routine

We performed experiments on two adult rhesus monkeys (Macaca mulatta, 1 male and 1 female) weighing 8-9 kg. The monkeys were surgically implanted with a head-holding device (Evarts 1968), a scleral search coil for monitoring eye movements (Judge et al. 1980), and a recording cylinder over the intraparietal sulcus. After recovery from surgery, the animals engaged in daily training or experimental sessions lasting 2-6 h. The monkeys were trained to perform a two-alternative, forced-choice direction discrimination task near psychophysical threshold. The monkeys were also trained on a variety of fixation and saccadic eye movement tasks as described below.

The monkeys worked for liquid rewards, and their daily water intake was therefore controlled. All surgical and behavioral procedures were in accordance with the U.S. Department of Health and Human Services (National Institutes of Health) Guide for the Care and Use of Laboratory Animals (1996).

Visual stimuli

Visual stimuli were generated on a PC/486 computer using a Pepper SGT⁺ graphics board (Number 9 Computer) attached to a Sony multiscan monitor (60 Hz noninterlaced) placed 57 cm away from the monkey. The system displayed fixation and saccade targets as well as the dynamic random-dot motion stimuli used for the direction discrimination experiments. The motion display was similar to stimuli used in previous investigations (e.g., see Britten et al. 1992).

Random dots were plotted within a circular aperture of 5-10° diam. Each dot was displayed for one video frame and then replotted 50 ms (3 video frames) later either at an appropriate spatial displacement for apparent motion (typically 3-7°/s velocity) or at a random location. The probability that a particular dot would be displaced in motion is termed the motion coherence, expressed throughout the paper as a percentage. For example, if the coherence is 50%, then a dot that appears in frame 1 has a 0.5 probability of coherent displacement in video frame 4 and an equal chance of being randomly replaced somewhere else in the viewing aperture. Dots that first appear in video frame 2 are not seen in frames 3 and 4 and are subsequently plotted with the appropriate displacement (or randomly) in video frame 5, and so on. The dots were white on a black background and plotted at a density of 16.7 dots per deg² per s, as in previous studies.

For some experiments, we used the same sequence of random dots for all trials at each coherence-direction combination. The manipulation did not lead to any detectable difference in the LIP response, and we have therefore combined these experiments with those in which a fresh random-number seed was used on every trial.

Electrophysiological recording

We recorded neural activity using tungsten microelectrodes (impedance 0.8-1.2 M at 1 kHz; FHC) inserted into the cortex through a 23-gauge stainless steel guide tube that punctured the dura mater. The tip of the guide tube was either in the superficial layers of area 7a or in the intraparietal sulcus, outside of the cortex. The guide tube was held in place by a plastic grid fitted inside the recording chamber (Crist Instruments). The grid enabled us to record from the same location along the bank of the intraparietal sulcus for several days.

Signals were amplified and viewed on an oscilloscope screen. Single units were isolated on the basis of voltage waveform using a voltage-time window discriminator (Bak Electronics). The time of each action potential was stored on computer disk to the nearest millisecond, along with the time of trial events that identified the time of fixation, stimulus onset, stimulus offset, and saccade. Records of eye position were stored to disk (250 samples/s) on a portion of the experiments. Data acquisition and experimental control were accomplished using a PC/486 running a real-time data acquisition system (Hays et al. 1982). The trial events, spikes, and eye position data were analyzed off-line using software tools developed in Matlab (The Mathworks).

Behavioral tasks

The primary goal of the study was to examine the responses of neurons during performance of a motion discrimination task similar to one used in previous investigations of areas MT and MST (Britten et al. 1992, 1996; Celebrini and Newsome 1994; Newsome and Paré 1988; Salzman et al. 1992). For the present study, neurons were selected on the basis of their responses during saccadic eye movement tasks, described below. For all tasks, the monkey was required to fixate a small red spot (the fixation point, FP) until its extinction. If at any time, the gaze fell outside of a 2 × 2° window centered on the FP, the trial was aborted. The window accommodated the small variation in eye position from trial to trial, but the monkey's gaze on any one trial was typically stable. A brief description of each task follows.

DELAYED SACCADES WITHOUT AND WITH MEMORY. Neurons were screened by their responses in a delayed saccade task. On fixating a central spot, a bright red saccade target appeared in the periphery. The monkey was required to maintain fixation until the fixation spot was extinguished and was then required to make a saccade to the target within 500 ms. The delay period between onset of the saccade target and offset of the fixation point ("go" signal) was randomized from 0.5 to 2.0 s. We sampled LIP using this task, searching for those neurons that discharged during the delay period.

MOTION DISCRIMINATION TASK. After delineating the boundaries of the RF, we set up a direction discrimination task after the design illustrated in Fig. 2. One target, henceforth called T1, was placed in the RF of the neuron under study, while a second target, T2, was placed well outside the RF (often in the opposite hemifield). The stimulus aperture was positioned so that the coherent dots moved toward one or the other target on each trial. We positioned the stimulus aperture so as to minimize stimulation of any visual receptive field.

View larger version (19K): [in this window] [in a new window]   Fig. 2. Motion discrimination task used to study neurons in the lateral intraparietal area (area LIP). The monkey performs a 1-interval 2-alternative forced-choice direction discrimination task. The difficulty of the task is controlled by varying the fraction of random dots that move coherently. The direction and motion strength are randomly chosen on each trial. The monkey is trained to indicate its judgment of direction by making an eye movement to 1 of 2 targets that appear to either side of the random-dot motion stimulus. A: task geometry. Neurons selected for study emit sustained responses during delayed eye movement tasks when the target appears in a portion of the visual field, termed the response field (RF; gray). The discrimination task is arranged so that the direction of random-dot motion instructs a subsequent gaze shift into or away from the RF. One target (T1) appears in the RF. The other target (T2) and the random dots are placed outside the RF. B: time diagram of events in the discrimination task. The motion-viewing period lasted 0.5, 1.0, or 2.0 s. FP, fixation point.

(1)

PASSIVE VIEWING OF RANDOM-DOT MOTION. We examined the response to random-dot motion during trials in which the monkey simply fixated. No saccade targets appeared on these trials, and the monkey was rewarded simply for maintaining fixation throughout the motion-viewing period. The dots appeared in the same location as in the discrimination task, outside the neuron's RF. The strength of motion was 51.2% coherence, which matched the strongest motion used in the discrimination experiments. This fixation task was often performed in a separate block of trials but was sometimes randomly interleaved with discrimination and delayed saccade trials. The task is the only one in which saccade targets do not appear shortly after fixation.

DELAYED SACCADES IN THE PRESENCE OF MOTION DISTRACTOR. This task examines the response to visual motion during preparation of a saccadic eye movement that is specified by a single target. The task resembles the discrimination task with the important exception that only one saccade target appears throughout the trial. The motion coherence was 51.2% and the direction was toward or away from the target. Both the direction of motion and the target location were randomized and independent. The monkey was rewarded for making a saccade to the one target. Importantly, the direction of motion had no bearing on the monkey's reward. This task was always performed in a separate block of trials to distinguish it from the discrimination task. Because this task potentially reinforces a dissociation between motion direction and eye movement response, we included it only after obtaining data on the other tasks.

Data analysis

Raw data were stored as spike events timed to the nearest millisecond. These responses were collated into trials along with various time markers to compute standard peristimulus time histograms and rasters, and to count spikes occurring between trial events. Analysis was performed off-line using custom software developed in Matlab (The Mathworks). Each of the intervals comprising our trials (from target onset to motion onset, from motion onset to offset, and from motion offset to the extinction of the fixation point) contained a variable amount of time. We therefore present our data with respect to different event markers (e.g., motion onset). To compute summary statistics, we used the average spike rate between two trial events or in epochs aligned to common trial events (e.g., 1st 500 ms of motion-viewing period).

We performed standard comparisons of means using t- and F-tests. When examining results across the population of neurons in our data set, we applied multiple regression models in which cell identity was incorporated as an independent categorical variable. For example, to analyze the effect of motion strength on neural response (Fig. 9, A-D) we fit the model

(2)

where y is the spike rate measured in a designated epoch on correct T1 choice (or correct T2 choices); COH represents motion strength;¹ I_unit serves as an indicator function (I_unit = 1 if unit = i and 0 otherwise; such variables are often referred to as a dummy variables); and  represents the residual error, which is assumed to obey a normal distribution. The fitted coefficient, , along with its confidence interval provides an estimate of the effect of motion strength on response across the 104 neurons, allowing for differences in level of activity among the neurons (as estimated by the fitted coefficients ₁ ... ₁₀₄).

To test whether saccade direction affects the response, we calculated the probability of obtaining an F-statistic under the null hypothesis, H₀:  = 0. The F-statistic is derived from the extra sum of squares obtained by fitting a reduced model in which  = 0 (Draper and Smith 1966). If there are m data points and n neurons, then for two models that differ by k = 1 degrees of freedom

(3)

where SS_full and SS_red are the residual sum of squares for the full model and the reduced model fits, respectively. For ease of presentation, we often show the mean response among a group of neurons, but all hypothesis tests were performed using multiple regression and the extra sum of squares principle. We refer to this procedure in the text as a nested F-test and describe null hypotheses by noting which coefficients are set to zero. Although some assumptions can be criticized, this regression strategy (and the variants we pursue in this paper) furnishes estimates and confidence intervals that reflect appropriately the differences in firing rates among neurons and differences in the degree of uncertainty that neurons contribute (based primarily on differences in the number of trials obtained).

ANALYSIS OF PREDICTIVE ACTIVITY. We computed a predictive index that describes the association between neural response and the monkey's decision. The index approximates the ability of the experimenter to predict the monkey's behavior from the neural response. It is the probability that a random sample of the neural response associated with one behavioral choice would exceed the neural response associated with the other behavioral choice. Denoting the response associated with the two choices by y₁ and y₂, this is the joint probability over all possible criteria, , of observing y₁ =  and y₂ <  

(4)

Equation 4 can be estimated by computing the area under a receiver-operating-characteristic (ROC) curve obtained from the two response distributions (Britten et al. 1992; Green and Swets 1966). We used an epoch of 250 ms to obtain the spike counts, y₁ and y₂.

Saccadic eye movements

For 45 neurons we maintained records of the monkey's eye position during discrimination and saccade trial types. Eye position was sampled at 1 kHz per horizontal and vertical channel and stored on disk at 250 Hz per channel. From these eye position traces we derived the beginning and endpoint of each saccade, its amplitude (AMP), direction (DIR), peak velocity (VMAX), duration (DUR), latency (LAT), and accuracy (ACC). We defined accuracy as the reciprocal of the RMS distance from the mean endpoint. We were interested in whether trial-to-trial variation in the saccadic eye movement affected the neural response.

Histology and identification of recording sites

The animals were killed with an overdose of pentobarbital sodium (Nembutal) and perfused through the heart with saline followed by a 10% Formalin fixative. Tissue blocks containing the region of interest were equilibrated in 30% sucrose, then cut in 48-µm sections using a freezing microtome. Sections at regular intervals through the intraparietal sulcus were stained for cell bodies with cresyl violet and for myelinated fibers by the method of Gallyas (Gallyas 1979). We confirmed that our recordings were from neurons in the lateral bank of the intraparietal sulcus. Figure 3 illustrates a typical histological section containing several electrode tracks. The guide tube was directed toward the lateral bank of the IPS (visible in adjacent sections), and electrode tracks from this guide tube coursed down the lateral bank for several millimeters before exiting into white matter. Although we cannot reconstruct individual penetrations made over the course of many months, it is clear that the bulk of our recordings were from the more posterior and medial region of LIP, corresponding to the region of LIP that projects to the frontal eye field and area 8Ar (Andersen et al. 1990; Cavada and Goldman-Rakic 1989; Petrides and Pandya 1984; Schall et al. 1995).

View larger version (78K): [in this window] [in a new window]   Fig. 3. Representative histological section of recording sites from one monkey. A: the approximate plane of section shown in B and C. B and C: low- and high-power micrographs (Myelin stain). Electrode tracks can be seen coursing through the posterior bank of the intraparietal suclus. The area in the rectangle is magnified in C. Arrows mark electrode tracks through area LIP. IPS, intraparietal sulcus; STS, superior temporal sulcus; LS, lunate sulcus.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Basic response properties on delayed saccade tasks

We recorded from 104 neurons in area LIP of 2 adult rhesus monkeys. All of the neurons included for analysis were active during a delayed saccade task and exhibited a clear preference for targets in a restricted portion of the visual field, termed the response field (RF; see METHODS). In nearly all cases we ensured that such delay-period activity did not represent a visual response to the saccade target by extinguishing the target after 200 ms and requiring the monkey to make a memory-guided saccade. Figure 4 illustrates such responses for one LIP neuron. The monkey made memory-guided saccades to eight test locations, which were arranged concentrically around the fixation point at an eccentricity of 10°. The response rasters are arranged concentrically in the figure to denote the saccade direction for each raster. The response was largest when the remembered target was to the left of fixation. When the target appeared outside the RF, the response was attenuated until after the saccade. The mixture of visual, delay-period, and perisaccadic responses apparent in these rasters has been described by other investigators (Barash et al. 1991a,b; Colby et al. 1996; Gnadt and Andersen 1988; Platt and Glimcher 1997). We used the delay period activity to guide placement of choice targets and random dots in the direction discrimination task.

View larger version (19K): [in this window] [in a new window]   Fig. 4. Neural activity associated with memory-guided saccadic eye movements. In this screening procedure, targets appeared in 1 of 8 locations. The responses are aligned to the onset of the target that was flashed on and immediately off. The monkey made an eye movement to its remembered location when the fixation point was extinguished (). Rasters and peristimulus time histograms (PSTHs) are arranged to illustrate the angle of the target. The PSTHs do not include activity after the initiation of the saccadic eye movement. This neuron exhibited activity in the delay period before saccadic eye movements to the left. The polar graph in the middle of the figure shows the response as a function of target location. The response is the average spike rate for the period from target offset until extinction of the fixation point.

Response during motion discrimination

Our primary goal was to ascertain how such neurons respond when the instruction for the saccade is a motion stimulus presented outside the neuron's RF. In this setting, a saccade into or away from the RF indicates the monkey's judgment of direction. We reasoned that the development of neural activity related to the animal's choice might yield insight into the neural underpinnings of decision formation within the cortex. An example from a typical experiment is illustrated in Fig. 5. The responses shown in the left column accompanied trials in which the monkey decided that motion was toward the RF and made a saccade to the corresponding target (T1). For all three motion strengths shown in the figure, the response increased during the motion-viewing period and remained elevated throughout the delay period. Compare this pattern of responses to those accompanying the opposite decision (right column). During the motion-viewing period, the response diminished and remained attenuated through the delay period until the monkey made its saccade to the target outside the RF (T2). For both choices, the largest response modulations occurred during the motion-viewing period, which is the interval in which the monkey must arrive at its judgment of direction. Importantly, the modulation apparent in Fig. 5 does not reflect the sensory stimulus per se: substantial choice-related modulation occurred on the 0% coherence trials, which contained no net motion (bottom row), and on error trials as well (see Fig. 11). Moreover the modulated activity levels persisted throughout the delay period, after the random dots were extinguished. The response seems to reflect the monkey's decision about direction, rather than the actual motion content of the sensory stimulus.

View larger version (43K): [in this window] [in a new window]   Fig. 5. Activity of a neuron in LIP during the motion discrimination task. The cartoon at the top indicates whether the monkey's behavioral response was an eye movement into or out of the response field (gray disk). For the nonzero motion strengths, motion direction is shown by the arrow in the circle. Rasters and PSTHs are shown aligned to 2 events. In the left portion of each axis, the responses are aligned to the onset of motion, which is then followed by a 2-s motion-viewing period. In the right portion of the axes, the delay period response is shown aligned to saccade initiation. This neuron modulated its activity early in the motion-viewing period and in accordance with the monkey's direction judgment and pending eye movement. Responses were more enhanced and more profoundly depressed when the motion strength was greater. Only correct trials are shown for the 12.8 and 51.2% coherent motion strengths.

In our paradigm, the monkey can plan an appropriate saccadic eye movement as soon as a decision is made about the direction of motion in the stimulus, raising the possibility that the activity of neurons like the one illustrated in Fig. 5 simply reflects preparation for moving the eyes. This possibility is reinforced by the fact that similar predictive activity has been seen during performance of this task in overtly oculomotor structures such as the frontal eye field (Kim and Shadlen 1999) and superior colliculus (Horwitz and Newsome 1999a).

Closer analysis of the data reveals, however, that neural activity in LIP cannot be explained entirely by motor preparation. The histograms in Fig. 5, for example, suggest that the predictive activity varied in intensity as a function of motion strength. The upper set of responses was obtained when the monkey viewed a strong motion stimulus. These trials were easy, and this is reflected in a rapid rise of activity early in the trial. The average spike rate during motion viewing was 39.2 ± 1.3 spikes/s (mean ± SE) for T1 choices and 13.9 ± 0.6 spikes/s for T2 choices. For the more difficult discriminations, shown at the bottom of the figure, the response modulation occurred later in the motion viewing period and never attained the level seen at the strong motion coherences (32.4 ± 1.3 and 17.5 ± 1.3 spikes/s during the motion-viewing period for T1 and T2 choices, respectively).

For the neuron in Fig. 5, decisions for motion away from the RF (T2 choices; right column) were accompanied by a suppression of activity that varied little across motion strengths. However, for many LIP neurons the effect of motion strength was more apparent for T2 choices than for T1 choices, as illustrated in Fig. 6. When the monkey viewed the 0% coherent display and chose the target outside the RF (bottom right raster and PSTH), the average response during the motion-viewing period was 12.4 ± 1.3 spikes/s. When a strong motion stimulus was directed away from the RF, correct T2 choices were associated with an average response of 7.6 ± 0.7 spikes/s (top right; P = 0.0012, t-test).

View larger version (56K): [in this window] [in a new window]   Fig. 6. Activity of another LIP neuron during the motion task. Conventions are the same as in Fig. 5. This neuron also modulated its activity during the motion viewing and delay period in accordance with the monkey's choice, but motion strength had only a modest effect on the degree of enhancement.

All of the neurons in our data set responded more strongly when the monkey decided that motion was toward the neuron's RF, and for most, this difference was evident during the motion-viewing period. For a few neurons (4 of 104), however, the response did not indicate the monkey's choice until the delay period; that is, after the random-dot motion stimulus was turned off. Figure 7 illustrates this unusual pattern of activity. This neuron responded selectively throughout the delay period of the saccadic eye movement tasks (A and B), but did not strongly indicate the monkey's decision during the motion-viewing period of the discrimination task. During the delay period, however, the response modulated in a manner that reflected the impending saccade and thus the monkey's decision (C and D). The change in firing rate became evident about 200 ms after the random-dot motion was turned off. We emphasize that this pattern of response was rare in LIP, although it occurs with some regularity in prefrontal areas that are connected to LIP (Kim and Shadlen 1999). The finding is important, however, because it demonstrates that selecting neurons based on their presaccadic activity did not guarantee that their responses would be modulated during the period of motion viewing.

View larger version (48K): [in this window] [in a new window]   Fig. 7. A neuron that fails to indicate the monkey's judgment during the motion viewing period. A and B: response during the delayed saccade task. In this control experiment, the target appeared either inside or outside the RF, at the same locations employed in the discrimination task (T1 or T2). The response is greater during the delay period preceding eye movements to the RF. Responses are aligned to target onset and arranged in order of trial duration. C and D: raster and PSTH from all correct choices using all nonzero motion strength stimuli. Responses are aligned to motion onset and arranged in order of the delay period duration. The neuron did not modulate its response until after the motion-viewing period. The response could be a neural correlate of an intended gaze shift or shift in attention, but the modulation occurs in the wrong time frame to reflect formation of a decision about motion direction. This neuron was exceptional.

The pattern of responses exemplified in Figs. 5 and 6 were representative of the population of LIP neurons encountered in this study. Figure 8 shows the mean response from 104 neurons plotted as a function of time, aligned to 2 events during the trial. On the left, activity recorded during motion viewing is aligned to the onset of random-dot motion; on the right, the activity recorded during the delay period is aligned to the monkey's saccadic eye movement. The solid curves were obtained from the trials in which the monkey judged motion to be toward the RF. Dashed curves reflect the opposite choice. Only correct choices were included in this analysis, except for the weakest motion strength (0% coherence, red), which provides no basis to distinguish correct from incorrect.

View larger version (26K): [in this window] [in a new window]   Fig. 8. Population response from 104 LIP neurons during the direction discrimination task. The average firing rate is plotted as a function of time during the motion-viewing and delay periods. Solid and dashed curves are from trials in which the monkey judged direction toward and away from the RF, respectively. Error trials are not shown. Both the time course and magnitude of the response are affected by the strength of random-dot motion, particularly during the motion-viewing period.

There are several interesting features in this graph. Like the single units in Figs. 5 and 6, the magnitude of the response reflects the monkey's choice, increasing for T1 choices and decreasing for T2 choices. The rise and fall in spike rate begins in earnest 175 ms after onset of the dots (P < 0.01, t-tests performed on 1st derivative) and continues throughout the motion-viewing period. For trials in which the monkey judges motion to be toward the RF, the increase in activity saturates during the delay period, culminating in a burst of activity just before and during the saccade. For judgments away from the RF, the responses reach an average attenuation of 4-5 spikes/s below baseline during the delay period.

This basic pattern of responses holds qualitatively for all motion strengths, but the traces differ in the exact time course and amplitude of the discharge. Stronger motion stimuli lead to more profound elevation/depression of the responses, and the modulation occurs earlier, on average, for stronger motion, particularly when it is toward the RF. These effects are more apparent during the motion-viewing period than during the delay period. By the time of the saccade, the response is nearly identical for all T1 choices, regardless of the motion strength that led to the decision. The same is true for all saccades to T2. At the time of the saccade, therefore, the average response simply reflects one or the other alternative.

We used a regression analysis to quantify the effect of stimulus strength on neural response (see METHODS, Eq. 2). Figure 9, A-D, illustrates the effect of motion strength on the mean spike rate obtained from four 1/2-s epochs that spanned the motion-viewing period. In each epoch, the response varied with motion strength, increasing when motion was toward the RF and decreasing when motion was toward T2 (). These effects were quite modest, especially in comparison to the overall differences in activity associated with T1 and T2 choices (e.g., compare  and  at any motion strength). The strongest effects were seen in the second epoch (Fig. 9B), where the response increased by 2.7 spikes/s on average with increasing motion strength toward T1 (95% CI = 2.0 to 3.6 spikes/s, P < 10¹⁴, nested F) and decreased by 4.2 spikes/s over the range of motion strengths toward T2 (CI = 3.6-4.7 spikes/s, P < 10¹⁵). The smaller effects seen in the other three epochs were also significant (P values range from 0.02 to 10¹²).

View larger version (32K): [in this window] [in a new window]   Fig. 9. Effect of motion strength on neural response. A-D: mean spike rate plotted as a function of motion strength in 4 1/2-s epochs during the motion-viewing period.  and , correct judgments of motion toward and away from the RF, respectively. Standard error bars are smaller than the circles. The lines are least-square fits to the data with motion strength transformed to an ordinal scale. Motion strength affected the response in all epochs, but the effect was most apparent during the 1st second of motion viewing. E and F: effect of motion strength on each neuron's response when variation in saccadic eye movement is also considered. The effect of motion strength is deduced from a multivariate regression model in which the average neural response during the motion-viewing period is approximated as a function of motion strength and variation in the saccadic eye movement response (Eq. 5). The histograms summarize the change in each neuron's average spike rate during motion viewing that was attributed to an increase in motion strength from weakest (0%) to strongest (51.2%) coherence. Shading denotes a significant effect of motion strength when factors related to variation in eye movements are incorporated (P < 0.01, nested F). Error trials were not included in this analysis.

The result suggests that LIP neurons do not simply encode the endpoint of a planned saccade but reflect through their discharge the quality of the sensory information that instructed the eye movement. However, this interpretation rests on the presumption that all eye movements to a visual target are identical, which is false. We therefore considered the possibility that eye movements varied with the difficulty of the task, and that this variation accounts for the change in neural response heretofore associated with the strength of random-dot motion.

We extended the linear regression analysis to incorporate various descriptors of the saccadic eye movements. The analysis was performed on a subset of the data consisting of 45 neurons (30 from monkey E, 15 from K) for which we had records of eye position. For each trial, we extracted six descriptors of the saccadic eye movement: latency, amplitude, direction relative to the target, accuracy, maximal speed, and duration. Across the 45 experiments, we found small but significant inverse variations of saccadic latency and saccade duration with stimulus strength (P < 10⁷ and P < 10⁴, respectively; nested F). The other four saccade descriptors were more variable in their association with motion strength, but in any given experiment one or more of these were often significant. We therefore included all of these factors along with motion strength in a multivariate regression analysis, fitting the model

(5)

where Y is the spike rate measured from the epoch of interest (e.g., the 1st second of motion viewing). The fit to Eq. 5 allows us to test whether motion coherence (COH) affects the neural response in a manner that cannot be accounted for by variation in saccadic eye movements. This is a test of the null hypothesis, ₁ = 0, which is evaluated using a nested F-test (see METHODS, Eq. 3). We fitted the model separately for each neuron and for the two saccade directions, omitting error trials (on average 130 trials per neuron per direction; range 21-459). We performed the regression on each neuron individually because there was no reason to assume that variation in saccade parameters would affect all cells in the same way (e.g., shorter saccades might lead to an increase or a decrease in response depending on the exact location of the target within a neuron's RF). Thus for each neuron we considered the possibility that one or more of the saccade descriptors would affect the response in a manner that could have masqueraded as a coherence effect. This concern turns out to be minor.

The histograms in Fig. 9, E and F, depict the change in response that accompanied an increase in motion strength from 0 to 51.2% coherence, after controlling for the potential confounding effect of eye movement variation (from the fit to Eq. 5). The result is comparable to the simple regression obtained for the whole data set (Fig. 9, A-D) in which we ignored variation in saccade metrics. On average, there was a 3.9-spike/s increase in response across the range of motion strengths toward the RF (95% CI = 3.0-4.8 spikes/s; P < 10¹⁵, nested F) and a 1.9-spike/s decrease in response for motion away from the RF (CI = 1.2 to 2.7 spikes/s; P < 10⁶). Individual neurons with significant F ratios (P < 0.01; H₀: ₁ = 0) are shown by the shaded portion of the histogram. In all cases, significant regressions revealed the expected relationship between motion strength and neural response: enhancement with stronger motion toward the RF and suppression with stronger motion away from the RF. We conclude from this analysis that variation in saccade metrics does not explain the response modulation accompanying variation in the strength of random-dot motion.

Neural reflection of behavioral bias?

Before motion onset, one might expect neural activity to be completely uninformative about the monkey's decision, but this is not so. Examination of Fig. 8 reveals that the response was slightly stronger before the monkey was shown motion that led eventually to a T1 choice, especially for trials with weaker motion stimuli (red and green curves). The difference in activity ranged from 2 spikes/s for the weakest motion (95% CI = 1.53-2.48 spikes/s) to 0.4 spikes/s at the highest motion strength (CI = 0.21-1.14 spikes/s; P < 0.01 for all but the 2 largest motion strengths, nested F). We interpret this early response modulation as a possible correlate of decision bias: a predisposition to choose T1 or T2 before viewing the motion stimulus (Basso and Wurtz 1998). When the monkey is biased in favor of a T1 choice, activity is stronger at the outset of the trial; when the bias favors T2, activity is smaller than average at the outset. Of course, such variation is likely to precede trials regardless of the strength of the ensuing motion. However, when the motion is strong, the direction of moving dots dictates the monkey's decision; trials beginning with a T1 or T2 bias end up distributed among both sets of correct choices. Conversely, when the motion strength is weak, the monkey's initial bias affects the outcome of the trial, with the result that more trials with an initial T1 bias actually end in T1 choices.

This scenario would produce the small differences in response preceding the onset of random-dot motion when the monkey makes T1 or T2 choices, but only when motion is weak. Firm conclusions about the source of these signals would require analysis of neural activity while behavioral bias is systematically manipulated. Such experiments, carried out recently by Platt and Glimcher (1999), have shown that signals related to behavioral bias indeed exist in LIP. We suspect that our data reflect the same underlying phenomena.

Predicting the decision

The data in Fig. 8 show that the activity of LIP neurons evolves in time, raising the question, when and how well do LIP neurons predict the monkey's choice? To address these issues, we performed an ROC analysis to compute an index of the neuron's predictive activity during the course of the discrimination. The index reflects the degree of separation between the responses associated with choices into and away from the RF and can be interpreted as a probability of correctly classifying a response as belonging to either choice set (see METHODS, Eq. 4).

Figure 10A plots for a single LIP neuron the predictive index as a function of time for five stimulus strengths. As the monkey viewed the random-dot motion, the neuron predicted the monkey's decision with increasing accuracy. This was also our impression during the recording experiments. While listening to the spike discharge over the loudspeaker, we experienced an increasing sense of confidence in predicting the monkey's decision as the trial progressed. By the end of the viewing period, the discharge from the neuron shown in Fig. 10A was nearly flawless in its predictive power, indicating that there was almost no overlap between the distributions of responses associated with T1 and T2 choices. During the delay period, the response remained highly predictive of the monkey's behavior, as evidenced by the curves on the right side of the plot. Although the curves in Fig. 10 bear resemblance to cumulative functions, the calculation is based only on spikes encountered within ±125 ms of the time indicated on the abscissa.

View larger version (21K): [in this window] [in a new window]   Fig. 10. Predictive power of the neural response. The ordinate on these graphs estimates the capacity to predict the monkey's choice from a 250-ms sample of the neural response, based on an ROC analysis (see METHODS). A: predictive activity computed from one neuron. The neural responses associated with T1 and T2 choices are initially similar, leading to chance association between neural response and the monkey's choice. During motion viewing, the predictive power increases such that by the delay period, the neural responses accurately reflect the impending choice. The time course is more rapid for the stronger motion strengths. B: average predictive power from 104 neurons in LIP. The average illustrates the dependence on motion strength.

The sigmoidal evolution of predictive activity was evident at all motion strengths, but the neuron became predictive sooner at the stronger motion strengths. This observation is better appreciated in the population averages, illustrated in Fig. 10B. For the easier discriminanda, LIP activity was more predictive of the monkey's decision, and the predictive activity emerged earlier in the trial. Consistent with the bias effect discussed in the preceding section, weak predictive activity was evident prior to onset of the motion stimulus for the two weakest motion strengths. The prolonged temporal evolution of activity during motion viewing suggests a process in which LIP neurons accumulate information toward a plateau state that can guide subsequent behavior.

Errors

An advantage of the threshold discrimination task is that it affords an opportunity to examine trials in which the monkey makes errors, thereby providing a natural dissociation between sensory instruction and behavioral response. When the monkey viewed weak motion stimuli, at or below psychophysical threshold, many choices were incorrect. Figure 11 shows an example of the responses obtained from one neuron on trials in which the monkey viewed 12.8% coherent motion, just above psychophysical threshold. The four plots form a contingency table: the top and bottom rows show the responses when the monkey chose the direction toward and away from the RF, respectively. The left and right columns reflect motion direction toward and away from the RF, respectively. Accordingly, the top left and bottom right plots represent trials performed correctly (Fig. 11, A and D), whereas trials in the top right and bottom left represent error trials (Fig. 11, B and C).

View larger version (44K): [in this window] [in a new window]   Fig. 11. Comparison of errors with correct discriminations at a near-threshold motion strength. The direction of 12.8% coherent random-dot motion was toward the RF in the left column of responses and away from the RF on the right. A: correct judgments of motion toward the RF; mean response (±SE) during the 2-s motion-viewing period was 40.2 ± 1.7 spikes/s (n = 40 trials). B: errors in which the monkey viewed motion away from the RF but chose the direction corresponding the target in the RF (T1); mean response = 26.9 ± 1.7 spikes/s (n = 15). C: errors in which the monkey viewed motion toward the RF but chose T2; mean response = 25.3 ± 3.7 spikes/s (n = 10). D: correct T2 choices; mean response = 15.0 ± 0.8 spikes/s (n = 35). For clarity, only 20 trials are shown in the rasters accompanying correct trials (A and D).

The data show that both the monkey's choice and the visual stimulus influenced the activity of this LIP neuron (P < 10⁷ for both effects, 2-way ANOVA with nested F-statistic, as in Eq. 3). The response was most profoundly modulated on correct trials, in which the monkey's choice and the direction of stimulus motion covaried (compare A and D). The two panels of error trials generated roughly equal responses that were intermediate between those in A and D, indicating that, near psychophysical threshold, behavioral choice and motion direction exerted roughly equal effects on the activity of this neuron during motion viewing. This pattern of responses lends further support to the notion that LIP encodes both qualities of the stimulus as well as the monkey's behavioral response.

The pattern of results illustrated for the single neuron in Fig. 11 was evident on a population basis as well. Because few errors occur when the motion cues are strong, we combined data across all experiments to accumulate a sufficient number of error trials for statistical analysis. The graphs in Fig. 12 show average responses aligned to the onset of random-dot motion and the moment of saccade initiation. The black curves illustrate responses when the monkey chose T1; the gray curves correspond to T2 choices. The solid curves represent correct choices; the dashed curves depict the error trials. When the motion strength was weak, responses were similar for correct trials and errors (Fig. 12A, COH = 3.2%). For intermediate motion strengths (B and C), however, the dashed curves fall between the solid curves. Neural activity in LIP remains correlated with the monkey's choice on error trials (i.e., is "predictive"), but the effect was smaller than for correct trials. Notice that the differences between correct and error trials persist until just before the saccadic eye movement.

View larger version (22K): [in this window] [in a new window]   Fig. 12. Comparison of errors with correct discriminations using different motion strengths. Averaged response from 104 neurons is shown during the motion-viewing and the delay periods, aligned to motion onset and saccade initiation, respectively. Black and gray curves depict trials in which the monkey judged motion to be toward or away from the RF, respectively. Dashed curves are error trials. Motion strength is weakest in A and strongest in E. Note that the number of error trials diminishes at the higher motion strengths.

At the two highest motion strengths, the pattern was different. At a coherence of 25.6% (~2 times threshold), the responses on the error trials were nearly indistinguishable, on average, for T1 and T2 choices. The discharge only became predictive of the monkey's impending eye movement during the delay period, about 500 ms before the saccade. At the strongest motion strength (51.2% coherence) the order of the curves reversed: the response was stronger when the monkey erroneously chose the target outside the RF. It is as if the neuron was reporting the proper choice (the direction of stimulus motion), but the monkey changed its mind late in the trial, perhaps as a result of a distraction or lapse of attention. Note that these last curves represent a small number of trials (T1 errors: 122 of 2,567 trials; T2 errors: 64 of 2,433 trials).

One further detail deserves mention. Notice that at the higher motion strengths, average neural activity predicts the monkey's errors in the period before the random dots are shown (Fig. 12, D and E, dashed lines). We noted a similar effect in Figs. 8 and 10 for correct choices at the weakest motion strengths, which we interpreted as a neural correlate of the monkey's behavioral bias state. The fact that the same effect is apparent for error trials, and most strikingly at high coherences, suggests that the monkey's bias might have influenced the monkey's erroneous choices. This is the expected pattern of results if an appreciable fraction of errors at high coherences are explained by lapses (e.g., distraction) and a tendency to default to the current bias state.

Motion sensitivity

The pattern of activity observed on correct and error trials demonstrates that both visual stimulus motion and eye movement direction influence the activity of LIP neurons. To determine whether the visual discriminanda alone activate LIP neurons, we recorded responses to strong motion stimuli from 93 neurons while the monkey performed a passive fixation task. As in the discrimination task, the direction of motion was either toward or away from the RF, and the random dots appeared in an aperture outside the nominal RF. No targets appeared on these trials, however, and there was no delay period. The monkey received a liquid reward simply for maintaining fixation throughout the stimulus presentation period.

We often observed a weak response to the random-dot motion stimulus that was slightly stronger for motion toward the RF, as illustrated in Fig. 13, A-C, for a typical cell. Figure 13A shows that weak, directionally biased responses occurred in a block of fixation trials obtained before the monkey performed the discrimination task (mean response = 5.3 ± 0.5 spikes/s vs. 2.5 ± 0.4 spikes/s, P < 0.0005, t-test). We computed a direction index (DI) for each cell using the convention

(6)

The DI for the block of trials preceding the discrimination was 0.53, demonstrating a moderate bias. In a second block of trials, the same passive fixation conditions were randomly interleaved among motion discrimination trials. In this behavioral context, the responses to passive motion were stronger and more directional, as shown in Fig. 13B (7.3 ± 1.4 vs. 2.4 ± 0.4 spikes/s; DI = 0.66; P < 0.003). We also obtained a third block of fixation trials after the block of interleaved fixation and discrimination trials. As illustrated in Fig. 13C, responses were typically reduced and directionality was slightly weaker (2.2 ± 0.4 vs. 0.8 ± 0.2 spikes/s, DI = 0.63; P < 0.006). These trends were evident in the population. Figure 13D shows the mean responses for all neurons tested in this manner. The difference in mean responses of ~3 spikes/s observed in passive fixation trials (Fig. 13D, Before) is about one-half the response modulation attributable solely to motion strength during discrimination trials (e.g., a net change of 2.7 + 4.2 = 6.9 spikes/s for the 2 directions shown in Fig. 9B). This result lends additional support to the hypothesis that neurons in LIP reflect visual inputs as well as signals related to motor preparation. However, the fact that the "visual" responses were stronger and more directional when the fixation trials were interleaved with direction discrimination trials (Fig. 13D, During) suggests that covert motor planning can augment these responses. Although we rarely observed any overt saccades on fixation trials, the monkey presumably has more of a tendency to associate visual stimulus direction with a specific saccade in the context of an experimental block dominated by discrimination trials.

View larger version (25K): [in this window] [in a new window]   Fig. 13. LIP neurons show a weak, context-dependent, direction bias during passive viewing of random-dot motion. A-C: response from a single neuron to 51.2% coherent motion toward or away from the RF. The rasters and PSTH depict the response during the 2-s viewing period. The cell's RF was centered at 10° eccentricity; random dots were shown in a 2.5° radius aperture centered at the fixation point. We obtained data for this cell in 3 separate blocks: A, before the monkey performed the discrimination task; B, randomly interleaved with discrimination trials; and C, after the discrimination block. D: average response to passive viewing for all neurons tested in this manner (n = 75, 100, and 48 for before, during, and after). Dark bar, mean ± SE for motion toward the RF; light bar for motion away from RF.

In 20 neurons, we performed an additional control experiment to dissociate further the directional (sensory) response from oculomotor preparation. Rather than releasing control of oculomotor planning (as on the passive fixation trials), we instructed the monkey to prepare an eye movement that was unrelated to the moving random dots. In this block of trials, a single saccade target was presented at the beginning of each trial, either inside or outside the RF. On one-third of the trials, the monkey simply executed a delayed saccade to this lone target. On the remaining trials, random-dot motion was shown outside the RF for 1-2 s, but its direction was unrelated to the location of the saccade target. The monkey was thus encouraged to ignore the random dots and to make an eye movement to the location of the single target. As before, motion was toward or away from the RF, and the target appeared either inside or outside of the RF. These trials differed from the discrimination trials in two ways: the motion strength was always strong (51.2% COH), and only one saccade target was present.

Figure 14 illustrates results for a typical neuron. In the top row (A and B), a single target appeared in the RF, resulting in a vigorous response in anticipation of the saccade. The response was weaker, however, on trials in which motion was directed away from the RF (means: 38 ± 1.5 vs. 32 ± 1.8 spikes/s, P < 0.04, t-test). When the target appeared outside the RF (C and D), the response was suppressed, but to a greater degree when motion was away from the RF (6.7 ± 1.1 vs. 3.4 ± 0.7 spikes/s, P < 0.02). Clearly, the response of this neuron was dominated by the direction of the pending saccade, but the activity level also depended weakly on motion direction. This pattern was evident over the population of 20 neurons tested in this manner, as shown by the scatter plots (Fig. 14, E and F) and average response functions (Fig. 14G). Before the onset of the dot motion, the response was determined by the location of the target, inside or outside the RF. After motion onset, however, the response was affected by the direction of motion. On trials in which the target was inside the RF, the neurons responded less vigorously for motion away from the RF (Fig. 14G, dashed black curve) than for motion toward the RF (solid black curve). The overall level of responses decreased substantially on trials in which the saccade target was outside the RF, but the responses were nevertheless greater when motion was toward the RF than when it was away. The effect of motion direction was evident until just before the saccadic eye movement (arrows).

View larger version (22K): [in this window] [in a new window]