| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Department of Physiology, Centre for Neuroscience Studies and Canadian Institutes of Health Research Group in Sensory-Motor Systems, Queen's University, Kingston, Ontario, Canada
Submitted 23 June 2008; accepted in final form 23 July 2008
 |
ABSTRACT |
|---|
|
|
|
INTRODUCTION |
|---|
|
; Schall 2001; Schultz 2006). When a saccade is initiated and where it is directed can be modeled by determining the region that first surpasses a threshold level of activity on topographically organized maps of potential saccade goals (Trappenberg et al. 2001; Usher and McClelland 2001; Wilimzig et al. 2006). This selection process has been examined predominantly under conditions in which deterministic strategies were optimal; that is, either sensory cues or differences in expected payoffs indicated the "correct" choice for maximizing the intake of reward. Under these conditions, the quality of sensory instruction influences the rate at which activity rises toward threshold (Hanes and Schall 1996; Shadlen and Newsome 2001), whereas differences in expected payoffs predisposes baseline activation in favor of one of the potential actions (Carpenter and Williams 1995; Dorris and Munoz 1998; Ikeda and Hikosaka 2003).
In competitive situations, however, there often is no single correct action and instead a mixed-strategy is required to prevent opponents from exploiting predictable play (Fundenberg and Tirole 1991). During repeated play of "rock-paper-scissors," for example, dynamic interactions between the two players drives the game toward a mixed-strategy equilibrium. For this game, the predicted equilibrium strategy is for each player to choose the available actions in equal proportions and stochastically from trial to trial (Nash 1950). Such a response pattern suggests that players perceive the relative desirability of each action, on average, as equal. Equal desirability is further inferred from the fact that there is no incentive for varying from this equilibrium strategy and that if a player perceived one action as more desirable, they presumably would choose that action exclusively. Debate has arisen over what "drives" the selection process on individual trials if, overall, subjects are indifferent between the available actions and there are no instructive sensory cues in which to guide behavior (Aumann 1985; Rubinstein 1991). Understanding the neural processes leading to stochastic action selection has broad significance because a wide range of interpersonal, geo-political, pursuit-evasion, evolutionary, and competitive sports relationships require such mixed strategies (Driver and Humphries 1988; Maynard Smith 1982; Miller 1997; Palacios-Heurta 2003; Shinar et al. 1994; Smith 1982; Walker and Wooders 2001).
Although optimal strategies have been outlined in game theory (Fundenberg and Tirole 1991; Nash 1950) and behavioral economists have introduced descriptive models that approximate human choice patterns (Camerer et al. 2002; Erev and Roth 1998), only recently have the neural processes underlying mixed-strategy decision-making been examined. These neurophysiology experiments have focused on evaluative processes involved in assessing the outcome of past actions and rewards and in representing the desirability of choice stimuli (Barraclough et al. 2004; Cohen and Ranganath 2007; Dorris and Glimcher 2004; Seo and Lee 2007). Little is known, however, about the processes immediately preceding action selection during mixed-strategy tasks. The current experiments use a behavioral measure of motor preparation to gain insight into the spatial and temporal allocation of mixed-strategy action selection.
We put forth two possible processes, based on previous economic and neurophysiology considerations, which could lead to mixed-strategy action selection. First, the unbiased preparation hypothesis posits that selection processes associated with competing actions are equivalent and this leads to unbiased motor preparation. If, on average, actions are equally desirable over many trials of a mixed-strategy game, then it is plausible that actions are equally desirable on individual trials as well. In support of this, no bias is observed in the activity related to the selection and preparation of upcoming actions when the value of those actions are equivalent during the period of uncertainty preceding instructive sensory cues (Cisek and Kalaska 2005; Dorris and Munoz 1998; Platt and Glimcher 1999).
The second biased preparation hypothesis stems from an alternative theoretical perspective. Although actions may be equally desirable on average during mixed-strategy equilibria, slight differences may exist in their desirability from trial to trial (Harsanyi 1974). Neurophysiological evidence in support of this hypothesis comes from free-choice experiments in which neuronal activities associated with choice alternatives are predictive of upcoming actions (Coe et al. 2002; Dorris and Glimcher 2004; Sugrue et al. 2004). Moreover, under conditions of equal desirability, motor preparation signals are influenced by the previous sequence of events (Dorris et al. 2000). These small, initial trial-by-trial differences in selection processes could evolve to bias the allocation of motor preparation in advance of choice stimuli.
To test between these two possibilities, humans played an oculomotor version of the mixed-strategy game "matching pennies" using saccadic eye movements to indicate their choices. The otherwise covert process of saccade preparation was probed by the ability of visual distractors presented in advance of the choice targets to trigger erroneous saccades known as oculomotor captures (Theeuwes et al. 1998). The use of oculomotor captures is similar in its logic to how electrical microstimulation of the visuosaccadic circuitry has been used to examine the ongoing formation of perceptual-based decisions (Gold and Shadlen 2000, 2003). In those experiments, the deviation in the endpoints of electrically evoked saccades provided an instantaneous read-out of the degree to which developing perceptual decisions shaped activity patterns in brain regions responsible for producing the appropriate motor response. Here the type of saccadic response following noninvasive visual distractors provided the instantaneous read-out of the degree to which ongoing selection processes shape saccade preparation. Specifically, the probability that a distractor evokes an oculomotor capture and, when it fails to do so, the probability of choosing each of the subsequent targets are indicative of the level of saccade preparation at the time and location at which the distractor is presented (Dorris et al. 2007; Milstein and Dorris 2007). Unequal allocation of either oculomotor captures or subsequent target choices would provide support for the biased preparation hypothesis. Equal allocation of these behavioral measures would provide support for the unbiased preparation hypothesis.
Importantly, we are not trying to make the claim that mixed-strategy actions are selected exclusively by the brain's motor structures. Although oculomotor captures measure underlying saccade preparation most directly, other cognitive processes, such as those involved in decision-making on a higher level of abstraction and in visuospatial attention, are also likely important for this selection process. The use of oculomotor captures takes advantage of the fact that potentially diverse cognitive processes involved in this selection process must ultimately be consolidated to produce a single action. Therefore our claim is that oculomotor captures provide insight into how the summative selection process shapes the spatial and temporal allocation of saccade preparation.
|
|
METHODS |
|---|
|
). All procedures were approved by the Queen's University Human Research Ethics Board. Subjects were seated in front of a computer monitor with their heads stabilized on a chin rest positioned 59 cm from the center of a 17-in CRT monitor (refresh rate: 100 Hz) that spanned 32° of their central visual field. Left eye position was recorded at 250 Hz with resolution of 0.1° using an infra-red eye tracker system (Eyelink II, SR Research). Real-time data-acquisition software (Gramalkn, Ryklin Software) was used for stimuli presentation and data collection. Data analysis was performed off-line using MATLAB, version 7.04 (Mathworks) on a Pentium 4 personal computer.
Each task consisted of randomly interleaved standard trials (Fig. 1A, 75%) and distractor trials (Fig. 1B, 25%). Subjects were required to maintain central gaze fixation throughout the 800-ms presentation of the fixation point and after its removal during a fixed 600-ms warning period. Subjects indicated their choice by directing a saccade to one of two targets presented simultaneously 8° right and left of center. The fixed warning period and known target locations facilitated advanced saccade preparation (Dorris and Munoz 1998). Following choice selection, a red box appeared around one of the targets for 400 ms, indicating the computer opponent's choice (see following text). Each trial ended with a central display of the monetary payoff that lasted 1,000 ms followed by a 1,000-ms intertrial interval.
|
) despite instructions not to look to the distractor and reward being withheld for doing so. Mixed-strategy task
Subjects competed in an oculomotor version of the game matching pennies against a dynamic computer opponent (Fig. 1). This opponent performed statistical analyses on the subject's history of previous choices and payoffs to uncover systematic biases in their choice strategy (see algorithm 2 from Lee et al. 2004 for specific details). If both players chose the same target, the subject won $0.04; otherwise no monetary reward was received. Importantly, subjects were fully informed of the rules of the game and that they were playing a strategic game against a dynamic, competitive computer opponent.
To examine how task timing influenced strategic responses, all subjects also performed a version of the mixed-strategy task in which the fixed warning period was extended from 600 to 1,200 ms. Under this condition, distractors were equally likely to be presented at 100, 300, 500, 800 and 1,100 ms into the 1,200-ms warning period.
Pure-strategy task
The pure-strategy task (Fig. 1) was identical to the mixed-strategy task except the computer opponent always chose the same target for an entire block of trials, and the payoff was reduced from $0.04 to $0.02 per trial. The rewarded direction alternated between the left and right targets across blocks of trials.
INSTRUCTED TASK. The instructed task was also identical to the mixed-strategy task except that a single saccadic target was presented on each trial with equal probability to the left or right. Subjects received $0.02 for successfully acquiring the target with a saccade.
Data analysis
Subjects came in on four separate days and completed four 220-trial blocks of each task. The first day was a practice session, and these data were not included in the final analysis. Only 7 of the 10 subjects completed the pure-strategy task. The first 20 trials of each block were discarded from analysis to allow subjects time to adjust to the new task conditions.
A correct saccade was defined as the first saccade initiated between 120 and 350 ms after target onset that landed within 3° of the target. An oculomotor capture was defined as a saccade initiated between 70 and 220 ms after the distractor onset that landed within 5° of the distractor location. The temporal and spatial constraints were relaxed slightly for distractor-triggered oculomotor captures because these saccades are known to be of shorter latency and hypometric relative to slower correct saccades (Milstein and Dorris 2007; Theeuwes et al. 1999).
The degree of response stochasticity was quantified using entropy (Lee et al. 2004). Entropy describes the measure of uncertainty about the state of the observed system. The entropy, H, can be calculated as follows
 |
, whereby pk is the probability of finding a sequence k in the array of subject's outcomes. If behavior is truly stochastic, each of these sequences should be represented with equal probability corresponding to maximum entropy of 3 bits.
We subjected data from distractor trials to a three-way repeated-measures ANOVA with task type, distractor timing, and the target response direction as factors. A 
2 test was used to test whether there were differences in proportions of responses to distractors presented at different times during the warning period. To test whether the proportion of responses increased or decreased we compared the first (i.e., 100 ms) and last (i.e., 500 ms) time of distractor presentation using 95% confidence intervals for populations of proportions (Utts and Heckard 2007).
|
|
RESULTS |
|---|
|
The first results section analyzes the pattern of distractor-directed oculomotor captures to determine how motor preparation was temporally allocated in advance of strategic response. First, to establish that distractors were effective probes of underlying saccade preparation, the pattern of oculomotor captures was examined during the pure-strategy task in which the planned saccade was known with near certainty. During a representative block, subjects chose the rewarded target exclusively during standard trials (Fig. 2A) and, importantly, only distractors presented at the rewarded location triggered oculomotor captures (Fig. 2B). In total, subjects chose the rewarded target 99.5% of the time during standard trials, and oculomotor captures were directed almost exclusively toward distractors flashed at rewarded rather than unrewarded locations (Fig. 3A; 2 test, P < 0.001). This conjunction between the direction of saccadic targets and oculomotor captures suggests that distractors effectively probed the spatial allocation of saccade preparation.
|
|
). Overall, the percentage of saccades directed to each target did not differ from 50% during these two task (instructed: 49.6 ± 0.9% left, t-test, P > 0.05; mixed-strategy: 48.9 ± 0.8% left, t-test, P > 0.05) nor did they differ between these two tasks (t-test, P = 0.55). To measure stochasticity, we calculated entropy in patterns of choices (see METHODS). Instructed and mixed-strategy tasks both approached maximal entropy of three bits of information (instructed: 2.95 ± 0.007% bits; mixed-strategy: 2.94 ± 0.009% bits; t-test, P > 0.05). Consequently, any differences that exist between these two tasks during the following analysis of distractor trials cannot be accounted for by any differences in overall response patterns but presumably reflect the instructed versus voluntary nature of these two tasks. During the instructed task, oculomotor captures were directed to both left and right distractors; this is consistent with the uncertainty of the upcoming target location (Fig. 2D). Overall the pattern of oculomotor captures did not differ toward left or right distractors (not shown; P > 0.05). In addition, the overall percentage of oculomotor captures increased as distractors were presented later in the warning period (Fig. 3C; ANOVA, P < 0.001).
Having established the pattern of oculomotor captures when the upcoming response was nearly certain (the pure-strategy task) and uncertain (the instructed task), we now examine the pattern of oculomotor captures during the mixed-strategy task. Like the two other tasks, the percentage of oculomotor captures increased when distractors were presented later in the warning period (Fig. 3E; P < 0.001). Moreover, the pattern of oculomotor captures did not differ toward left or right distractors (not shown; P > 0.05). Together, this pattern of oculomotor captures suggests that during the mixed-strategy task saccade preparation begins in advance of target presentation and increases as the time of target presentation approaches.
Increasing the warning period delayed the time course of saccade preparation (Fig. 4). Initially (i.e., 
300-ms distractor presentation), the percentages of oculomotor captures did not differ between short and long warning period blocks but these percentages diverged when distractors were presented later in the warning period (i.e., 500-ms distractor presentation; P < 0.01). By the end of each of the respective warning periods, the percentage of oculomotor captures reached similar levels.
|
, 2007; Godijn and Theeuwes 2002). Spatial preparation of mixed-strategy responses: target-directed saccades
The analysis of oculomotor captures revealed that during the mixed-strategy task saccade preparation increased as the time of target presentation approached. It is important to recognize that the percentage of oculomotor captures indicates how saccade preparation is allocated probabilistically across blocks of trials rather than on a trial-by-trial basis. Equal proportions of left and right oculomotor captures during the mixed-strategy task could arise if saccades were prepared equally in both directions on every trial (i.e., unbiased preparation hypothesis) or biased in a particular direction on every trial (i.e., biased preparation hypothesis). For the latter to be the case, however, the direction of the planned saccade must vary stochastically from trial to trial. An analysis of target choices during distractors trials follows, whose purpose is to distinguish whether saccade preparation is spatially allocated in a biased or unbiased manner.
This section analyzes the choice of targets on those distractor trials in which the distractor failed to trigger a saccade. The following logic is critical for establishing this analysis: saccades planned stochastically either toward the left or right, coupled with distractors presented equally likely at either location, results in distractors that are 50% likely to be presented at the same location to which a saccade is being prepared on any given trial. Therefore target-directed saccades that occur during distractor trials will be classified as (Fig. 1B): 1) those directed toward the target on the same side as the distractor (Tsame), and 2) those directed toward the target on the side opposite the distractor (Topp). The assumption underlying this analysis is that the selection process continues to culminate toward choosing the target that would have occurred had the distractor not been presented.
To test this assumption, target-directed saccades were first analyzed during the pure-strategy task. During the representative block, the subject chose the rewarded target regardless of where the distractor was presented (Fig. 2B, black traces), and this observation held for over 97% of all trials across subjects. This finding suggests that ongoing saccade preparation processes continued largely undisturbed on those trials in which the distractor failed to trigger an oculomotor capture.
Target choices during the pure-strategy task were then analyzed relative to distractor location rather than reward location (Fig. 3B). The percentage of Topp did not vary with the time of distractor presentation (2 test, P > 0.05) remaining near 50% throughout the warning period (Fig. 3B, —). This pattern arose because oculomotor captures were successfully withheld during the 50% of trials in which the distractor was presented on the unrewarded side (Fig. 3A, 
) and saccades were subsequently directed toward the target on the opposite, rewarded side. The percentage of Tsame, however, decreased as the time of target presentation approached (Fig. 3B, - - -; 2 test, P < 0.001). This pattern arose because as saccade preparation toward the rewarded target increased during the warning period, distractors presented on that same side were more likely to trigger an oculomotor capture. Importantly, conducting this target-directed analysis under conditions with a known saccade goal establishes how Topp and Tsame differ when saccade preparation is biased toward one location.
There was no a priori reason that saccade preparation should be spatially biased during the instructed task because the target was equally likely to be presented in either direction. Consistent with this, the percentages of both Tsame and Topp decreased as the time of target presentation approached (Fig. 3D, Tsame, P < 0.001; Topp, P < 0.05) and these curves did not differ significantly from each other (P > 0.05).
Target choices following distractor presentation during the mixed-strategy task more closely resembled that of the pure-strategy task than the instructed task; the percentage of Tsame responses decreased (2 test, P < 0.001), whereas the percentage of Topp responses did not vary with the time of distractor presentation (2 test, P > 0.05; Fig. 3F). Importantly, the Topp and Tsame curves significantly differed at the end of the warning period, suggesting a bias existed in favor of one response just before the target presentation in the mixed-strategy task (Fig. 3F; P < 0.001).
To describe the overall effects, we subjected data from distractor trials to a three-way repeated-measures ANOVA with task type, distractor timing, and target response direction (Tsame vs. Topp) as factors. There was an effect of the task type (P < 0.0001), distractor timing (P < 0.0001), and target response direction (P < 0.0001) on the proportion of target-directed saccades following distractor presentation. There were significant interactions between task type and distractor timing (P < 0.05), task type and target response direction (P < 0.01), and distractor timing and target response direction (P < 0.05). Last, to differentiate saccade preparation processes underlying the instructed and mixed-strategy tasks, we conducted another three-way repeated-measures ANOVA that included only these two task types. We found a significant interaction between task type and target response direction (P < 0.005). This final statistical interaction is of particular importance because it highlights the differences in saccade preparation processes that exist between the instructed and mixed-strategy tasks.
|
|
DISCUSSION |
|---|
|
Temporal and spatial allocation of saccade preparation preceding mixed-strategy responses
This oculomotor version of matching pennies mimics many features of the more familiar manual version of the game (a.k.a., "odds-evens," "matching," or "choosing"). In the manual version, players indicate their choices by displaying either one or two fingers. Before game play, players must determine who will fulfill the role of odds and who will be evens. Odds wins if the players display different numbers of fingers (i.e., 1/2 or 2/1), and evens wins if the players display the same number of fingers (i.e., 1/1 or 2/2). To facilitate simultaneous presentation, players must also agree on the number of "primes" or fist-pumps that will be required before displaying their manual choices. The fixed warning period aids subjects in initiating their saccade choices within a narrow temporal window (Munoz et al. 2000) (i.e., 120–350 ms after target presentation) and thus performs a role analogous to priming. Oculomotor captures increased as distractors were presented later in the warning period (Fig. 3), and this increase was delayed in time when the duration of the warning period was extended (Fig. 4). Both of these observations suggest that motor preparation peaked coinciding with the predictable timing of the target stimuli. This makes sense from a strategic standpoint; maintaining motor preparation at a high sustained level could lead to premature responses thus providing one's opponent with a strategic advantage, whereas maintaining motor preparation at a low sustained level could result in delayed responses and disqualification from that round of play. Recently described behavioral and neural effects associated with predictably timed responses (Ding and Hikosaka 2007; Janssen and Shadlen 2005; Maimon and Assad 2006) likely played a role in the timing of the strategic responses described here.
The predictable target locations associated with this oculomotor matching pennies game also enabled us to examine the spatial allocation of motor preparation. This spatial analysis may not have been feasible with complexly encoded hand configurations typical of the manual version of the game. Moreover, probing with visual distractors provided insight into the spatial allocation of saccade preparation that would otherwise be difficult given the stochastic nature of choice selection during mixed-strategy games. Specifically, the analysis of target choices following successfully withheld oculomotor captures rests on the underlying assumption that the same target would have been selected had the distractor not been presented. That subjects overwhelmingly continued to choose the rewarded target under the predictable pure-strategy task partly bears this assumption out.
Overall the pattern of target choices following distractors (Fig. 3) shows that during the mixed-strategy task, preparatory processes became biased in favor of one response before the presentation of target stimuli. This locus of saccade preparation varied stochastically from trial to trial. The extent to which preparatory processes were spatially biased was not as pronounced preceding strategic responses as when the rewarded saccade was known with full foreknowledge [i.e., compare the separation in the Topp and Tsame curves for the mixed (Fig. 3F)- and pure-strategy (Fig. 3B) tasks]. The differing time course and extent of saccade preparation bias may reflect that ongoing deliberation takes place during the mixed-strategy task compared with early commitment during the pure-strategy task.
The current results are important because they suggest that accumulation-toward-threshold models, which have been so successful in describing simple perceptual decision-making, may also be applicable to the formation of strategic decisions. The rate of activation accumulation preceding perceptual decisions is largely driven by the quality of sensory evidence (Hanes and Schall 1996; Shadlen and Newsome 2001), whereas baseline activation is largely shaped by prior information such as the probability or magnitude of reward (Dorris and Munoz 1998; Ikeda and Hikosaka 2003). During the mixed-strategy task, however, both of these selection factors are lacking. First, there is no sensory evidence indicating the correct choice; in fact, subjects are situated in the dark during the warning period. Second, subjects are presumably indifferent on average between available responses because their overall payoffs are equal once the mixed-strategy equilibrium is established (Nash 1950). We speculate on two mechanisms that could lead to selection biases proceeding stochastic responses. One possibility is that noise in the accumulation signal randomly biases the selection process in favor of one of the options, which, coupled with local recurrent excitation and distal competitive inhibition, further strengthens this selection path over time (Dorris et al. 2007; Wong and Wang 2006). Another possibility is that the recent history of responses and rewards subtly biases subsequent selection processes. In this case, a player would never actually be indifferent to the available responses because of small perturbations in subjective desirability between the options from trial to trial (Harsanyi 1974). It was not feasible to examine this possibility in the current data set because interleaved distractor trials prohibitively disrupted the natural sequence of events. However, previous studies examining the choice patterns during mixed-strategy games suggests that the previous history of choices and rewards may indeed have a subtle effect on selecting upcoming choices (Corrado et al. 2005; Dorris et al. 2000; Lau and Glimcher 2005; Lee et al. 2004; Thevarajah and Dorris 2007). Of course, following a "win-stay/lose shift" strategy too strictly would lead to predictable behavior that could be exploited by one's opponent.
Outstanding issues
Although our results are consistent with the gradual accumulation of preparatory activation toward one saccade threshold over another, they could also be achieved if the level of activation increased abruptly on each trial (i.e., a "Eureka!" transition). The pattern of increasing oculomotor captures could arise from this latter mechanism if such step transitions occurred more frequently as the time of the sensory trigger approached. Oculomotor captures only describe the probabilistic time course of saccade preparation across blocks of trials. Direct neuronal recording within visuosaccadic structures will be required to distinguish whether there is a gradual or step transition in preparation signals during single trials.
Finally, it is worth reiterating that oculomotor captures more accurately reflect the temporal and spatial allocation of saccade preparation rather than decision or attention processes. Under some conditions, the formation of perceptual decisions appear to share a common neural substrate with preparatory signals in brain regions involved in generating the appropriate motor action (Gold and Shadlen 2000, 2003; Juan et al. 2004). Unlike perceptual decisions that require the decoding of immediate sensory evidence, strategic decisions can be made well in advance of stimulus presentation and even before the beginning of a trial. Although the decision-making process can occur well in advance to prevent premature or delayed strategic responses, it is still prudent for motor preparation to reflect the specific timing of the task. Nevertheless it would be illogical for motor preparation to precede the decision process. Therefore we conclude that the decision process also occurs in advance of the trigger stimuli during mixed-strategy tasks, either before, or simultaneous with, motor preparatory processes.
|
|
GRANTS |
|---|
|
|
|
ACKNOWLEDGMENTS |
|---|
|
|
|
FOOTNOTES |
|---|
Address for reprint requests and other correspondence: M. C. Dorris, Dept. of Physiology, Queen's University, Botterell Hall, Rm. 440, Kingston K7L 3N6, ON, Canada (E-mail: dorrism{at}biomed.queensu.ca)
|
|
REFERENCES |
|---|
|
Barraclough DJ, Conroy ML, Lee D. Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7: 404–410, 2004.[CrossRef][Web of Science][Medline]
Camerer CF, Ho TH, Chong JK. Sophisticated experience-weighted attraction learning and strategic teaching in repeated games. J Econ Theory 104: 137–188, 2002.[CrossRef][Web of Science]
Carpenter RH, Williams ML. Neural computation of log likelihood in control of saccadic eye movements. Nature 377: 59–62, 1995.[CrossRef][Web of Science][Medline]
Cisek P, Kalaska JF. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45: 801–814, 2005.[CrossRef][Web of Science][Medline]
Coe B, Tomihara K, Matsuzawa M, Hikosaka O. Visual and anticipatory bias in three cortical eye fields of the monkey during an adaptive decision-making task. J Neurosci 22: 5081–5090, 2002.
Cohen MX, Ranganath C. Reinforcement learning signals predict future decisions. J Neurosci 27: 371–378, 2007.
Corrado GS, Sugrue LP, Seung HS, Newsome WT. Linear-Nonlinear-Poisson models of primate choice dynamics. J Exp Anal Behav 84: 581–617, 2005.[CrossRef][Web of Science][Medline]
Ding L, Hikosaka O. Temporal development of asymmetric reward-induced bias in macaques. J Neurophysiol 97: 57–61, 2007.
Dorris MC, Glimcher PW. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 44: 365–378, 2004.[CrossRef][Web of Science][Medline]
Dorris MC, Munoz DP. Saccadic probability influences motor preparation signals and time to saccadic initiation. J Neurosci 18: 7015–7026, 1998.
Dorris MC, Olivier E, Munoz DP. Competetive integration of visual and preparatory signals in the superior colliculus during saccadic programming. J Neurosci 27: 5053–5062, 2007.
Dorris MC, Pare M, Munoz DP. Neuronal activity in monkey superior colliculus related to the initiation of saccadic eye movements. J Neurosci 17: 8566–8579, 1997.
Dorris MC, Pare M, Munoz DP. Immediate neural plasticity shapes motor performance. J Neurosci 20: RC52, 2000.
Driver PM, Humphries DA. Protean Behaviour: The Biology of Unpredictability. Oxford, UK: Oxford Univ. Press, 1988.
Erev I, Roth AE. Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am Econ Rev 88: 848–881, 1998.[Web of Science]
Fundenberg D, Tirole J. Game Theory. Cambridge, MA: MIT Press, 1991.
Glimcher PW. The neurobiology of visual-saccadic decision making. Annu Rev Neurosci 26: 133–179, 2003.[CrossRef][Web of Science][Medline]
Godijn R, Theeuwes J. Programming of endogenous and exogenous saccades: evidence for a competitive integration model. J Exp Psychol Hum Percept Perform 28: 1039–1054, 2002.[CrossRef][Web of Science][Medline]
Gold JI, Shadlen MN. Representation of a perceptual decision in developing oculomotor commands. Nature 404: 390–394, 2000.[CrossRef][Web of Science][Medline]
Gold JI, Shadlen MN. The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands. J Neurosci 23: 632–651, 2003.
Hanes DP, Schall JD. Neural control of voluntary movement initiation. Science 274: 427–430, 1996.
Harsanyi JC. Equilibrium-point interpretation of stable sets and a proposed alternative definition. Manage Sci Ser A Theory 20: 1472–1495, 1974.
Ikeda T, Hikosaka O. Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39: 693–700, 2003.[CrossRef][Web of Science][Medline]
Janssen P, Shadlen MN. A representation of the hazard rate of elapsed time in macaque area LIP. Nat Neurosci 8: 234–241, 2005.[CrossRef][Web of Science][Medline]
Juan CH, Shorter-Jacobi SM, Schall JD. Dissociation of spatial attention and saccade preparation. Proc Natl Acad Sci USA 101: 15541–15544, 2004.
Lau B, Glimcher PW. Dynamic response-by-response models of matching behavior in rhesus monkeys. J Exp Anal Behav 84: 555–579, 2005.[CrossRef][Web of Science][Medline]
Lee D, Conroy ML, McGreevy BP, Barraclough DJ. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res Cogn Brain Res 22: 45–58, 2004.[CrossRef][Medline]
Maimon G, Assad JA. A cognitive signal for the proactive timing of action in macaque LIP. Nat Neurosci 9: 948–955, 2006.[CrossRef][Web of Science][Medline]
Maynard Smith J. Evolution and the Theory of Games. Cambridge, UK: Cambridge Univ. Press, 1982.
Miller GF. Protean primates: the evolution of adaptive unpredictability in competition and courtship. In: Machiavellian Intelligence II: Extensions and Evaluations, edited by Whiten ABRW. Cambridge, UK: Cambridge Univ. Press. 1997, p. 312–340.
Milstein DM, Dorris MC. The influence of expected value on saccadic preparation. J Neurosci 27: 4810–4818, 2007.
Munoz DP, Dorris MC, Pare M, Everling S. On your mark, get set: brain stem circuitry underlying saccadic initiation. Can J Physiol Pharmacol 78: 934–944, 2000.[CrossRef][Web of Science][Medline]
Nash JF. Equilibrium points in N-person games. Proc Natl Acad Sci USA 36: 48–49, 1950.
Palacios-Heurta I. Professionals play minimax. Rev Econ Stud 70: 395–415, 2003.[CrossRef][Web of Science]
Platt ML, Glimcher PW. Neural correlates of decision variables in parietal cortex. Nature 400: 233–238, 1999.[CrossRef][Web of Science][Medline]
Rubinstein A. Comments on the interpretation of game theory. Econometrica 59: 909–924, 1991.[CrossRef][Web of Science]
Schall JD. Neural basis of deciding, choosing and acting. Nat Rev Neurosci 2: 33–42, 2001.[Web of Science][Medline]
Schultz W. Behavioral theories and the neurophysiology of reward. Annu Rev Psychol 57: 87–115, 2006.[CrossRef][Web of Science][Medline]
Seo H, Lee D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J Neurosci 27: 8366–8377, 2007.
Shadlen MN, Newsome WT. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86: 1916–1936, 2001.
Shinar J, Forte I, Kantor B. Mixed strategy guidance: a new high-performance missle guidance law. J Guidance, Control Dynam 17: 129–135, 1994.[CrossRef][Web of Science]
Smith JM. Evolution and the Theory of Games. Cambridge, UK: Cambridge Univ. Press, 1982.
Sugrue LP, Corrado GS, Newsome WT. Matching behavior and the representation of value in the parietal cortex. Science 304: 1782–1787, 2004.
Theeuwes J, Kramer AF, Hahn S, Irwin DE. Our eyes do not always go where we want them to go: capture of the eyes by new objects. Psychol Sci 9: 379–385, 1998.[CrossRef][Web of Science]
Theeuwes J, Kramer AF, Hahn S, Irwin DE, Zelinsky GJ. Influence of attentional capture on oculomotor control. J Exp Psychol Hum Percept Perform 25: 1595–1608, 1999.[CrossRef][Web of Science][Medline]
Thevarajah D, Dorris MC. Influence of previous choices and rewards on superior colliculus activity is modulated by behavioral context. Soc Neurosci Abstr 719.9, 2007.
Trappenberg TP, Dorris MC, Munoz DP, Klein RM. A model of saccade initiation based on the competitive integration of exogenous and endogenous signals in the superior colliculus. J Cogn Neurosci 13: 256–271, 2001.[CrossRef][Web of Science][Medline]
Usher M, McClelland JL. The time course of perceptual choice: the leaky, competing accumulator model. Psychol Rev 108: 550–592, 2001.[CrossRef][Web of Science][Medline]
Utts JM, Heckard RF. Mind on Statistics. Pacific Grove, CA: Thompson, 2007.
Walker M, Wooders J. Minmax play at Wimbledon. Am Econ Rev 91: 1521–1538, 2001.[Web of Science]
Wilimzig C, Schneider S, Schoner G. The time course of saccadic decision making: dynamic field theory. Neural Netw 19: 1059–1074, 2006.[CrossRef][Web of Science][Medline]
Wong K, Wang X. A recurrent network mechanism of time integration in perceptual decisions. J Neurosci 26: 1314–1328, 2006.
This article has been cited by other articles:
|
|
 |
|
  D. Thevarajah, A. Mikulic, and M. C. Dorris Role of the Superior Colliculus in Choosing Mixed-Strategy Saccades J. Neurosci., February 18, 2009; 29(7): 1998 - 2008. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
TOP
ABSTRACT
INTRODUCTION
