| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
York Centre for Vision Research, Canadian Institute of Health Research Group for Action and Perception, Departments of Psychology, Biology, and Kinesiology and Health Sciences, York University, Toronto, Ontario, Canada
Submitted 25 March 2004; accepted in final form 1 November 2004
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXGRANTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXGRANTSREFERENCES |
|---|
; Robinson 1968; Sparks 2002; Sparks and Mays 1990). For one-dimensional (1-D) saccades the geometry of this process is trivial (e.g., Jürgens et al. 1981; Scudder 1988), because for 1-D rotations retinal error and motor error are equivalent. However, for real 3-D eye rotations, a visuomotor reference frame transformation is required to map RE onto the ME command (Crawford and Guittion 1997; Hepp et al. 1993).
The fundamental reason for this is that visual signals originate in the retina and are therefore defined in an eye-fixed frame, whereas Listing's law constrains the eye to rotate around certain axes defined in a head-fixed coordinate system (Crawford and Guitton 1997). As a result, simulations of a fixed mapping between the 2-D retinal error and the 3-D motor error command produce movements that diverge erroneously, as a nonlinear function of initial eye orientation. Moreover, because this arises from the geometry of projecting an eye-fixed spatial vector onto effectors that do not rotate about eye-fixed axes, essentially the same problem arises for visually guided gaze shifts involving both the eyes and head (Klier et al. 2001) and visually guided arm movements (Crawford et al. 2000), only with potentially much larger errors.
Behavioral studies have shown that the human saccade generator does not make such eye errors (Klier and Crawford 1998), even for express or memory-guided saccades (Henriques and Crawford 2001), but rather maps RE onto different saccade vectors as a function of initial eye orientation. In particular, it has been shown that one visual RE (measured in eye-fixed coordinates) must be mapped onto saccade trajectories with orthogonal components that diverge by as much as ±15° or more in the oculomotor range, depending on the size of the saccade and initial eye position (Crawford and Guittion 1997; Klier and Crawford 1998). Thus there must be some physiological mechanism that accounts for eye orientation in the visuomotor transformation.
In theory, this problem could be solved by fixing the eye muscle pulling directions with respect to the eyes, but this would violate Listing's law, producing large pseudo-random torsional positions at the end of each saccade that do not occur in real life (Crawford and Vilis 1991; Tweed and Vilis 1990a). (Arranging neck and shoulder muscles in eye coordinates to solve the problem is even less plausible.) Instead, eye muscle pulling directions appear to be optimized for Listing's law, which requires saccade velocity axes to tilt halfway, not all the way, with eye position (Demer et al. 1995; Quaia and Optican 1998). This leaves the question, how does the brain implement the reference frame transformation required to map visual signals onto the kind of motor commands appropriate to provide accurate eye movements that satisfy the constraints of Listing's law?
One clue is that we may be able to sketch out the region within the brain where this transformation takes place. The superior colliculus (SC) appears to encode visual and motor signals much like the retina, in a 2-D topographic map (Hepp et al. 1993; Van Opstal et al. 1991). Moreover, stimulation of the SC in the head-unrestrained animal evokes gaze shifts with fixed directions in eye coordinates and, conversely, vary as a function of initial gaze position when plotted in space coordinates (Klier et al. 2001). This suggests that the SC encodes a gaze command in retinal coordinates, and the position-dependent reference frame transformation is implemented downstream. Further, this transformation is probably complete in the oculomotor system at the level of the premotor short-lead burst neurons because these appear to encode displacements in eye orientation within a 3-D head-fixed coordinate system aligned with Listing's plane (Crawford 1994; Crawford and Vilis 1992; Henn et al. 1989).
Thus the reference frame transformation for saccades appears to occur between the SC and the brain stem burst neurons (perhaps with help from side-loop structures like the cerebellum) (Lefevre et al. 1998; Quaia et al. 1999), although we have little clue as to how this might occur. In models of the saccade generator that touch on this problem, the reference frame transformation involves a stage in which the retinal goal of the saccade is transformed into a head-fixed desired gaze direction, from which the system could then compute the correct head-fixed kinematics to drive a 3-D saccade (Crawford and Guitton 1997; Glasauer et al. 2001; Tweed and Vilis 1990; Zee et al. 1976). This includes network models that used eye position "gain fields" in a distributed neural net to transform RE into a head-centric representation of target direction (Zipser and Andersen 1988). Such head-centric representations have never been observed physiologically in the cortical saccade control centers, the SC, or brain stem (Batista et al. 1999; Colby and Goldberg 1999; Colby et al. 1995; Russo and Bruce 1996; Sabes et al. 2002; Sparks 1988, 1989).
This leaves open the question of how the required reference frame transformations take place. Moreover, any inquiry into this question also has to consider that the same transformations must also deal with other computational problems, such as the degrees of freedom problem. Because the gaze command signal from the SC is 2-D (Hepp et al. 1993; Klier et al. 2003; Van Opstal et al. 1991), whereas the brain stem burst generator possesses a 3-D coordinate system that can generate eye rotations about any axis (Crawford and Vilis 1992; Henn et al. 1989), something between must select the right axis for Listing's law (which only allows saccades to reach eye orientations that are rotated from primary position about an axis in Listing's plane). As discussed above, the implementation of Listing's law places constraints on the visuomotor reference frame transformation. Most models have included a Listing's law box that calculates a desired eye orientation in Listing's plane (Crawford and Guitton 1997; Glasauer et al. 2001; Tweed and Vilis 1990a).
Finally, the circuits that derive saccadic motor commands must also deal with the spatiotemporal transformation. The spatiotemporal transformation arises in the oculomotor system because the SC encodes saccade metrics with the use of a place code on a topographic map, whereas the reticular burst neurons must supply a rate code in the appropriate coordinate system to the extraocular muscles (Becker and Klein 1973; Robinson 1974; Westheimer and Blair 1973). A number of theoretical and physiological investigations have considered this problem (e.g., Hepp and Henn 1983; Optican and Quaia 2002; Tweed and Vilis 1990a), including the question of how this fits within dynamic feedback loops for saccade control that are beyond the scope of the present investigation. However, the physiological transformation from a topographic code to a coordinate system vector representation is probably inseparable from the oculomotor reference frame transformation.
The problem facing neurophysiologists trying to solve these questions experimentally, as we see it, is that we really do not know what type of signals to look for on a neuron-by-neuron basis. In fact, it is quite possible and likely that the solution is right at the tip of our electrodes and we do not know it. Most current investigators would agree that signal processing in the brain—even for the feedforward transformations in the brain stem—is a distributed process (Sparks 2002). Therefore one potentially fruitful approach is to train simple neural nets to perform these transformations, and then analyze the workings of such networks to gain insights that we can apply to the real system (Krommenhoek and Wiegerinck 1998; Zipser and Andersen 1988). If we can understand how a simple neural net works, where we have complete knowledge of the inputs, outputs, and architecture, as well as the ability to manipulate these values at will, then we might hope to understand the limited amount of data that one can collect from the real brain.
The current investigation applies this approach to understand how visual information encoded at the level of the SC (or any similar topographically organized, eye-centered structure) might be transformed into a geometrically correct signal to drive brain stem burst neuron populations. Note that the focus of our study was to see how such a network would implement the reference frame transformation, but the other transformations (spatiotemporal and 2-D to 3-D) were implicit in our network. For example, our networks produced a motor displacement code (initial motor error for a saccade) that could easily be converted into a rate code for instantaneous motor error, by placing it within a local feedback loop (Crawford and Guitton 1997). The networks described below do not perform this complete transformation, but do perform the transformation from a topographic "place code" to a coordinate system that could encode saccade dynamics. Similarly, our networks were required to convert a 2-D visual input into a motor command that actively maintained ocular torsion at zero (to comply with Listing's law). Thus these networks implicitly learned to solve the 2-D to 3-D transformation and one aspect of the spatiotemporal transformation, while also learning to perform the eye-to-head reference frame transformation that was the focus of our investigation.
Unlike most previous neural net studies of reference frame transformations (e.g., Krommenhoek et al. 1993; Zipser and Andersen 1988), we used the noncommutative mathematics of 3-D rotations in our input–output relations (Crawford and Guitton 1997; Tweed and Vilis 1987); and, unlike our previous study of this topic (Smith and Crawford 2001a), these inputs/outputs were biologically inspired, based on the known topography of visual signals in the SC and coordinate systems observed in the brain stem saccade generator. The results suggest that an understanding of visuomotor reference frame transformations may be obtainable within the framework of currently known physiology.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXGRANTSREFERENCES |
|---|
Network architecture
Figure 1A shows a diagrammatic view of the network structure and the general nature of its input–output signals that were used in this study. Networks were composed of 3 layers: an input layer consisting of 513 units; a hidden layer with 4, 9, 16, 25, 36, or 49 hidden units; and an output layer consisting of 6 units. When referring to a specific size, or class, of network we use the number of hidden units as a nomenclature. Thus a network with 9 hidden units will be referred to as a 9-unit network. All networks had feedforward connections only (i.e., no connections existed between any units within a layer).
|
Input–output signals
Networks were provided with 2 input signals: 1) an eye position (EP) signal specifying initial eye position in orthogonal 3-D head-centric coordinates, where each EP was constrained to be within a 100° diameter ocular motor range (OMR); and 2) a retinal error (RE) signal, in eye coordinates, that was specified on a 2-dimensional (2-D) map and was constrained to be within 80° of map center. In addition, the target location specified by the RE was constrained to be within 50° of primary position so that the model would not attempt to saccade beyond the oculomotor range. The teaching signal specified the vectorial motor error (ME) required to drive the eye from its current position to the one specified by the visual target location, such that the target was acquired by gaze and final EP landed in the normal "Listing's plane" range of 3-D eye positions observed during head-fixed saccades (Crawford and Guitton 1997). That is, we chose a teaching signal such that the head-fixed torsional component of EP was set to zero. However, this does not invalidate the 3-D aspect of the model because this 3rd degree of freedom must be as strictly controlled by the networks as the other two.
Visual inputs
The most important input to the saccade generator for visually guided saccades is vision itself. For physiological inspiration, we based the properties of the visual inputs to our network on neurophysiological recordings from the superficial layers of the SC. We chose this structure because it probably encodes visual information to be used by the saccadic system in a form that is relatively unprocessed.
Figure 1B shows a magnified representation of the visual inputs to the network as illustrated in Fig. 1A. The "floor" of the figure represents the full extent of the virtual visual map in the vertical and horizontal dimensions (–90° to 90°). As shown, the input units to the network were topographically arranged where the density of input units decreased with distance from the center of the map. That is, the input units monitored a central region covering 2° of extent with a spacing of 1°, whereas adjacent annular areas covering 4°, 10°, and 
80° of extent encoded resolutions of 2°, 4°, and 10°, respectively (solid dots). Note that because of the viewpoint in this figure, the units covering the central 1° and 2° areas coalesce into a homogeneous representation. Each of the input units had an eye-fixed visual receptive field determined by a Gaussian function with a maximum activity level of 1 at its center.
Also shown is a vertically displaced pictorial representation of 2 Gaussian receptive fields at 2 different distances from the center of the map, one at horizontal and vertical coordinates of (–20°, 20°) and the other at (40°, –40°). Projected onto the map is the corresponding contour outline of each of the 2 Gaussian curves' receptive fields. As illustrated, the width of each input unit's receptive field depended on its (virtual) topographical location. So, for example, the unit located at (20°, 20°) has a receptive field width (sigma) of 7.8°, whereas the unit located at (40°, –40°) has a receptive field width of 14.8°. In general, the sigma value for the Gaussian receptive field of the units ranged from 1° to 20° with increasing eccentricity from the center of the map. This approach was intended to mimic the larger receptive field sizes of eccentric locations found in sensory structures such as the SC and the retina. The receptive field widths used here were estimated from data presented in Cynader and Berman (1972).
Visual target inputs to the networks were computed from 2-D vectors that specified the horizontal and vertical integer components of peripheral targets on the virtual 2-D map. The vector components of the visual targets were specified to the nearest degree and covered an area within an 80° visual range (large circle). Each of these "stimuli" then activated a number of the visual input units, depending on the location of the stimulus on the virtual topographic map and the width of the Gaussian receptive field of the units in that part of the map.
Eye position inputs
Based on our previous studies (Crawford and Guitton 1997; Klier and Crawford 1998; Smith and Crawford 2001a) we knew that our network would require a 3-D EP input to solve the reference frame transformation for saccades. This could come from proprioceptive sensors in the eye muscles, but most oculomotor models assume that it arises as an efference copy from the oculomotor "neural integrator" that provides the motor signal to hold eye position between saccades (Crawford 1994; Quaia et al. 1999; Robinson 1975; Tweed 1997). However, no previous model of the saccadic visuomotor transformation that we know of has used a realistic representation of these 3-D neural integrator signals.
Most recent studies of 3-D eye movements have described eye orientations using vectors that are parallel to the axis of rotation that brings the eye to that position from some central reference position. Listing's law states that during saccades with the head fixed, these eye orientation vectors all align with a head-fixed plane (Tweed and Vilis 1990). The gaze direction at one particular reference position, the primary position, is orthogonal to the special plane of vectors, called Listing's plane, and parallel to the head-fixed torsional axis (Westheimer 1957).
The oculomotor neural integrator is a distributed structure with a critical region for horizontal integration occurring in the nucleus prepositus hypoglossus (Cannon and Robinson 1987; Cheron and Godaux 1987) and a critical region for vertical/torsion integration in the midbrain interstitial nucleus of Cajal (Crawford et al. 1991; Fukushima et al. 1990). Unlike the sensory codes found in visual structures, the brain stem neural integrator does not use a topographic map but rather represents similar directions in clustered populations, each controlling individual coordinate axes. Experimental studies suggest that the 3-D oculomotor neural integrator is organized in a head-fixed coordinate system similar to that of the eye muscles, but aligned with Listing's plane. That is, with the vertical axis (for the horizontal component of eye orientation) aligned within Listing's plane, and the torsional–vertical coordinate axes aligned in the horizontal plane orthogonal to Listing's plane (Crawford 1994; Crawford et al. 1991). Thus to encode a purely horizontal position, only the horizontal integrator needs to be activated, but the torsional–vertical coordinates are arranged in such a way that, e.g., an upward EP would require the coactivation of an up-counterclockwise population of neurons on the left side of the midbrain and an up-clockwise population on the right side (Crawford et al. 1991).
To mimic the geometric output of such a structure, we started with a 3-D coordinate system for angular orientation of the eye (torsional, vertical, and horizontal) in orthogonal, right-handed head-centric coordinates, and using the right-hand rule to define the direction of rotation (Tweed and Vilis 1990b). These coordinates were then rotated 45° about the vertical axis to form a new coordinate system (Fig. 1A: "Eye Position Signal") symmetric about Listing's plane as described above (Crawford 1994; Crawford et al. 1991).
A second physiological constraint is that real neurons do not have negative firing rates. To represent "negative directions" the vestibuloocular system, including the neural integrator, has developed a push–pull mechanism where positive and negative values correspond to modulations up or down about a background firing rate (Fukushima et al. 1990, 1992; King et al. 1981). To mimic this, we transformed the 3-D EP vector into a 6-D vector that represented the angular rotation around each of the 3 axes by a yoked pair of components. Component pairs 1 and 2, 3 and 4, and 5 and 6 encoded the torsional, vertical, and horizontal components of rotation, respectively (in right-hand coordinates). Each component ranged between 0 and 1, where a balance of activity, indicated by the values (0.5, 0.5), for a single pair, represents 0° rotation around a particular axis.
In calculating the values for each of the 6-D components we used the following formulation
 |
Motor outputs
We wanted the motor output signal of the network to represent the 3-D motor error command that drives all reticular formation short-lead burst neurons (one synapse up from both motoneurons and the neural integrator) at the beginning of saccades. Little is known about the complex synaptic physiology of the inputs to short-lead burst neurons. These may include direct input from the SC, from "long-lead" burst neurons in the reticular formation, burst-driver neurons, and vestibular-related inputs (e.g., Hepp and Henn 1983; Kaneko and Fukushima 1998; Moschovakis and Highstein 1994; Schnyder et al. 1985; Sparks 1988, 1989; Stanton et al. 1988). Therefore it is easier to define the physiological correlate of this command according to its target (i.e., as the command coded by the total ensemble of synaptic input to the short-lead burst neurons), which are much better understood. Equivalently, one can view this command as representing the initial motor error encoded by the total ensemble of short-lead burst neurons at the commencement of a saccade.
We encoded motor error as the vectorial change in eye orientation (
E) required to move the eye from its initial 3-D orientation (Ei) to the final desired orientation (Ed) and takes the form E = Ed – Ei. This command has been shown (Crawford and Guitton 1997) to be the appropriate code to drive the oculomotor short lead burst neurons if the latter code is "rate of change in 3-D eye orientation" & Edot; (Hepp et al. 1994; Quaia and Optican 1998; Suzuki et al. 1995). This burst signal has been shown in simulations (Crawford and Guitton 1997) to work equally well at driving a 3-D plant model with head-fixed muscle pulling directions (Tweed and Vilis 1987) or in a "pulley" plant model where the eye muscle pulling directions tilt according to the half-angle rule for Listing's law (Demer et al. 1995; Quaia and Optican 1998). For the head-fixed plant model, the burst signal has to be modified downstream by position signals (Crawford 1994). That is, if the burst neurons encode angular velocity in head coordinates (explicitly coding the axis tilts for Listing's law) then the reference frame transformation problem is magnified, producing twice the deviation between the retinal code and the motor code as a function of eye position.
We then needed to choose a coordinate system to encode this command. Again, because little is known about the way that short-lead burst neuron inputs code the spatial aspects of motor error, we took our cue from the short-lead burst neurons themselves. These neurons are arranged into populations within anatomic nuclei that appear to use a spatial coding scheme similar to that of the neural integrator; that is, they control saccade components in a similar way to the semicircular canals and eye muscles, forming a set of coordinate axes that are aligned with and symmetric to Listing's plane (Crawford and Vilis 1992; Suzuki et al. 1999).
Like the neural integrator, saccade burst neurons are arranged into pairings that control opposite saccade directions. There are also indications that these neurons have a "background firing rate," in the sense that their firing rates are well above zero for saccades in the direction orthogonal to their preferred direction (Cullen and Guitton 1997; Van Gisbergen et al. 1981). In this sense, burst neurons also appear to use a push–pull arrangement to linearize their output within a certain range. Therefore taking these as clues as to what might be the appropriate signal to drive such neurons, we coded motor output using the same 6-dimensional coordinate system that was used for the neural integrator.
For example, to encode a 15° purely upward eye movement, which would be represented by the 3-D rotation vector (0, –15, 0) in standard right-hand coordinates, in our 6-D vector components 1 and 2 evaluate to (0.5, 0.5), whereas components 3 and 4 code (0.425, 0.575) and components 5 and 6 encode (0.5, 0.5). Similarly, an 80° rightward movement from an initial eye position of, say, 50° to the left of center, would be encoded by the activations (0.5, 0.5), (0.5, 0.5), and (0.1, 0.9). Note that saccades of this amplitude were the maximum allowable because supplied RE never exceeded 80° and initial EP was always within 50° of center.
Network training
Learning in the networks was accomplished by using the standard back-propagation algorithm (Rumelhart et al. 1986) with the addition of a momentum term (10%). Weights were updated incrementally, that is, weights were updated after the presentation of each exemplar (for a more complete description of this learning algorithm, see Smith and Crawford 2001a).
Briefly, learning in the network occurs when each output unit computes its error term (E) from the teaching signal (the mean squared error between network output and the desired output), computes its weight correction term Wij (by multiplying the input to the unit [f(i)], the error computed above), which is used to update the weight matrices, and then multiplies the weight correction term by a learning constant (
) that, in these networks, was set to 0.5. That is
 |
W. Of course, the error is passed through the same weights that generated the current output pattern (before the update of the weights). Thus units that contributed more to the error in the pattern will be changed more by the weight update procedure, whereas those units contributing little to the current pattern error will be changed little.
In our networks, the output signals of all units were positive real values ranging from 0 to 1. This output constraint was accomplished by using a standard sigmoid transfer function, specified by
 |
is the range of the sigmoid (0 to 1); to achieve this range, = 2 (because the maximum 
is later subtracted). 
is the slope of the linear portion of the sigmoid ( = 1); is the maximum of the sigmoid ( = 1). (For a complete mathematical description of back-propagation see the APPENDIX of Smith and Crawford 2001.)
To train the networks we used a series of combinations of initial eye positions and retinal errors selected randomly from the following training set: Initial eye positions were laid out in a horizontal–vertical grid with 20° spacing, centered on (0, 0) and limited to fall within the round 50° OMR. Initial eye positions always had zero torsion in Listing's coordinates (i.e., they were in Listing's plane). Each initial eye position had associated with it 56 REs where such eye position and retinal error pairings would not exceed the 50° OMR. These REs were chosen randomly from an array of directions resembling a "star pattern" (imagine a cross superimposed on an x), such that all of the cardinal and oblique directions were represented in the training set. For amplitude, each direction had 7 REs located at 2°, 5°, and then in 10° increments from 10° through 50°. (In preliminary trials we found that training on the small retinal errors <10° was necessary to avoid bizarre behavior in very small saccades.)
To compute the ideal motor error (ME) for these visual and eye position combinations we followed the algorithm outlined in Crawford and Guitton (1997), also used in our previous study (Smith and Crawford 2001). This algorithm provides the motor error (E) required to take the eye from its current 3-D orientation to the orientation in Listing's plane that satisfies the desired 2-D gaze direction. In brief, using quaternion representations, we first computed the desired gaze relative to the eye by converting the 2-D visual signal into a desired gaze signal in eye coordinates (DGeye). Using this convention, the first and second components of 3-D gaze were the vertical and horizontal measurements of RE and the third component was a forward-pointing unit vector. We then rotated DGeye by initial eye position (Ei), which results in the desired 2-D gaze relative to the head (DGhead). Next DGhead was put through a Listing's law operator (Tweed and Vilis 1990), resulting in a desired 3-D eye position command (Ed). Finally, the ME was then computed by subtracting Ei from Ed by first converting the quaternion representation to vectors while maintaining the same coordinate system. These vectors were then scaled as a function of the angle of rotation and converted into the 6-D format required by the neural networks. Thus the error signal for the back-propagation algorithm was the difference between these computed ideal values and the actual values output by the network.
We initially trained 12 networks with the following increments of hidden unit number: 4, 9, 16, 25, 36, and 49, training 2 of each size to determine the minimum network size required to learn the task. We defined this as the minimum number of units required to successfully reduce the sum of squared error of the network output to <0.01. This training goal was chosen because it resulted in a network output within about 1° of ideal performance. Figure 2 shows the error curves generated by a representative network of each size as indicated by the associated number. The y-axis represents the ongoing error (in degrees) during training, whereas the dashed line indicates the training goal. The x-axis indicates the number of epochs (where an epoch indicates one pass through the entire training set). Note that, although networks typically started with errors of >0.25 (about 20°), values above this are cut off the scale of the figure. All networks quickly reduced their training error to below 0.25 in an initial near-vertical drop. After our initial observations, we tried smaller increments of network size to establish the exact threshold.
|
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXGRANTSREFERENCES |
|---|
Because all of the networks that we accepted for analysis generated saccades accurate to within 1° of ideal performance on average, clearly they were able to generate essentially accurate saccades. Here we checked certain crucial aspects of their performance before considering their "neural coding mechanisms." In particular, we carefully checked the position-dependent aspects of their performance related to the reference frame transformation.
The saccade reference frame transformation requires that the network map any one RE onto different MEs as a function of EP (Crawford and Guitton 1997). To determine how well the networks performed compared with ideal behavior, we examined the networks with a test set consisting of initial eye positions at 5° intervals from 40° left to 40° right with 0° centered at the straight-ahead eye position. Each position was tested with a 30° upward and a 30° downward RE (Fig. 3A). Another similar test set was also used except that initial eye positions ranged from 40° down to 40° up where each position was tested with a 30° leftward and a 30° rightward RE (Fig. 3B). Thus there were 16 representative eye positions and 32 retinal errors associated with each of these tests. These combinations of eye positions and retinal errors were chosen so that the requisite movements would remain inside the 50° oculomotor range within which the networks were trained.
|
). For illustration purposes we show only half of the tested eye positions in increments of 10° rather than 5°. In Fig. 3A, we show the vertical saccade task (horizontal eye positions paired with vertical REs), whereas in Fig. 3B we show the horizontal saccade task (vertical eye positions paired with horizontal REs). In both plots, the origin represents the primary position as defined by Listing's law, whereas the large circle represents the 50° OMR within which the networks were trained. The traces within each graph depict the RE of the supplied eccentric target (dashed trace with star), the ideal ME performance based on the correct geometry (—
—), and the actual ME output of the trained network (—
—). The RE is plotted in retinal coordinates, whereas EP and ME are plotted in head-centric coordinates.
Note that RE (dashed trace with star) and ideal ME (——) diverged from one another in a position-dependent fashion, as demonstrated previously both theoretically and experimentally (Crawford and Guitton 1997; Klier et al. 1998; Smith and Crawford 2001b). This is the position-dependent reference frame transformation that the network had to learn: if it did so, actual performance (——) would follow the ideal ME vectors (——), whereas the absence of any position-dependent transformation would cause the actual network output to follow the RE vectors (dashed lines ending in a star). The actual output of the network closely followed that of ideal performance, and in these graphs often occlude it, with a mean error for this network in this task of 0.55° (SD: 0.43°). For the horizontal saccade task (Fig. 3B) the mean network error was 0.67° (SD: 0.37°). Across both of these performance tests the network's mean error was 0.62° (SD: 0.40°). Thus this network learned the position-dependent visuomotor transformation within the 1° of error required by our training goal, similar to the performance observed in human saccades (Klier et al. 1998).
To quantify performance across networks, we compared network performance with that of a model that does not take eye position into account, but that simply maps RE onto ME without accounting for the different reference frames (Crawford and Guitton 1997). Figure 3 C–F, shows the results of this comparison for all 9-unit networks. The open circles represent the response of the previously tested 9-unit network (A/B), whereas the thin 2nd-order fits illustrate the responses of the other 9-unit networks. Plots C and E show the vertical saccade task, whereas D and F show the horizontal saccade task, both corresponding to the graphical depictions in the left column. In each panel, the values along the x-axis correspond to these initial eye positions, whereas values along the y-axis represent the angular difference (in degrees) between the directions of ideal motor performance and either actual network performance or the values predicted for the no-position-compensation model. Perfect position compensation would result in all performance values aligning with the abscissa. The thick dashed line represents errors in the predicted motor performance of the no-position-compensation model.
Residual position-dependent directional errors in the actual performances were nonsystematic and small in all of the networks. A performance estimate of the residual errors across these networks revealed that network outputs on average were 16% of those predicted by no eye position compensation in the vertical saccade task (Fig. 3 C and E,), whereas network outputs in the horizontal saccade (Fig. 3 D and F,) task were 17% of the "no position compensation" prediction. Similar patterns of behavior for 16-, 25-, 36-, and 49-unit networks were observed (not shown), but with even lower residual errors. Moreover, errors in torsional eye position from Listing's plane (not shown) were so low as to be uninteresting: they were essentially zero for networks with <36 units and <0.03° in the larger networks.
Visual receptive fields of hidden units
Having established that our biologically inspired networks had learned the correct transformations at the behavioral level, we set out to determine the "neural mechanisms" for these transformations. To characterize the visual receptive field of a saccade-related neuron, investigators systematically move pinpoints of light to cover the visual space and record when the target neuron is active (Hamed et al. 2001; Hubel and Wiesel 1959; Russo and Bruce 2000). In this way, a visual response profile of the target cell can be constructed. We followed a similar procedure to characterize the visual receptive field of each hidden unit; that is, we sequentially "stimulated" every possible target location on the visual input map with a unit stimulus and recorded the output of each hidden unit (while eye position was fixed at center).
The resultant visual response profiles for a typical 9-unit network are shown in Fig. 4A, showing all 9 hidden units (1–9). (The same network is used in subsequent figures to facilitate comparisons between figures.) For completeness, we show the entire visual map that ranges from –90 ° to 90° in both the horizontal and vertical dimensions, although the actual visual range of a unit was restricted to ±80° (white circle overlaid on the receptive field of unit 1). Clearly, the visual receptive fields of the hidden units are complex with regions of highest sensitivity (red) corresponding to the unit's preferred direction, regions of lowest sensitivity (black), and graded transitional regions between these. The complexity of these visual receptive fields is reminiscent of those found in the lateral intraparietal area (e.g., Hamed et al. 2001).
|
We performed such a principal-components analysis with the eye looking at the straight-ahead primary reference position. The original receptive fields were then remapped into the new data space formed by the principal components. Figure 4 B–I, shows the results of this analysis for exemplary 9-unit (left column, B–E) and 36-unit (right column, F–I) networks.
In both of these networks (and across all 9 of the networks tested: 5 of the 9-unit class and 2 each of each of the 16-, 25-, 36-, and 49-unit classes) 4 principal components accounted for about 95% of the variance in the data (80% by the first 2 and 15% by the remaining 2). The dominant, first 2 components (Fig. 4 B, C and F, G,) always showed a fairly simple organization. First, each component had an antipodal structure with oppositely tuned maximal and minimal response zones. Second, the orientations of the grading for the first (B and F) and second (C and G) components were orthogonal. This appears to provide the basis coordinates for specifying target direction in 2-dimensional space. The other 2 remapped receptive fields (D and E; H and I) showed a more complex structure capturing some of the complexity in the original receptive fields. Again, these 4 components captured 95% of the visual target data required to produce accurate saccades, but this is not to imply that only a few units in the trained networks do the majority of the work with the remaining units contributing little to the solution. In the complete networks this information is distributed across all of the units and is not present in this orthogonalized and optimized form. Also, as we shall see, it is the interaction between different units that is critical for the overall performance of the model.
Reference frames for the sensory signal
The previous section describes the visual receptive fields of hidden units at one, fixed central eye position. To determine the reference frame for these visual receptive fields (i.e., are they fixed relative to the eye, relative to the head, or some other alternative) we had to retest these visual receptive fields at different eye positions. Figure 5 shows the visual receptive field surface of hidden unit 6 from the same 9-unit network illustrated in Fig. 4; however, now the network was tested with eye position fixated at 4 different eye positions: 30° up, down, left, and right. It is important to note that this receptive field map is plotted in retinal coordinates; that is, responses are mapped according to the locations of the stimuli relative to the fovea, in eye-fixed coordinates. If the visual receptive fields were fixed in head (or space) coordinates, they would shift in these plots in the direction and amount opposite to the eye position shift. However, no such shift was observed. If one observes the key topological features in the receptive field, such as the location of maximal and minimal response areas, one can see that the hidden unit visual receptive field appears to stay absolutely fixed relative to the simulated retina.
|
Note first, that these maximum points were almost always found at the edge of the visual range, resembling the "open-ended" response fields observed in some SC units (Freedman and Sparks 1997). More important, we have coded the maximal response points for each of the 4 eye positions as a different sized circle. Note that each set of circles forms a perfectly overlapping set of concentric rings; in other words these points overlap perfectly in eye coordinates with absolutely no change in direction or distance relative to the central fovea. A similar result for a 16-unit network is shown in Fig. 5F, and indeed we found the same result held for all networks. Thus the topological organization of the visual receptive fields in our hidden units was always absolutely eye-fixed. The central remaining question here—as in real neurophysiology—is: how do these eye-fixed visual receptive fields get mapped onto the correct saccade vector in motor coordinates?
Relationships between visual, eye position, and motor sensitivity vectors
In our previous network (Smith and Crawford 2001a), which solved the identical geometric problems using a simpler vector-only input–output organization, the hidden units formed certain invariant functional classes that subdivided the transformation into specific parallel-task modules. For example, a dominant class of units, called the vector propagation class, were organized into an orthogonal coordinate system where the retinal error tuning vector and the motor error tuning vector were more or less aligned, which was shown to provide the main drive that moved saccades in the correct direction. We also showed that the eye-position–dependent modification of motor error was supplied by a smaller module of units, called position-opposite, because eye position and motor tuning were more or less opposite in direction. This modularity was revealed when we computed a series of "sensitivity vectors."
Because this was a previously fruitful approach we began with the same type of analysis here. That is, we looked for a similar organization within and between the visual, eye position, and motor error output of the hidden units by constructing a similar series of sensitivity vectors. That is, we constructed 1) a sensitivity vector for maximal and minimal visual activity, 2) a sensitivity vector for eye position tuning, and 3) a sensitivity vector for motor tuning. The visual sensitivity vector was simply the vector from the origin to the location of the maximum and minimum values in the visual receptive field (see Fig. 4A for an example of the visual receptive fields of a 9-unit network). Other methods, like the calculation of the "center of mass" of these visual activation fields, did not appear to provide better information.
The eye position sensitivity vector was the computed 3-D vector coded by the 6 activation weights of the appropriate inputs to the hidden units (the last 6 weight values of the input to hidden unit weight matrix). To accomplish this, we first performed the inverse of the transformation that we originally used to convert the 3-D eye position vector into a 6-D representation (see above). Operating on the resulting 3-D vector, we then counterrotated it to restore it into standard Cartesian coordinates.
For the motor sensitivity vector we used the weights between the hidden and the output layers. That is, we computed the direction and magnitude of the motor sensitivity vector for each hidden unit based on its projection weights to the output layer using the same inverse procedure as described above for the eye position activations (also, see METHODS and Smith and Crawford 2001a). The horizontal and vertical components of these sensitivity vectors are plotted in Fig. 6 for each of the 9 hidden units in the network.
|
10°). For example, an inspection of units 2, 3, 4, 6, 8, and 9 shows little alignment between visual and motor tuning [mean of 93° (SD: 62°)], whereas the remaining units (1, 5, and 7) show a somewhat better alignment with a mean angular difference of 47° (SD: 26°).
In addition, Smith and Crawford (2001a) found a clear subclass of units with opposite motor and eye position tuning (i.e., with the difference between motor and eye position tuning 
150°). However, between motor and eye position tuning vectors in the illustrated network we found a mean relative difference of 76° (SD: 59°). As well peak visual tuning and eye position tuning vectors showed a mean relative rotation of 82° (SD: 69°), unlike Smith and Crawford (2001). Despite intensive scrutiny, we were unable to discern any clear-cut functional relationships or groupings in the angular relationships between visual, motor, and eye position tuning vectors of this or any other network. Rather, we found that units had widely distributed visual, motor, and position tuning, with no clear functional relations between them. Thus although the receptive fields are oriented across units, they do not align to form a "coordinate system" (such as ordinary or rotated Cartesian coordinates ("+" or "x" in form) as was found in Smith and Crawford (2001).
The same conclusions remained when we approached this question more formally. A visual and quantitative cluster analysis (using standard Matlab routines) did not reveal the grouping characteristics seen in Smith and Crawford (2001). The relationships between the sensitivity vectors found here were rather loose and broadly tuned in this and in all 9-unit networks. It would appear that these networks, in simulating the distributed nature of the input and output signals, performed the visuomotor transformation in a more distributed manner than that evidenced previously in Smith and Crawford (2001). (Also, see on-line supplementary Fig. 1.) In light of this, our subsequent analysis focused on properties of the network that might give rise to this more distributed solution.
Motor coding in the hidden unit layer
As stated above, our hidden units showed widely dispersed motor tuning. Our next step was to characterize the position dependency of this motor tuning. In particular, we determined the reference frame and coding of the hidden unit motor output: the contribution that activation of each unit makes to the behavior of the network. This is determined by the final connection weights of the hidden unit to the output layer (at least within the linear working range of the output layer, corresponding to saccade components of ±60°). In fact, because we know that the output layer codes a fixed-vector 3-D eye displacement vector in head coordinates, and we know that the connections from each hidden unit to the output layer are fixed at the end of training, the motor output coding of hidden units was predetermined: the hidden units code fixed-vector eye orientation displacements in head coordinates, just like the output layer.1
|
40° down in B). The open circles represent the baseline control condition—how well the network fixated these positions with a zero RE input; these "fixations" stayed within a mean window of 3°. This shows the background noise for our "stimulations." From each of these eye positions we then simulated stimulation by setting the output of the hidden units to 0.5 (chosen for illustration purposes). These "stimulations" drove the eye to a new location, as indicated by the filled circles. Note that the invoked saccades of the network are primarily vertical when unit 6 is stimulated, and primarily horizontal when unit 4 is stimulated, which agrees with their preferred motor tuning directions (see Fig. 6). More important, these saccade vectors were fixed (when plotted in head coordinates), independent of initial eye position. This resulted in a series of perfectly parallel vectors. Repeating this test with the largest stimulation value (1.0) produced "saccades" well outside the OMR (saccades of about 100°) but in exactly the same directions for each of the units. Similar results were obtained with different networks and units (not shown).
Thus each of the hidden units in our networks simultaneously existed in 2 reference frames: an eye-centered reference frame in their response to visual inputs and a head-centered reference frame in their fixed-vector output command. From this we can conclude two things: First, that the reference frame transformation was somehow occurring within the hidden unit layer. Second, that it was not occurring at the level of individual units because each such unit provided a fixed directional mapping of vision to movement independent of eye position.
Interactions between visual and eye position inputs
Somehow eye position signals were modulating the visual responses in our hidden unit layer to provide the position-dependent transformation illustrated in Fig. 3, but without shifting the topology of their eye-centered receptive fields. Could the mechanism take the form of a "firing rate" modulation like the classical gain fields of Zipser and Andersen (1988)? To determine this, we examined the overall profile of the visual receptive field as a function of eye position (Salinas and Abbott 2001). We constructed cross sections or "slices" of the visual receptive fields using the eye position sensitivity vector (Fig. 6) as a guide.
Figure 8 A and B,, shows the results of this investigation for unit number 2 of the same 9-unit network used in previous illustrations. Figure 8A shows the resultant cross sections with eye positions arrayed along their preferred axis at 20° spacing centered on the origin (see inset), whereas Fig. 8B shows the slices taken at eye positions lying along the axis orthogonal to the preferred one. The x-axis in both plots shows the full range of the visual map (±90°), although only the area between the dashed lines represents visual input to the network during training. Examination of Fig. 8B shows that in the direction orthogonal to the preferred axis, eye position does not modify the visual response because all 5 curves superimpose onto a single trace.
|
; Zipser and Andersen 1988). That is, we can compare trace 3—the visual response when the eye is positioned at the straight-ahead primary reference position—with the traces when the eye is deviated from straight ahead (traces 1, 2, 4, and 5). It is apparent that visual sensitivity is increased as the eye moves in the preferred direction, whereas visual sensitivity is decreased as the eye moves away from the preferred direction, without shifting the peak laterally. Figure 8C shows the full set of preferred position axis slices for all 9 units of this network. The central trace in each plot is the visual response of the unit with the eye at reference position. Note that the range of visual response for units is different but that all units show their response is modified by eye position with increasing sensitivity in the preferred direction and decreasing sensitivity in the opposite direction. This type of behavior was verified across all successfully trained networks by randomly choosing 2 units from each network.
At first glance the visual response of these units to changes in eye position along the preferred position axis appeared to be multiplicative. For example, traces 1, 2, 4, and 5 in Fig. 8A can be well approximated by multiplying trace 3 by some factor, and the same held for several other units in Fig. 8C. However, for other units (e.g., 7 and 8), addition/subtraction of a bias component appeared to be more important than the multiplicative component of the gain field. A more quantitative analysis showed that the actual relationship was pseudo-linear, being approximated (to 98% of the data) in each unit by a fit like y = mx + b, where y is the activation of the unit for a particular visual stimulus location and eye orientation, x is the activation at primary eye position, and m and b are constant multiples of eye position along the preferred axis. In some units (such as unit 2 in Fig. 8C), the m-term dominated, whereas in other units (such as unit 8) the b-term dominated. Examples of this analysis are provided in the APPENDIX. Because these responses were similar to, but somewhat more complex than, linear gain fields, we will henceforth refer to them by the generic term "eye position fields."
"Lesioning" the eye position fields
Presumably, the eye position fields were implicated in the reference frame transformation in our model. To confirm this, we characterized the performance of the network without these position modulations. In effect, we lesioned the position inputs. We conducted this lesion study using the same task as illustrated in Fig. 3 and in this figure conventions are the same. First we eliminated any eye position information altogether during the test by setting the output of the input units that provide eye position information to 0.0 (Fig. 9 A and C,, left column). Note that both the directions and amplitudes of the motor responses of the network, which should be essentially 30° vertical in panel A and 30° horizontal in panel C (as in Fig. 3), were wildly off, containing large horizontal and vertical biases. Thus the complete removal of the eye position signal apparently removed certain expected offsets that were required for its basic function. The more subtle but more relevant effect, though, is that all position dependencies are lost. Instead of converging slightly as required for the reference frame transformation (see Fig. 3 A and B,) the network now produced parallel, fixed-vector motor outputs independent of initial position. This suggests, in addition to offset problems, that the reference frame transformation had been lost.
|
The reference frame transformation: rotating the population vector
The final stage of this investigation was to understand how our network was effecting the position-dependent reference frame transformation illustrated in Fig. 3. To understand this, one needs to synthesize the data from the preceding sections. First, we have found that each hidden unit possesses an eye-centered visual receptive field, but a fixed-vector output that is independent of eye position in head coordinates. This means that the reference frame transformation is occurring within the hidden unit layer, but it also means that individual hidden units cannot do it alone: each has a fixed visuomotor mapping independent of eye position.
Second, because this fixed-vector motor coding in the hidden layer is widely dispersed across different units (Fig. 6), it must be the overall population vector that determines the motor output. Such mechanisms have been found in areas such as motor cortex (Georgopoulos et al. 1986) and the superior colliculus (Lee et al. 1988; Sparks et al. 1976). In other words, it is the overall pattern of output vectors from the hidden layer, summed and transformed by the sigmoidal functions of each unit at the output layer, that determines ME. Therefore there must be some mechanism that rotates the hidden unit population vector (Georgopoulos et al. 1989) into the correct correspondence with the population visual vector as a function of eye position.
We have already seen (Fig. 9B) that without eye position fields, the network reverts to a fixed visual-motor mapping. Conversely, the eye position fields have the capacity to adjust the individual contributions of the hidden unit motor vectors, as a function of eye position, to the overall population vector for the saccade. We hypothesized that their function was to adjust these individual contributions so that the overall population vector (in head coordinates) is rotated in an eye position–dependent manner into correspondence with the visual input vector (in eye coordinates). Indeed, when one synthesizes the results summarized above, this appears to be a mathematical certainty. Figure 10 confirms that this was the mechanism for the reference frame transformation in our model.
|
). The simulations show the population motor response of the network to an upward retinal error of 30° under 3 different horizontal eye positions: 30° left (top row), primary position (middle row), and 30° right (bottom row). For illustration purposes, we show results from a 49-unit network (right column), a 25-unit network (middle column), and a 9-unit network (left column). Focusing first on the 49-unit network, we see that the "properly filtered" population vector gives the correct response for the 3 different eye positions—the mild-position–dependent convergent pattern documented in Fig. 3. In each case, this corresponds to a complex pattern of activation in the individual units: these are activated for a wide array of directions such that only a slight predominance in one particular direction gives the necessary population vector. Close examination of the individual unit vector contributions shows that it is not their directions (these are fixed tuning vectors), but rather their relative contributions (length x activation) that vary with eye position, mediated by the eye position fields described above. Again, this is what causes the population vector to tilt as a function of eye position.
In the case of the 49-unit network, the motor population vector is approximated fairly well in the vector average of hidden unit contributions (dashed line). The results from the smaller networks are similar, except that in this case the vector average provides a much poorer approximation of the behavior—largely because it does not account for the nonlinear saturation effect at the output stage. Thus a rotation of the population vector appears to be the "neural mechanism" for the reference frame transformation in our network.
It should be noted that, although the overall vector output is within the linear range of our sigmoid transfer function (up to about 60° saccades in our networks), the responses of some the individual hidden units are far out in the nonlinear range. This is why computing a linear vector average from these units results in a poor estimate of the actual motor output vector. This is most nearly true for the 9-unit network. However, in the 49-unit network, individual hidden units have smaller contributions to the overall output of the hidden layer (note that the scale of the axes is different in Fig. 10). Because these smaller responses are within the linear range, they sum quite well to produce a population vector close to the actual output.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXGRANTSREFERENCES |
|---|
; Xing and Andersen 2000a, b; Zipser and Andersen 1988). Numerous saccade models inspired by the work of D. A. Robinson include a calculation of either desired eye position or desired target direction in head coordinates (e.g., Grossberg and Kuperstein 1986; Zee et al. 1976). Part of the original appeal of these models was that they seemed to solve the problem of space constancy during saccades (Hallet and Lightstone 1976; Mays and Sparks 1980) by storing saccade targets into a frame that would be independent of subsequent eye movements. However, it has since been shown that this problem is probably solved by remapping saccade goals within an eye-centered frame during eye movements (Bruce and Goldberg 1985; Duhamel et al. 1992; Moschovakis 1996).
Instead, the current study deals with the geometric problem of transforming such eye-centered visual codes into the appropriate 3-D motor code to drive the eye muscles, a topic that has received much less attention (Crawford and Guitton 1997; Hepp et al. 1993; Klier and Crawford 1998; Klier et al. 2001; Smith and Crawford 2001a). This problem is best illustrated here in Fig. 3, which shows how a given retinal input corresponds to different saccade vectors as a function of initial eye orientation. Being dependent on both initial orientation and saccade length in a nonlinear fashion, these tilts are generally modest within the oculomotor range, but in the head-free range of gaze control they can be as great as 90° (Klier et al. 2001).
It been shown that the human saccade generator solves this problem, generating accurate saccades essentially independently of initial eye orientation (Henriques and Crawford 2001; Klier and Crawford 1998). However, eye muscles cannot solve this problem independently without violating other constraints (Crawford and Guitton 1997); that is, here we assumed they are rigged to optimize Listing's law according to the recent work of Demer et al. (2000). Moreover, other visuomotor systems with very different anatomy, such as the eye-head gaze control system (Klier et al. 2001) and the arm control system (Crawford et al. 2000), encounter and solve essentially the same geometric problem. Thus this is a central problem in visuomotor control, and yet we know next to nothing about its neural solution.
To obtain physiological realism in our model, we also had to address two other computational problems solved by the saccade generator: first, the static aspect of the spatiotemporal transformation, inasmuch as our model had to learn how to transform a topographic visual code into a coordinate system for a movement command; second, the degrees of freedom problem, in that it had to transform a 2-D visual input into a 3-D command for saccades in Listing's plane. Insofar as our model learned to generate accurate saccades from all initial eye positions with negligible torsion (compared with ±1–4° torsion in monkeys and humans, depending on the task; Ferman et al. 1987; Klier and Crawford 1998; Straumann et al. 1991; Tweed and Vilis 1990a; Van Opstal et al. 1996), clearly it solved all of these problems. The question is, how and what does this say about real physiology?
Correspondence to physiology
In a previous theoretical study, we trained a neural network to solve the same geometric problem posed here and were able to identify a precise mechanism: the network divided the task into parallel modules, including one class of units that performed the main visuomotor mapping and other classes that performed the position-dependent aspects of the task (Smith and Crawford 2001a). However, in that network we killed physiological realism in both the input and output layers for the sake of simplicity and clarity. Based on our current, very different results, this clearly influenced the solution derived at the level of the intermediate (hidden) layer.
In the current study we attempted to make the inputs and outputs of the network physiologically realistic (Fig. 1), in terms of the geometric aspects most closely related to the problems we have studied. In particular, we have used an input layer that topographically represented space as a series of eye-centered visual receptive fields, and eye position inputs and motor outputs based on what we know about the 3-D coordinate systems of the brain stem saccade generator, complete with push–pull organization. The learning rule used (back-propagation) is no more physiologically realistic than any other known rule because no one knows which rules the brain actually uses. However, it likely captures the essence of supervised learning in the saccade generator, which probably uses visual feedback about errors to drive learning (Frens and Van Opstal 1994; McLaughlin 1967; Miller et al. 1981; Vilis et al. 1985). Our general assumption was that our hidden layer was modeling processes that would likely be downstream from the superior colliculus (Klier et al. 2001), but one can only say for certain that we were modeling transformations similar to those that occur somewhere between a very early visual stage in the saccade generator (like the superficial layers of the SC) and the 3-D brain stem burst generator, leaving much of the known system in between.
In light of this, essentially every property that we have observed in our trained hidden unit layer (where tested experimentally) corresponds to the known physiology of this system, in some cases much better than previous models. For example, the complexity of the visual receptive fields observed in our hidden units (Fig. 4) is reminiscent of those found in the lateral intraparietal area (LIP; e.g., Hamed et al. 2001). Because these complex properties persist in the principal components, showing similar features across different networks (Fig. 4 B–I,), they appear to be essential for the functioning of our networks. It has been noted that 2-D neural net models of a transformation from an eye-centered to a head-centered map of visual space do not produce this complexity (Zipser and Andersen 1988) unless additional features are added to the learning rule (Brozovic et al. 2003). The current study demonstrates that these receptive field properties can emerge naturally when a more realistic 3-D input output geometry is simulated.
Another physiologically realistic property observed in our network is in the gain-field–like modification of visual responses to eye orientation (Figs. 8 and 11). Such eye-position fields have been observed experimentally at almost every stage of the saccade generator, from occipital cortex (Nakamura and Colby 2002), to LIP (Andersen et al. 1990), to the SC (Van Opstal et al. 1995), and even elements of the saccade generator located in the brain stem reticular formation (Hepp and Henn 1983). Originally thought to provide the basis for transforming an eye-centered visual map into a head-centered map (Zipser and Andersen 1988), this idea has lost strength with the accumulated evidence that the higher levels of saccade generation works largely through a sequence of eye-centered vectorial codes (Batista et al. 1999; Colby et al. 1995; Georgopolus 1982; Russo and Bruce 1996; Sabes et al. 2002; Sparks 1988, 1989; Van Gisbergen et al. 1981), which can be updated (remapped) during eye movements (Duhamel et al. 1992; Nakamura et al. 2002). One proposition is that gain fields are involved in this remapping process (Xing and Andersen 2000a). However, the role of eye position gain fields in remapping has been questioned (White III and Snyder 2004). Moreover, the current investigation shows that similar responses emerge automatically in the transformation of eye-centered visual signals into the vectorial command appropriate to generate saccades.
|
) and indeed in most cortical areas involved in visuomotor transformations (Andersen 1995; Bisley and Goldberg 2003; Georgopoulos 1986; Maunsell et al. 1983). It has recently been pointed out that a simple vector average of individual neuron motor tuning vectors, scaled by their activity, probably does not adequately represent the direction of motion controlled by this population (Scott and Gribble 2001). This property can also be seen in our network (Fig. 10), where a vector average only approximates the actual population vector in some cases. This is because in our network, as in the real brain, the kinematics encoded by a population of neurons depends on the transformations performed on this information in subsequent steps, which tend to be nonlinear.
Finally, it is striking that our model achieved the visuomotor reference frame transformation for saccades without forming any head-centered visual receptive fields. Again, an exhaustive series of single-unit recording studies over 3 decades has failed to reveal any such receptive fields in the physiological saccade generator (Colby and Goldberg 1999; Gottlieb et al. 1998; Klier et al. 2001; Moschovakis and Highstein 1994; Nichols and Sparks 1995; Scudder 1998; Umeno et al. 1997; Walker et al. 1995). A possible exception is the ventral intraparietal area (Duhamel et al. 1997), but there is no evidence that this area contributes directly to saccades (Colby and Goldberg 1999). As in the real brain, the visual receptive fields in the hidden units of our model were entirely eye-centered, whereas the output layer encoded a saccade vector in brain stem motor coordinates. In this respect, our model is more realistic than any previous models of the saccade generator that have ignored the 3-D reference frame transformation and/or used explicit head-centered representations.
The 3-D saccadic reference frame transformation
As noted above, it is striking that neither this network nor our previous network (Smith and Crawford 2001a) ever developed head-centered representations of visual space. Clearly such representations are not necessary to implement a reference frame transformation in the saccade generator, and at this point in time it seems clear that future experimental studies should no longer be guided by this assumption. Here we consider how the network did perform its reference frame transformation.
Again, this transformation amounts to rotating the visual vector, coded originally in ocular coordinates, into alignment with the correct motor vector in head coordinates as a function of eye orientation and saccade length (Crawford and Guitton 1997; Klier and Crawford 1998; Klier et al. 2001). To understand the basics of how our network did this, it is necessary first to note that neurons—artificial or real—may exist simultaneously within different coordinate frames (Mitchel and Zipser 2001; Pouget et al. 2003). In this case, our individual hidden units coded visual space in the 2-D eye-centered code provided by the visual input layer, and their motor output in the 3-D head-fixed vector code of the output layer to which they project. The relationship between these two is fixed, so that no individual unit could produce a reference frame transformation. This is where the eye position fields come in: by adjusting the contribution of each unit to the overall population vector encoded at the output layer, the latter was rotated into its correct alignment. This population mechanism clearly provided the basis for the reference frame transformation in our model, and considering the level of physiological realism in the spatial aspects of this model (see preceding section), appears to provide a compelling explanation for the same process in the brain.
One potential problem with using a rate code to control movement parameters is that it might be hard to disambiguate from general changes in rate related to other sources, such as attentional effects (Goldberg et al. 2002). However, this is not a problem for the current mechanism because the rotation of the population vector is not dependent on overall changes in activation or even a target-related modulation, but rather a very specific position-dependent balance between units coding different directions of saccade motor error. For example, a resulting code that provides a greater contribution of upward-oriented units as opposed to downward-oriented units to make a leftward population vector tilt up. Moreover, the use of a push–pull organization on the output end of our model, as in the real oculomotor system, tends to cancel equal increases or decreases in the activity of inputs coding opposite directions.
The more subtle, nonlinear aspects of the transformation in our network were presumably embedded in the specific organization of the eye position field effects in our hidden units (see APPENDIX) and their geometric relationships of their preferred axes to the visual and motor tuning vectors across the population. In our previous simpler, much less realistic model, we were able to identify specific relationships between these vectors, classify them into parallel modules, and equate these modules to specific functions (Smith and Crawford 2001a). However, application of the same methods to the current network failed to reveal any underlying modular order in the organization. This may be because, as suggested by Robinson (1992), a certain degree of modularity in the input and output layers is required for modularity to develop in the hidden unit layers. However, it will probably require a more comprehensive study, across different types of neural nets, to understand the precise factors that determine such modularity.
Listing's law and the spatiotemporal transformation
Although the purpose of our study was to study the visuomotor reference frame transformation for saccades, this could not be done realistically without including a 2-D to 3-D transformation and spatiotemporal transformation (in the static sense of mapping a topographic code onto a coordinate system code). Clearly the network learned these, in the sense that it learned to generate the motor errors for accurate saccades that obeyed Listing's law using the machinery that we provided. We did not design our tests to explicitly test the mechanisms for these transformations—our tests focused on the reference frame question—but some general conclusions can be drawn. One important one is that one cannot reject the existence of a feedforward transformation for Listing's law on the basis that no one has found a nucleus or set of nuclei corresponding to a "Listing's law box." In our model Listing's law had to be obtained through an overall balance of all the hidden layer weights to the units of the output layer that encode vertical–torsional saccade components in a push–pull musclelike coordinate system aligned with Listing's plane, as observed in burst neurons (Crawford and Vilis 1992), the neural integrator (Crawford 1994), and motoneurons themselves (Suzuki et al. 1995, 1999). There was no "Listing's law" unit. It is probably a waste of time to look for a "Listing's law" center or unit because in such networks the solution must be distributed through all of the projections to the 3-D saccade generator to work. More specific predictions are provided in the following sections.
Testable predictions of the model
Although several aspects of our model, described above, mirror the known physiology, there are other elements of our theoretical findings that have not yet been tested in real physiology. Before discussing this, it must be noted that some aspects of our neural net analysis are far from trivial in the real brain. In particular, in our model it was an easy matter to determine the precise motor output code of the hidden units by reconstructing their synaptic connections or by "stimulating" individual units. This is not technically possible in the real brain, and this measure of the causal contribution of a unit to behavior is not identical to correlating unit activity to a saccade. The closest one might come is to reconstruct the contribution of the unit through spike-triggered averaging with muscular activity—again, a technically enormous problem.
Given these caveats, there are still several feasible experiments that could be done to test our model. First, our model predicts that eye-centered visual receptive fields will persist up until the level of the reference frame transformation. This is not so trivial to demonstrate as it might sound. Surprisingly, no one has ever tested whether any neurons in the visual system or oculomotor system have truly eye-centered receptive fields (Crawford and Guitton 1997). To do so would require that receptive fields be tested at different eye positions and mathematically rotated (as a function of 3-D orientation) into the correct frame. To date, such a 3-dimensional reference frame analysis has been done only for stimulation data, which suggested that the superior colliculus was organized in retinal coordinates (Klier et al. 2001). This needs to be confirmed with unit recording data. If not confirmed, this would not negate the findings of the current study (we know that the saccade system starts in visual coordinates and ends in motor coordinates), but it would require a reinterpretation of what our hidden units represent.
However, recall that our visual input layer was derived from superficial superior colliculus visual properties and that the output commands were derived from the burst neuron coordinates. This leaves out most of the higher-level saccade system in between these two (including both the intermediate/deep motor layers of the superior colliculus and cortical saccades structures that form the targets and inputs, respectively, of the sensory and motor layers (Wurtz and Albano 1980). In addition, we have assumed that the output of the superior colliculus uses a similar eye-centered code (Klier et al. 2001). Whether this is the case, our hidden units could be interpreted against any of these structures. For example, our hidden units could be interpreted as explaining the complex visual receptive fields/gain fields observed in the LIP area (Andersen et al. 1990; Snyder et al. 1998).
Second, our model predicts that at any level where a visuomotor reference frame transformation occurs, there should be an eye position–dependent dissociation between the visual and motor tuning of the neuron. (For example, our hidden units coded visual responses as fixed vectors in eye coordinates, but motor responses as fixed vectors in head coordinates.) In neurons with separate visual and motor responses, this might be tested with the kind of analysis suggested in the previous paragraph, but performed separately for the 2 burst components. Unfortunately though, neural activity in single units does not necessarily correlate with the motor influence of the unit on behavior (Georgopoulos et al. 1986; Scott et al. 2001; Sparks and Gandhi 2003), so these effects might show up only at the level of a population response, or in comparing the position dependencies of visual receptive fields compared with stimulus-evoked movements. Moreover, if Klier et al. (2001) are correct, this stage might only happen downstream from the superior colliculus for gaze shifts, perhaps in "long-lead" burst neurons, but the analogous stage might be identifiable at the cortical level for arm movements.
Third, our model predicts that, tested completely, visuomotor neurons involved in reference frame transformations should show both gain and bias responses to eye position (Fig. 8 and APPENDIX). Such responses would be difficult to distinguish if one were not actively looking for them because, indeed, most of our hidden units showed both types of modulation. Moreover, if one were able to identify the source of this eye position information, i.e., in a thalamic or brainstem pathway from either the eye muscles themselves or the brainstem neural integrator (Steinbach 1987), then one should be able to obliterate those visual eye position fields by lesioning this pathway. Perhaps more important, our model predicts that this would devastate the ability of the system to correctly map vision onto motor commands as a function of eye orientation—in other words, producing the kind of errors shown in Fig. 9.
Finally, our study relied on the assumption that saccade motor error is most likely to be encoded as a vector displacement code, best suited to drive saccadic burst neurons that encode the first time derivative of 3-D eye orientation, in line with most recent theoretical and experimental studies (Crawford and Guitton 1997; Quaia and Optican 1998; Quaia et al. 1999). However, hard proof of this is still lacking. Indeed, it has recently been shown that some "velocity" neurons in the smooth pursuit system may encode angular velocity (i.e., the explicit axis of eye rotation in head coordinates, rather than eye orientation derivatives) (Angelaki and Dickman 2003). It is important for the oculomotor field to pursue more such studies, requiring 3-D eye recordings in conjunction with brain stem neurophysiology (Crawford et al. 2004). However, as described elsewhere (Crawford and Guitton 1997; Tweed and Vilis 1990) if burst neurons encode angular velocity rather than derivatives, this places all the burden of the eye position–dependent reference frame transformation and Listing's law on the visuomotor transformation. Thus we predict that if we trained our network on such motor signals it would show similar, but even stronger position-dependent effects in the hidden units.
General applications
Another area where the transformations observed in our net might apply to real physiology is in the neural transformations for visually guided arm movement. A very similar geometric problem occurs in transforming vision into an arm movement command (Crawford et al. 2000, 2004), particularly if the initial command is for arm displacement in visual coordinates (Buneo et al. 2002). In other words, this displacement signal would have to be rotated into shoulder coordinates as a function of eye orientation (Crawford et al. 2000). Also, the sequence of signals in the cortical neurophysiology for arm movements appears to also have all the necessary ingredients for the population reference frame transformation observed here: including eye-centered reach commands in the "parietal reach region" (PRR; Batista et al. 1999), population-based motor vector commands in area 5 (Crammond and Kalaska 1989; Kalaska et al. 1983) and motor cortex (Georgopoulos et al. 1982), and eye position gain fields located through PRR and premotor cortex (Batista et al. 1999). Thus in principle, the same basic mechanism that we have described here could explain the visuomotor reference frame transformation for both eye and arm movements.
In conclusion as described above, many of the results reported here have already been observed in the real oculomotor system, but in piecemeal fashion, scattered across different areas of the brain. Perhaps the most useful general contribution of the current study might be to deflate some of the mystery of what these signals might be doing. If the brain is indeed using the same mechanisms that we observed in our net, then all of the information we need to solve this question for the real brain is already at hand; it just needs to be synthesized. The current network thus provides a working hypothesis for how the brain effects visuomotor reference frame transformations.
|
APPENDIX |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXGRANTSREFERENCES |
|---|
If the modification of visual response by eye position was a purely multiplicative gain field then one would expect that all of the traces would intercept the y-axis at the same location, whereas only the slope of the line (the gain) varied for different eye positions. Visual inspection of Fig. 11A shows that this is nearly true for the responses of unit 2. The y-intercept remained close to zero (–0.01, –0.01, 0.02, 0.05) from the bottom trace to the top trace, respectively, whereas the slope component varied (0.57, 0.77, 1.24, 1.44). On the other hand, for unit 8 (Fig. 11B) the bias component varied widely (–0.14, –0.09, 0.13, 0.28), whereas the slope remained more nearly constant (0.90, 0.98, 0.94, 0.82), showing that the linear approximation to these responses is more heavily dominated by a bias term than a gain-field mechanism. Thus our model predicts that visual–eye position interactions in the saccade generator will show both gain and bias terms.
Again, it is unlikely that neurophysiological recordings would allow one to discriminate linear gains and biases from segments of more complex nonlinear functions, like those shown in Fig. 11. However, it may be possible to discriminate the theoretical roles of these different terms to the overall network behavior by 1) deliberately replacing the learned nonlinear unit responses with linear functions, and then 2) separately manipulating the gains and biases of the responses. We have not presently undertaken such an analysis.
|
GRANTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXGRANTSREFERENCES |
|---|
Present address for M. A. Smith: Department of Neuroscience, University of Minnesota, 6–145 Jackson Hall, 321 Church Street SE, Minneapolis, MN 55455 (E-mail: smith788{at}umn.edu).
|
FOOTNOTES |
|---|
1 This simple mapping between hidden unit activity and its influence on behavior is an approximation that holds for saccade components up to 60 degrees. For hidden layer population activations that specify larger saccade components, these will be shrunk by the sigmoidal squashing functions of the motor output layer units. This will tend to shrink the amplitude of ME outputs for larger retinal errors, and cause complex position-dependent effects (for example, the "off-axis" ME components required for large horizontal or vertical REs would shrink less than the main component, whereas the opposite would be true for large oblique REs). This would be accompanied by distortions of the contribution of individual hidden units to the final motor output. This is theoretically interesting because it is not a property of individual hidden units (see Fig. 7), but an emergent property of the population, where hidden units indirectly affect the motor contribution of other hidden units by altering the responsiveness of the next layer. Similar factors are relevant for neurons "firing" near the limits of their range in the real brain: this may be why the larger components of saccades are slowed to match the speed of the smaller components—to keep the "burst neurons" for both components in the same part of their nonlinear range. However, these factors are less relevant for coding ME vectors working within their quasi-linear range, that is, for the simulations in the current manuscript. 
Address for reprint requests and other correspondence: J. D. Crawford, Centre for Vision Research, Computer Science Building, York University, 4700 Keele Street, Toronto, Ontario M3J 1P3, Canada (E-mail: jdc{at}yorku.ca)
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXGRANTSREFERENCES |
|---|
Andersen RA, Bracewell RM, Barash S, Gnadt JW, and Fogassi L. Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of macaque. J Neurosci 10: 1176–1196, 1990.[Abstract]
Angelaki DE and Dickman JD. Premotor neurons encode torsional eye velocity during smooth-pursuit eye movements. J Neurosci 7: 2971–2979, 2003.
Batista AP, Buneo CA, Snyder LH, and Andersen RA. Reach plans in eye-centered coordinates. Science 285: 257–260, 1999.
Becker W and Klein HM. Accuracy of saccadic eye movements and maintenance of eccentric eye positions in the dark. Vision Res 13: 1021–1034, 1973.[CrossRef][Web of Science][Medline]
Bisley JW and Goldberg ME. The role of the parietal cortex in the neural processing of saccadic eye movements. Adv Neurol 93: 141–157, 2003.[Medline]
Brozovic M, Cohen YE, and Andersen RA. Models of coordinate transformations for reaches toward an auditory target. Program No. 658.1. 2003 Abstract Viewer/Itinerary Planner. Washington, DC: Society for Neuroscience 2003. Online.
Bruce CJ, Goldberg ME, Bushnell MC, and Stanton GB. Primate frontal eye fields. II. Physiological and anatomical correlates of electrically evoked eye movements. J Neurophysiol 54: 714–734, 1985.
Buneo CA, Jarvis MR, Batista AP, and Andersen RA. Direct visuomotor transformations for reaching. Nature 416: 632–636, 2002.[CrossRef][Medline]
Cannon SC and Robinson DA. Loss of the neural integrator of the oculomotor system from brainstem lesions in the monkey. J Neurophysiol 57: 1383–1409, 1987.
Cheron G and Godaux E. Disabling of the oculomotor neural integrator by kainic acid injections in the prepositus-vestibular complex of the cat. J Physiol 394: 267–290, 1987.
Colby CL, Duhamel JR, and Goldberg ME. Oculocentric spatial representation in parietal cortex. Cereb Cortex 5: 470–481, 1995.
Colby CL and Goldberg ME. Space and attention in parietal cortex. Ann Rev Neurosci 22: 319–349, 1999.[CrossRef][Web of Science][Medline]
Crammond DJ and Kalaska JF. Neuronal activity in primate parietal cortex area 5 varies with intended movement direction during an instructed-delay period. Exp Brain Res 76: 458–462, 1989.[Web of Science][Medline]
Crawford JD. The oculomotor neural integrator uses a behavior-related coordinate system. J Neurosci 14: 6911–6923, 1994.[Abstract]
Crawford JD, Cadera W, and Vilis T. Generation of torsional and vertical eye position signals by the interstitial nucleus of Cajal. Science 252: 1551–1553, 1991.
Crawford JD and Guitton D. Visual-motor transformations required for accurate and kinematically correct saccades. J Neurophysiol 78: 1447–1467, 1997.
Crawford JD, Henriques DYP, and Vilis T. Curvature of visual space under vertical eye rotation: implications for spatial vision and visuomotor control. J Neurosci 20: 2360–2368, 2000.
Crawford JD, Medendorp WP, and Marotta JJ. Spatial transformations for eye-hand coordination. J Neurophysiol 92: 10–19, 2004.
Crawford JD and Vilis T. Axes of eye rotation and Listing's law during rotations of the head. J Neurophysiol 65: 407–423, 1991.
Crawford JD and Vilis T. Symmetry of oculomotor burst neuron coordinates about Listing's plane. J Neurophysiol 68: 432–448, 1992.
Cullen KE and Guitton D. Analysis of primate IBN spike trains using system identification techniques. I. Relationship to eye movement dynamics during head-fixed saccades. J Neurophysiol 78: 3259–3282, 1997.
Cynader M and Berman N. Receptive-field organization of monkey superior colliculus. J Neurophysiol 35: 187–201, 1972.
Demer JL, Miller JM, Poukens V, Vinters HV, and Glasgow BJ. Evidence for fibromuscular pulleys of the recti extraocular muscles. Invest Ophthalmol Vis Sci 36: 1125–1136, 1995.
Demer JL, Oh SY, and Poukens V. Evidence for active control of rectus extraocular muscle pulleys. Invest Ophthalmol Vis Sci 41: 1280–1290, 2000.
Duhamel JR, Bremmer F, BenHamed S, and Graf W. Spatial invariance of visual receptive fields in parietal cortex neurons. Nature 389: 845–848, 1997.[CrossRef][Medline]
Duhamel JR, Colby CL, and Goldberg ME. The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255: 90–92, 1992.
Ferman L, Collewijn H, and Van Den Berg AV. A direct test of Listing's law. I. Human ocular torsion measured in static tertiary positions. Vision Res 27: 929–938, 1987.[CrossRef][Web of Science][Medline]
Freedman EG and Sparks DL. Activity of cells in the deeper layers of the superior colliculus of Rhesus monkey: evidence for a gaze displacement command. J Neurophysiol 78: 1669–1690, 1997.
Frens MA and Van Opstal AJ. Transfer of short term adaptation in human saccadic eye movements. Exp Brain Res 100: 293–306, 1994.[Web of Science][Medline]
Fukushima K, Harada C, Fukushima J, and Suzuki Y. Spatial properties of vertical eye movement-related neurons in the region of the interstitial nucleus of Cajal. Exp Brain Res 79: 25–42, 1990.[Web of Science][Medline]
Fukushima K, Ohashi T, Fukushima J, and Kase M. Ocular torsion produced by unilateral chemical inactivation of the interstitial nucleus of Cajal in chronically labyrinthectomized cats. Neurosci Res 13: 301–305, 1992.[CrossRef][Web of Science][Medline]
Georgopoulos AP, Kalaska JF, Camaniti R, and Massey JT. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J Neurosci 2: 1527–1537, 1982.[Abstract]
Georgopoulos AP, Lurito JT, Petrides M, Schwartz AB, and Massey JT. Mental rotation of the neuronal population vector. Science 243: 234–236, 1989.
Georgopoulos AP, Schwartz AB, and Kettner RE. Neuronal population coding of movement direction. Science 233: 1416–1419, 1986.
Glasauer S, Dieterich M, and Brandt T. Central positional nystagmus simulated by a mathematical ocular motor model of otolith-dependent modification of Listing's plane. J Neurophysiol 86: 1546–1554, 2001.
Goldberg ME, Bisley J, Powell KD, Gottlieb J, and Kusunoki M. The role of the lateral intraparietal area of the monkey in the generation of saccades and visuospatial attention. Ann NY Acad Sci 956: 205–215, 2002.[CrossRef][Web of Science][Medline]
Goldberg ME and Bruce CJ. Primate frontal eye fields. III. Maintenance of a spatially accurate saccade signal. J Neurophysiol 64: 489–508, 1990.
Gottlieb J, Kusunoki M, and Goldberg ME. The representation of visual salience in monkey parietal cortex. Nature 391: 481–484, 1998.[CrossRef][Medline]
Grossberg S and Kuperstein M. Neural dynamics of adaptive sensory motor control, ballistic eye movements. North-Holland: Elsevier North-Holland, 1986.
Hallet PE and Lightstone AD. Saccadic eye movements towards stimuli triggered by prior saccades. Vision Res 16: 99–106, 1976.[CrossRef][Web of Science][Medline]
Hamed SB, Duhamel JR, Bremmer F, and Graf W. Representation of the visual field in the lateral intraparietal area of macaque monkeys: a quantitative receptive field analysis. Exp Brain Res 140: 127–144, 2001.[CrossRef][Web of Science][Medline]
Henn V, Hepp K, and Vilis T. Rapid eye movement generation in the primate: physiology, pathophysiology and clinical implication. Rev Neurol (Paris) 145: 540–545, 1989.[Web of Science][Medline]
Henriques DYP and Crawford JD. Testing the three-dimensional reference frame transformation for express and memory-delay saccades. Neurocomputing 38–40: 1267–1280, 2001.[CrossRef][Web of Science]
Hepp K and Henn V. Spatio-temporal recoding of rapid eye movement signals in the monkey paramedian pontine reticular formation (PPRF). Exp Brain Res 52: 105–120, 1983.[Web of Science][Medline]
Hepp K, Suzuki J, Straumann D, and Hess BJM. On the 3-dimensional rapid eye movement generator in the monkey. In: Information Processing Underlying Gaze Control, edited by Delgado-Garcia JM, Godeaux E, and Vidal PP. Oxford, UK: Pergamon, 1994, p. 65–74.
Hepp K, Suzuki Y, Straumann D, Hess BJ, and Henn V. Eye position dependence of innervation on-directions of motoneurons in the monkey. Ann NY Acad Sci 781: 629–632, 1996.[CrossRef][Web of Science][Medline]
Hepp K, Van Opstal AJ, Straumann D, Hess BJ, and Henn V. Monkey superior colliculus represents rapid eye movements in a two-dimensional motor map. J Neurophysiol 69: 965–979, 1993.
Hubel DH and Wiesel TN. Receptive fields of single neurones in the cat's striate cortex. J Neurophysiol 148: 574–591, 1959.
Jürgens R, Becker W, and Kornhuber H. Natural and drug induces variations of velocity and duration of human saccadic eye movements: evidence for a control of the neural pulse generator by local feedback. Biol Cybern 39: 87–96, 1981.[CrossRef][Web of Science][Medline]
Kalaska JF, Caminiti R, and Georgopoulos AP. Cortical mechanisms related to the direction of two-dimensional arm movements: relations in parietal area 5 and comparison with motor cortex. Exp Brain Res 51: 247–260, 1983.[Web of Science][Medline]
Kaneko CRS and Fukushima K. Discharge characteristics of vestibular saccade neurons in alert monkeys. J Neurophysiol 79: 835–847, 1998.
King WM, Fuchs AF, and Magnin M. Vertical eye movement-related responses of neurons in midbrain near intestinal nucleus of Cajal. J Neurophysiol 46: 549–562, 1981.
Klier EM and Crawford JD. Human oculomotor system accounts for 3-D eye orientation in the visual-motor transformation for saccades. J Neurophysiol 80: 2274–2294, 1998.
Klier EM, Wang H, and Crawford JD. The superior colliculus encodes gaze commands in retinal coordinates. Nat Neurosci 4: 627–632, 2001.[CrossRef][Web of Science][Medline]
Klier EM, Wang H, and Crawford JD. Three-dimensional eye-head coordination is implemented downstream from the superior colliculus. J Neurophysiol 89: 2839–2853, 2003.
Krommenhoek KP, Van Opstal AJ, Gielen CC, and Van Gisbergen JA. Remapping of neural activity in the motor colliculus: a neural network study. Vision Res 33: 1287–1298, 1993.[CrossRef][Web of Science][Medline]
Krommenhoek KP and Wiegerinck WA. A neural network study of precollicular saccadic averaging. Biol Cybern 78: 465–477, 1998.[CrossRef][Web of Science][Medline]
Lee C, Rohrer WH, and Sparks DL. Population coding of saccadic eye movements by neurons in the superior colliculus. Nature 332: 357–360, 1988.[CrossRef][Medline]
Lefevre P, Quaia C, and Optican LM. Distributed model of control of saccades by superior colliculus and cerebellum. Neural Networks 11: 1175–1190, 1998.[CrossRef][Web of Science][Medline]
Maunsell JHR and Van Essen DC. Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. J Neurophysiol 49: 1127–1147, 1983.
Mays LE and Sparks DL. Saccades are spatially, not retinocentrically, coded. Science 208: 1163–1165, 1980.
McLaughlin SC. Parametric adjustment in saccadic eye movements. Percept Psychophys 2: 359–362, 1967.[Web of Science]
Miller JM, Anstis T, and Templeton WB. Saccadic plasticity: parametric adaptive control by retinal feedback. J Exp Psychol Hum Percept Perform 7: 356–366, 1981.[CrossRef][Web of Science][Medline]
Mitchell J and Zipser D. A model of visual-spatial memory across saccades. Vision Res 41: 1575–1592, 2001.[CrossRef][Web of Science][Medline]
Moschovakis AK. Neural network simulations of the primate oculomotor system. II. Frames of reference. Brain Res Bull 40: 337–343, 1996.[CrossRef][Web of Science][Medline]
Moschovakis AK and Highstein SM. The anatomy and physiology of primate neurons that control rapid eye movements. Annu Rev Neurosci 17: 465–488, 1994.[CrossRef][Web of Science][Medline]
Nakamura K and Colby CL. Updating of the visual representation in monkey striate and extrastriate cortex during saccades. Proc Natl Acad Sci USA 99: 4026–4031, 2002.
Nichols JM and Sparks DL. Nonstationary properties of the saccadic system: new constraints on models of saccadic control. J Neurophysiol 73: 431–435, 1995.
Optican LM and Quaia C. Distributed model of collicular and cerebellar function during saccades. Ann NY Acad Sci 956: 164–177, 2002.[CrossRef][Web of Science][Medline]
Pouget A, Dayan P, and Zemel RS. Inference and computation with population codes. Annu Rev Neurosci 26: 381–410, 2003.[CrossRef][Web of Science][Medline]
Quaia C, Lefevre P, and Optican LM. Model of the control of saccades by superior colliculus and cerebellum. J Neurophysiol 82: 999–1018, 1999.
Quaia C and Optican LM. Commutative saccadic generator is sufficient to control a 3-D ocular plant with pulleys. J Neurophysiol 79: 3197–3215, 1998.
Robinson DA. Eye movement control in primates. Science 161: 1219–1224, 1968.
Robinson DA. The effect of cerebellectomy on the cat's vestibuloocular integrator. Brain Res 71: 195, 1974.[CrossRef][Web of Science][Medline]
Robinson DA. Oculomotor control signals. In: Basic Mechanisms of Ocular Motility and Their Clinical Implications (Wenner–Gren Centennial International Symposium Series), edited by Bach-y Nospaceritaqq P and Lennerstrand G. Oxford, UK: Pergamon, 1975, p. 337–374.
Robinson DA. Implications of neural networks for how we think about brain function. Behav Brain Sci 15: 644–655, 1992.[Web of Science]
Rumelhart DE, Hinton GE, and Williams RJ. Learning internal representations by error propagation. In: Parallel Distributed Processing, edited by Rumelhart DE and McClelland JL. Cambridge, MA: MIT Press, 1986, vol. 1, p. 318–362.
Russo GS and Bruce CJ. Neurons in the supplementary eye field of rhesus monkeys code visual targets and saccadic eye movements in an oculocentric coordinate system. J Neurophysiol 76: 825–848, 1996.
Russo GS and Bruce CJ. Supplementary eye field: representation of saccades and relationship between neural response fields and elicited eye movements. J Neurophysiol 84: 2605–2621, 2000.
Sabes PN, Breznen B, and Andersen RA. Parietal representation of object-based saccades. J Neurophysiol 88: 1815–1829, 2002.
Salinas E and Abbott LF. Coordinate transformations in the visual system: how to generate gain fields and what to compute with them. Prog Brain Res 130: 175–190, 2001.[Medline]
Scherberger H, Goodale MA, and Andersen RA. Target selection for reaching and saccades share a similar behavioral reference frame in the macaque. J Neurophysiol 89: 1456–1466, 2003.
Schnyder H, Reisine H, Hepp K, and Henn V. Frontal eye field projection to the paramedian pontine reticular formation traced with wheat germ agglutinin in the monkey. Brain Res 329: 151–160, 1985.[CrossRef][Web of Science][Medline]
Scott SH, Gribble PL, Graham KM, and Cabel DW. Dissociation between hand motion and population vectors from neural activity in motor cortex. Nature 413: 161–165, 2001.[CrossRef][Medline]
Scudder CA. A new local feedback model of the saccadic burst generator. J Neurophysiol 59: 1455–1475, 1988.
Smith MA and Crawford JD. Self-organizing task modules and explicit coordinate systems in a neural network model for 3-D saccades. J Comput Neurosci 10: 127–150, 2001a.[CrossRef][Web of Science][Medline]
Smith MA and Crawford JD. Implications of ocular kinematics for the internal updating of visual space. J Neurophysiol 86: 2112–2127, 2001b.
Snyder LH, Grieve KL, Brotchie P, and Andersen RA. Separate body- and world-referenced representations of visual space in parietal cortex. Nature 394: 887–891, 1998.[CrossRef][Medline]
Sparks DL. Functional properties of neurons in the monkey superior colliculus: coupling of neuronal activity and saccade onset. Brain Res 156: 1–16, 1978.[CrossRef][Web of Science][Medline]
Sparks DL. Neural cartography: sensory and motor maps in the superior colliculus. Brain Behav Evol 31: 49–56, 1988.[Web of Science][Medline]
Sparks DL. The neural encoding of the location of targets for saccadic eye movements. J Exp Biol 146: 195–207, 1989.
Sparks DL. The brainstem control of saccadic eye movements. Nat Rev Neurosci 3: 952–964, 2002.[CrossRef][Web of Science][Medline]
Sparks DL and Gandhi NJ. Single cell signals: an oculomotor perspective. Prog Brain Res 142: 35–53, 2003.[Web of Science][Medline]
Sparks DL, Holland R, and Guthrie BL. Size and distribution of movement fields in the monkey superior colliculus. Brain Res 113: 21–34, 1976.[CrossRef][Web of Science][Medline]
Sparks DL and Mays LE. Signal transformations required for the generation of saccadic eye movements. Annu Rev Neurosci 13: 309–336, 1990.[CrossRef][Web of Science][Medline]
Stanton GB, Goldberg ME, and Bruce CJ. Frontal eye field efferents in the macaque monkey. II. Topography of terminal fields in the midbrain and pons. J Comp Neurol 271: 493–506, 1988.[CrossRef][Web of Science][Medline]
Steinbach MJ. Proprioceptive knowledge of eye position. Vision Res 27: 1737–1744, 1987.[CrossRef][Web of Science][Medline]
Straumann D, Haslwanter T, Hepp-Reymond MC, and Hepp K. Listing's law for eye, head and arm movements and their synergistic control. Exp Brain Res 86: 209–215, 1991.[Web of Science][Medline]
Suzuki Y, Buttner-Ennever JA, Straumann D, Hepp K, Hess BJM, and Henn V. Deficits in torsional and vertical rapid eye movements and shift of Listing's plane after uni- and bilateral lesions of the rostral interstitial nucleus of the medial longitudinal fasciculus. Exp Brain Res 106: 215–232, 1995.[Web of Science][Medline]
Suzuki Y, Straumann D, Simpson JI, Hepp K, Hess BJ, and Henn V. Three-dimensional extraocular motoneuron innervation in the rhesus monkey. I: Muscle rotation axes and on-directions during fixation. Exp Brain Res 126: 187–199, 1999.[CrossRef][Web of Science][Medline]
Tweed D. Three-dimensional model of the human eye-head saccadic system. J Neurophysiol 77: 654–666, 1997.
Tweed DB and Vilis T. Implications of rotational kinematics for the oculomotor system in three dimensions. J Neurophysiol 58: 832–849, 1987.
Tweed DB and Vilis T. The superior colliculus and the spatiotemporal translation in the saccadic system. Neural Networks 3: 75–86, 1990a.[CrossRef][Web of Science]
Tweed DB and Vilis T. Geometric relations of eye position and velocity vectors during saccades. Vision Res 30: 111–127, 1990b.[CrossRef][Web of Science][Medline]
Umeno MM and Goldberg ME. Spatial processing in the monkey frontal eye field. I. Predictive visual responses. J Neurophysiol 78: 1373–1383, 1997.
Van Gisbergen JA, Robinson DA, and Gielen S. A quantitative analysis of generation of saccadic eye movements by burst neurons. J Neurophysiol 3: 417–442, 1981.
Van Gisberger JAM, Van Opstal AJ, and Tax AAM. Collicular ensemble coding of saccades based on vector summation. Neuroscience 21: 541–555, 1987.[CrossRef][Web of Science][Medline]
Van Opstal AJ, Hepp K, Hess BJ, Straumann D, and Henn V. Two-rather than three-dimensional representation of saccades in monkey superior colliculus. Science 252: 1313–1315, 1991.
Van Opstal AJ, Hepp K, Suzuki Y, and Henn V. Influence of eye position on activity in monkey superior colliculus. J Neurophysiol 74: 1593–1610, 1995.
Van Opstal AJ, Hepp K, Suzuki Y, and Henn V. Role of monkey nucleus reticularis tegmenti pontis in the stabilization of Listing's plane. J Neurosci 16: 7284–7296, 1996.
Vilis T, Yates S, and Hore J. Visual patching of one eye produces changes in saccadic properties in the unseeing eye. Brain Res 349: 290–292, 1985.[CrossRef][Medline]
Waitzman DM, Ma TP, Optican LM, and Wurtz RH. Superior colliculus neurons mediate the dynamic characteristics of saccades. J Neurophysiol 66: 1716–1737, 1991.
Walker MF, FitzGibbon EJ, and Goldberg ME. Neurons in the monkey superior colliculus predict the result of impending saccadic eye movements. J Neurophysiol 73: 1988–2003, 1995.
Westheimer G. Kinematics of the eye. J Opt Soc Am 47: 967–974, 1957.[Medline]
Westheimer G and Blair SM. Oculomotor defects in cerebellectomized monkeys. Invest Ophthalmol 12: 618, 1973.
White RL III and Snyder LH. A neural network model of flexible spatial updating. J Neurophysiol 91: 1608–1619, 2004.
Wurtz RH and Albano JE. Visual-motor function of the primate superior colliculus. Annu Rev Neurosci 3: 189–226, 1980.[CrossRef][Web of Science][Medline]
Xing J and Andersen RA. Memory activity of LIP neurons for sequential eye movements simulated with neural networks. J Neurophysiol 84: 651–665, 2000a.
Xing J and Andersen RA. Models of the posterior parietal cortex which perform multimodal integration and represent space in several coordinate frames. J Cogn Neurosci 12: 601–614, 2000b.[CrossRef][Web of Science][Medline]
Zee DS, Optican LM, Cook JD, Robinson DA, and Engel WK. Slow saccades in spinocerebellar degeneration. Arch Neurol 33: 243–251, 1976.
Zipser D and Andersen RA. A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature 331: 679–684, 1988.[CrossRef][Medline]
This article has been cited by other articles:
|
|
 |
|
  G. Blohm, G. P. Keith, and J. D. Crawford Decoding the Cortical Transformations for Visually Guided Reaching in 3D Space Cereb Cortex, June 1, 2009; 19(6): 1372 - 1393. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Brozovic, A. Gail, and R. A. Andersen Gain Mechanisms for Contextually Guided Visuomotor Transformations J. Neurosci., September 26, 2007; 27(39): 10588 - 10596. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. G. Constantin, H. Wang, J. C. Martinez-Trujillo, and J. D. Crawford Frames of Reference for Gaze Saccades Evoked During Stimulation of Lateral Intraparietal Cortex J Neurophysiol, August 1, 2007; 98(2): 696 - 709. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. F. Walker, J. Tian, and D. S. Zee Kinematics of the Rotational Vestibuloocular Reflex: Role of the Cerebellum J Neurophysiol, July 1, 2007; 98(1): 295 - 302. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. L White III and L. H Snyder Spatial constancy and the brain: insights from neural networks Phil Trans R Soc B, March 29, 2007; 362(1479): 375 - 382. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. M. Klier, H. Wang, and J. D. Crawford Interstitial Nucleus of Cajal Encodes Three-Dimensional Head Orientations in Fick-Like Coordinates J Neurophysiol, January 1, 2007; 97(1): 604 - 617. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Vesia, J. A. Monteon, L. E. Sergio, and J. D. Crawford Hemispheric Asymmetry in Memory-Guided Pointing During Single-Pulse Transcranial Magnetic Stimulation of Human Parietal Cortex J Neurophysiol, December 1, 2006; 96(6): 3016 - 3027. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Schoppik and S. G. Lisberger Saccades exert spatial control of motion processing for smooth pursuit eye movements. J. Neurosci., July 19, 2006; 26(29): 7607 - 7618. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. T. Crane, J. Tian, and J. L. Demer Kinematics of Vertical Saccades during the Yaw Vestibulo-ocular Reflex in Humans Invest. Ophthalmol. Vis. Sci., August 1, 2005; 46(8): 2800 - 2809. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
