JN Journal of Applied Physiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


J Neurophysiol 92: 1165-1174, 2004; doi:10.1152/jn.01245.2003
0022-3077/04 $5.00
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (23)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Hatsopoulos, N.
Right arrow Articles by O'Leary, J. G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Hatsopoulos, N.
Right arrow Articles by O'Leary, J. G.

Decoding Continuous and Discrete Motor Behaviors Using Motor and Premotor Cortical Ensembles

Nicholas Hatsopoulos1, Jignesh Joshi2 and John G. O'Leary1

1Department of Organismal Biology and Anatomy, University of Chicago, 60637; and 2Pritzker Institute of Biomedical Sciences and Engineering, Illinois Institute of Technology, Chicago, Illinois 60616

Submitted 22 December 2003; accepted in final form 4 March 2004


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
Decoding motor behavior from neuronal signals has important implications for the development of a brain–machine interface (BMI) but also provides insights into the nature of different movement representations within cortical ensembles. Motor control can be hierarchically characterized as the selection and planning of discrete movement classes and/or postures followed by the execution of continuous limb trajectories. Based on simultaneous recordings in primary motor (MI) and dorsal premotor (PMd) cortices in behaving monkeys, we demonstrate that an MI ensemble can reconstruct hand or joint trajectory more accurately than an equally sized PMd ensemble. In contrast, PMd can more precisely predict the future occurrence of one of several discrete targets to be reached. This double dissociation suggests that a general-purpose BMI could take advantage of multiple cortical areas to control a wider variety of motor actions. These results also support the hierarchical view that MI ensembles are involved in lower-level movement execution, whereas PMd populations represent the early intention to move to visually presented targets.


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
Recent developments in brain–machine interface (BMI) technology have demonstrated how the activity of a cortical ensemble can be used to control the position of an external device in real time. These efforts have mainly focused on the continuous control of a moving object such as a computer cursor or the end effector of a robotic arm (Carmena et al. 2003Go; Kennedy et al. 2000Go; Kostov and Polak 2000Go; Serruya et al. 2002Go; Taylor et al. 2003Go, 2002Go; Wolpaw et al. 1991Go). This is clearly a useful function for a BMI, given that many forms of motor behavior such as writing, drawing, and reaching require precise trajectory and path control. There are, however, other motor acts that can be viewed as equivalence classes, which may require discrete control to be selected (Buneo et al. 2003Go; Pesaran et al. 2002Go; Shenoy et al. 2003Go). These include movement initiation and termination, typing, and discrete grasp and postural configurations. It would thus seem necessary that a general-purpose BMI would require both forms of control mechanisms. Moreover, different cortical areas may be ideally suited for these different forms of control. Based on dual electrophysiological recordings in primary motor (MI) and dorsal premotor (PMd) cortices, we demonstrate a double dissociation in which ensemble activity in MI more accurately reconstructs continuous movement, whereas PMd ensemble activity can more effectively predict upcoming movements to discrete targets.


    METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
Behavioral tasks

Two macaque monkeys (Macaca mulatta) were operantly trained to perform an instructed-delay 8-direction center-out (CO) task and a random-sequence (RS) task by moving a cursor to targets by contralateral arm movements. Both tasks involved projecting the cursor and targets onto a horizontal, reflective surface in front of the monkey above the monkey's hand. The monkey's arm rested on cushioned arm troughs secured to links of a 2-joint robotic arm (KINARM system; Scott 1999Go) underneath the projection surface. The shoulder joint was abducted 90° such that shoulder and elbow flexion and extension movements were made in the horizontal plane.

The CO task involved movements from a center target to one of 8 peripherally positioned targets (5–7 cm distance). The task consisted of 3 epochs: 1) a 500-ms hold period during which the monkey was required to hold its hand over the center target, 2) a fixed instruction period (600 or 1,000 ms depending on the recording session) during which one of the 8 final targets appeared but the monkey was not allowed to move, and 3) a "GO" period during which the target began blinking. This informed the monkey to begin moving to the peripherally positioned target.

In the RS task, a sequence of 7 targets appeared on the projection surface. At any one time, a single target appeared at a random location in the workspace, and the monkey was required to move to it (Fig. 1A). As soon as the cursor reached the target, the target disappeared and a new target appeared in a new, random location. After reaching the 7th target, the monkey was rewarded with a drop of water. During this task the workspace was sampled thoroughly, which was important in developing a hand position decoder that could generalize adequately (Fig. 1B).



View larger version (18K):
[in this window]
[in a new window]
 
FIG. 1. Random-sequence (RS) task used to decode continuous hand position. A: schematic diagram depicting a sequence of 3 of the 7 randomly positioned targets (black squares) to which the monkey's hand is required to reach by the cursor (gray circle) on one trial of the RS task. A water reward is offered to the animal once the 7th target is reached. Each trial is composed of a novel sequence of 7 randomly positioned targets. B: actual hand positions traced out over multiple trials of the RS task, demonstrating a thorough sampling of the workspace.

 
Electrophysiology

Two silicon-based electrode arrays composed of 100 electrodes (1.0 mm electrode length; 400 µm interelectrode separation) were implanted in the arm areas of MI and PMd of each monkey (see Maynard et al. 1999Go for more details concerning the electrode array) (Fig. 2A). During a recording session, signals from ≤128 electrodes were amplified (gain, 150) and recorded digitally (14-bit) at 30 kHz per channel using a Cerebus acquisition system (Cyberkinetics, Salt Lake City, UT). Only waveforms (1.6 ms in duration) that crossed a threshold were stored and spike-sorted using Off-line Sorter (Plexon, Dallas, TX). Interspike interval histograms were computed to verify single-unit isolation by ensuring that <0.05% of waveforms possessed an interspike interval <1.6 ms. Figure 2B shows average (±1 SD) waveforms from one representative data set. Figure 2C shows the distribution of signal-to-noise ratios for this data set. Signal-to-noise ratios were defined as the difference in mean peak-to-trough voltage divided by twice the SD. All isolated single units used in this study possessed signal-to-noise ratios (rounded to the nearest whole number) of 3:1 or higher. A total of 10 data sets (7 using the RS and 3 using the CO tasks) for animal B and 4 data sets (2 using the RS and 2 using the CO tasks) for animal R were analyzed, where a data set is defined as all simultaneously recoded neural data collected in one recording session. Each data set contained between 32 and 143 simultaneously recorded units from MI and PMd (Table 1). The data sets used to decode continuous hand position contained equal numbers of units from MI and PMd by selecting the units with the largest signal-to-noise ratios that made both ensembles of equal size. Thus both ensembles consisted of "randomly" selected units from MI and PMd except for a possible bias for neurons with large cell bodies that would generate higher signal-to-noise ratios. All of the surgical and behavioral procedures were approved by the University of Chicago's IACUC and conform to the principles outlined in the Guide for the Care and Use of Laboratory Animals (National Institutes of Health publication no. 86–23, revised 1985).



View larger version (84K):
[in this window]
[in a new window]
 
FIG. 2. Dual multielectrode array recording from the primary motor (MI) and dorsal premotor cortices (PMd). A: placement of the 2 arrays in the right hemisphere of monkey B. Top right array is placed on the arm area of MI whereas the bottom left array is placed in the arm areas of PMd between the arcuate spur and the precentral dimple. CS, central sulcus. B: average (±1 SD) action potential waveforms from 20 single units in MI (left) and 20 units in PMd recorded simultaneously. C: distribution of signal-to-noise ratios from the 40 units shown in B. Signal-to-noise ratio is defined as the ratio between the average amplitude (peak-to-trough) and twice the SD.

 

View this table:
[in this window]
[in a new window]
 
TABLE 1. Data sets used in this study

 
Kinematics

The shoulder and elbow joint angles were sampled at 500 Hz by the robotic arm's motor encoders and acquired along with the neural data. The x and y positions of the hand were computed by transforming the joint angles into Cartesian endpoint positions using the forward kinematics equations for a 2-joint arm with the measured lengths of the upper arm and forearm of the monkey.

Decoding algorithms

Continuous reconstruction.    Reconstructions of Cartesian x and y endpoint positions were computed from the neural data using linear and nonlinear regression methods. The endpoint position at time t (pt) was predicted from the neural responses (rt–in) of multiple neurons (n) at multiple time points (ti) in the past. In the linear case, the optimal set of linear filter coefficients (fin) was found that minimized the sum squared difference between the predicted position (t) and the actual position using a closed-form solution (Warland et al. 1997Go)

(1)
where N is the total number of simultaneously recorded neurons and L is the filter length in number of bins. The neural response of an individual neuron at one time point was computed as the number of spikes evoked in a time bin of length B. The 3 parameters, N, LT (filter length in time, LT = L*B), and B, were systematically varied in this study: N ranging from 1 to 44 neurons, LT ranging from 0.2 to 1.4 s, and B ranging from 40 to 200 ms. As an example, a filter length of 1 s with a bin of 50 ms resulted in a filter composed of 20 bins. The performance of the linear model on each data set was estimated using 10-fold cross-validation, wherein the data set was randomly partitioned into 10 "folds," or equally sized disjoint subsets, and for each fold, a model was built leaving that fold out as testing data and using the remaining 9 folds as training data. Because the division into folds is random, the 10-fold cross validation procedure was repeated 10 times to better estimate the performance of the model.

In the nonlinear case, a 2-layer neural network was implemented using the Matlab Neural Network Toolbox. The input layer consisted of N x L input nodes representing the responses of multiple neurons at multiple time points. Sigmoidal activation functions were used between the input and hidden layer. Linear activation functions were used between the hidden and output layer, which consisted of one node representing the predicted position at time t, t. The number of hidden nodes, the learning algorithm, and the learning rate were selected using cross-validation techniques on one data set and then were used in the networks built for 4 other data sets, 2 from each animal. Ten hidden units, a learning rate of 0.01, and the scaled conjugate gradient (gradient descent) learning algorithm were selected as network parameters. Each data set was randomly partitioned into a training set (80% of the data), a validation set (10% of the data), and a test set (the remaining 10%), and the weights of each network were then trained so that the sum squared difference between the actual and predicted positions for the training set was minimized (although a global minimum was not necessarily reached). To prevent overfitting, the sum squared error for the validation set was computed after each training epoch, and training was stopped when the value began to increase. The test set was then used to estimate the generalization performance of the trained network. Because both the initialization of network weights and the division into training, test, and validation sets are random, the training and testing procedure was performed 10 times for each network.

Discrete classification.    Parametric and nonparametric classifiers were developed to assign individual trial responses to one of the 8 target/direction classes in the CO task. In the parametric case, a probability distribution model conditioned on each target/direction class was estimated for the response of a single neuron. Both Poisson and Gaussian distributions were used but the Poisson model generally performed better than the Gaussian, and so results from the Gaussian model are not presented. For the Poisson model, a mean (µ) parameter was estimated based on the sample mean response. Assuming conditional independence among the neurons, a joint probability distribution for the combined responses over all neurons was computed by multiplying the single-neuron distributions for each direction. Conditional independence is an approximation. However, spike count correlation coefficients were generally small between pairs of neurons during a 400-ms window in the instruction period. Average pairwise correlation coefficients (averaged over all pairs and directions in a data set) ranged from 0.002 to 0.02 for all but one data set, which had slightly higher but still small values ranging from 0.05 to 0.11. A maximum-likelihood classification rule was used such that a trial response was assigned to the target/direction class that maximized the likelihood of observing the response. Accuracy of the classifiers was estimated using leave-one-out cross-validation, wherein the class of each trial was predicted from a classifier built using all other trials, and the percentage of trials that were classified correctly is then computed to estimate the model performance.

A nonparametric 2-layer neural network similar to that described above (with sigmoidal activation functions at the output layer instead of a linear activation function) was used to classify individual trial responses into one of 8 target/direction categories. The output layer of the network possessed 8 nodes, one for each category. The training data used for the output layer consisted of an 8-dimensional vector with a value of 1 for the correct category and –0 for the other categories. During testing, the trial response was assigned to the category whose node was maximally activated. Results using the neural network on one data set demonstrated a similar improved performance using PMd versus MI and thus are not reported.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
Continuous decoding

We systematically varied the bin size B and the filter length LT, used to build the linear filter decoder to find the optimal set of parameters (Fig. 3). Although there were slight variations with bin size as well as variations depending on the decoded parameter and the cortical area used, we found that performance depended primarily on filter length. A filter length of 1 s was near optimal in terms of accounting for the largest percentage of the hand position variance and was used for all reported analyses.



View larger version (38K):
[in this window]
[in a new window]
 
FIG. 3. Continuous decoding performance as a function of filter length and bin size. A 3-D plot of the percentage of x hand position variance accounted for (r2) on cross-validation data using the optimal linear filter decoder on 20 single units in MI as a function of the filter length (in s) and bin size (in s).

 
We then compared the performance of a hand position decoder in reconstructing the x and y trajectories of the endpoint from two equally sized ensembles of MI and PMd neurons. A linear filter decoder reconstructed the current hand position using spike counts measured in 50-ms bins extending back 1 s in the past. The decoder was built using a random sample of 90% of successful trials and tested on the remaining 10%. Figure 4 shows the x and y trajectories for a particular 20-s test sample along with the neural reconstructions from the 2 ensembles. The MI ensemble reconstructs the trajectories more accurately, particularly for large deviations of the hand. To quantify this difference, we computed the mean (average over ten 10-fold cross-validation runs) percentage of total hand position variance accounted for (r2) on the test trials by each of the ensembles over 9 data sets (Fig. 5, A and B). MI ensembles (47 and 50% for x and y positions, respectively) consistently accounted for a larger percentage of the variance than did the PMd ensembles (difference of, 31% and 27% for x and y positions, respectively) with an average improvement of 50%, and 82% for x and y trajectories, respectively. The r2 using the MI ensemble was significantly higher than that of an equally sized PMd ensemble for all 9 data sets (P < 0.01, one-tailed t-test). A wide range of filter shapes and sizes associated with each neuron were observed. However, the absolute filter values averaged over all neurons indicate that the filter coefficients were generally weaker at time points further in the past (Fig. 5, A and B, insets).



View larger version (34K):
[in this window]
[in a new window]
 
FIG. 4. Reconstructions of hand position from simultaneously recorded neuronal ensembles from MI and PMd. Actual x (A) and y (B) hand trajectories (solid) are decoded from an ensemble of 20 units in MI (dashed) and 20 units in PMd (dotted). Neural and hand position data shown represent cross-validation data not used to build the linear filter decoder.

 


View larger version (24K):
[in this window]
[in a new window]
 
FIG. 5. Comparison in continuous decoding performance using equally sized ensembles from MI and PMd. A: r2 averaged over 10 repetitions of the 10-fold cross-validation procedure (see METHODS for description of cross-validation procedure) using the linear decoder for x hand position from MI (black) and PMd (white) ensembles from 9 data sets over 2 monkeys. B: same as A, except for y hand position. Error bars represent SDs over the 10 repetitions. MI ensembles consistently account for a larger percentage of the hand position variance than do equally sized PMd ensembles. Insets in A and B plot the absolute values of the filter coefficients averaged over all recorded neurons. Error bars represent SE. C: r2 averaged over 10 repetitions of the 10-fold cross-validation procedure using the neural network decoder. Error bars represent SE. Asterisks denote significant differences between MI and PMd ensembles (P < 0.05, one-tailed t-test).

 
Similar decoding results were observed using the nonlinear neural network on 4 of the data sets (Fig. 5C). A thorough analysis of nonlinear decoding algorithms (by searching all network parameters as well as by using parametric models) is beyond the scope of this study. Therefore we cannot conclude that the mapping between neural activity and hand position is essentially linear.

To examine the effect of ensemble size on decoding accuracy, we used a neuron-dropping procedure in which subsets of neurons were randomly sampled and used to build the decoder and cross-validated on 10% of the trials. Figure 6A (left and middle panels) show scatterplots of reconstruction performance values for one data set (measured in terms of r2) using N unique subsets for each ensemble size, where N is the total number of recorded neurons. We termed these subsets "unique subsets" because they were chosen so that no two subsets of size N were identical. That is, if AiK is the ith subset of K neurons, then at least one neuron that is a member of AiK is not a member of AjK, whenever i != j. Hyperbolic functions [f(x) = ax/(1 + ax)] (Wessberg et al. 2000Go) were fit to the mean reconstruction performance (Fig. 6A, third panels) values to extrapolate to near-perfect performance (Fig. 6A, fourth panels). Based on all 9 data sets, we predict that an ensemble size of 256 ± 38 (mean ± SE) in MI could account for 90% of the x hand position variance as compared with 438 ± 32 neurons in PMd. For the y hand position, the difference was even larger, with 222 ± 53 neurons in MI versus 710 ± 168 neurons in PMd accounting for 90% of the variance.



View larger version (31K):
[in this window]
[in a new window]
 
FIG. 6. Continuous decoding performance as a function of ensemble size. A: percentage of x (top panels) and y (bottom panels) hand position variance accounted for using random subsets of neurons of different sizes from MI (first panels) and PMd (second panels). Linear filter decoder was used with a bin size of 100 ms and a filter length of 1 s. Each point in the scatterplots represents the decoding performance from a random but unique subset of all the recorded neurons. Third panels directly compare the performance averaged over all subsets (asterisks) of MI and PMd ensembles along with the best-fit hyperbolic function (solid lines). Fourth panels plot the best-fit hyperbolic functions extrapolated out to ensemble sizes that would account for 90% of the hand position variance. Average performances based on the data are shown with asterisks. Oval curve in the left panel surrounds 2 individual neurons that account for the largest percentage of the variance. All points above the dashed line contain one or both of the 2 neurons that individually account for the largest percentage of the variance. Data are based on data set b030521 that contains a total of 20 neurons in each cortical area. B: same as A, except based on data set r031206 that contains 44 neurons in each cortical area. Third panels show the best-fit hyperbolic functions to the mean performance and the fourth panels show the best-fit logarithmic functions.

 
Despite the monotonic improvement in mean reconstruction performance with ensemble size, the scatterplots in Fig. 6A indicate a highly uneven distribution of performance, suggesting that certain neurons or groups of neurons perform much better than others. For example, the 2 neurons surrounded by the oval in the lower left-hand panel accounted for over 20% of the y position variance individually despite the average performance of approximately 5% for the remaining individual neurons. These high-performing "outlier" neurons were observed in all data sets in both monkeys. The high-performing neuron ensembles that appear above the dashed line in Fig. 6A (lower left-hand panel) always contained one of both of the two "outlier" neurons in this data set.

Hyperbolic functions have been shown previously to fit the monotonically increasing performance with ensemble size (Wessberg et al. 2000Go). However, there were clear deviations from a hyperbolic fit for the one data set that contained many more units in both cortical areas (r031206). Figure 6B shows the decoding performance as a function of ensemble size for this data set. The best-fit hyperbolic functions (third panels in Fig. 6B) overestimated the actual performance of the both cortical areas when the ensemble size was large. The fourth panels show the best-fit logarithmic function to the mean performance. Using this function for this data set to extrapolate in MI suggests that 402 neurons (averaged over x and y) would be required to account for 90% of the hand position variance as opposed to 204 neurons based on the hyperbolic fit. In PMd, 970 neurons would be required as opposed to 331 neurons using the best-fit hyperbolic function.

Discrete decoding

As previous studies have shown, neurons in PMd exhibit robust activity related to movement planning. By aligning perievent histograms on the onset of the instruction signal in the instructed-delay CO task, we often observed transient "signal-related" activity lasting about 300–400 ms whose magnitude varied with target direction (Fig. 7A, top) (Weinrich and Wise 1982Go). This early instruction-related activity was generally either weakly present or entirely absent in MI (Fig. 7A, bottom). To quantify this difference in "signal-related" activity, Fig. 7B compares the firing rate modulation (i.e., percentage change relative to the baseline firing rate in the first 100 ms after the instruction signal onset) averaged over the ensemble of recorded PMd neurons (solid line) and simultaneously recorded MI neurons (dashed line) for one data set.



View larger version (23K):
[in this window]
[in a new window]
 
FIG. 7. A: perievent time histograms from 3 single units in PMd (top panels) and 3 simultaneously recorded units in MI (bottom panels) over the 8 target directions in the center-out task. Histograms are aligned on the onset of the instruction signal during the instructed-delay period. Notice the "signal-related" transient modulation that occurs in PMd neurons during the first 500 ms of the instructed-delay period that is not present in MI neurons. Arrows point to the average movement onset time. B: firing rate modulation (i.e., change in firing rate from the baseline rate measured during the first 100 ms after instruction signal onset) averaged over the ensemble of PMd neurons (solid line, n = 28 neurons) and MI neurons (dashed line, n = 53 neurons) recorded in one data set. x-axis measures the time (in s) relative to the onset of the instruction signal. Error bars represent SE.

 
We compared the ability of equally sized MI and PMd ensembles to predict target/direction during the instruction delay period before movement onset. We developed a probabilistic model of the ensemble activity conditioned on each of the target directions and then used a maximum-likelihood rule to classify single trials of data into one of the 8 target/direction categories. Figure 8 plots the classification performance of both ensembles using spike counts measured in a sliding 200-ms window incremented in 10-ms steps. In contrast to continuous reconstruction, the PMd ensemble attained higher decoding performance compared to that of the MI ensemble. This was systematically examined over all data sets by using 20 random subsets of equally sized (20 neurons) ensembles of MI and PMd neurons and comparing their classification performance using a single 400-ms window of data beginning 100 ms after the onset of the instruction signal (Fig. 9A). The percentage of correctly classified trials using PMd was significantly higher than that of MI for all data sets (P < 0.01, one-tailed t-test). Classification performance was also compared using a single 1,000-ms window of data beginning at the instruction signal onset for the 4 data sets (2 data sets per animal) with a 1,000-ms instruction period (Fig. 9B). PMd ensembles continued to classify target/direction significantly better than equally sized MI ensembles, although the differential performance was not as strong.



View larger version (19K):
[in this window]
[in a new window]
 
FIG. 8. Discrete direction decoding as a function of time. Average percentage correct classification is plotted every 10 ms using 200-ms bins of spike count data from equally sized (29 neurons) ensembles from MI (dashed line) and PMd (solid line). Time is plotted with respect to the onset of the instruction signal occurring at the beginning of the instructed-delay period in the center-out task. A maximum-likelihood classifier assuming independent, Poisson firing neurons was used. Classification performance was cross-validated by testing on one trial that was left out when building the classifier and then repeating this procedure over all trials. Therefore performance represents the percentage correct classification averaged over all cross-validated trials.

 


View larger version (17K):
[in this window]
[in a new window]
 
FIG. 9. Summary results of discrete classification performance. A: average correct classification based on a single 400-ms bin of spike count data starting 100 ms after the instruction signal onset over 5 data sets from 2 monkeys. Classification performance is the average (±1 SE) over 20 random subsets of 20 neurons within each ensemble. B: average correct classification based on a single 1,000-ms bin of spike count data starting at the instruction signal onset over 4 data sets from 2 monkeys in which the instruction period was 1,000 ms. As in A, the classification performance is the average (±1 SE) over 20 random subsets of 20 neurons within each ensemble.

 
This performance difference was also evident by examining the effect of ensemble size on classification (Fig. 10). Despite considerable scatter in performance over different unique subsets of neurons (Fig. 10, left and middle panels), the average performance increased monotonically as in the continuous decoding case. However, instead of a hyperbolic function the effect of ensemble size on classification performance was fit well with a power function (PCC = a x Nb, where PCC is the percentage of correctly classified trials and N is the number of neurons in the ensemble) (Fig. 10, right panel). The mean exponent b, from the best-fit power functions, was 0.23 for MI versus 0.30 for PMd. Extrapolation using this power relationship suggested that an ensemble of 418 ± 61 (mean ± SE) neurons in PMd would be necessary to attain 90% correct classification performance as compared with 32,858 ± 30,247 neurons in MI. The large SE for MI is attributable to an outlier data set (b1030314), which indicated that 153,799 neurons would be required to attain 90% correct classification performance. After removing this outlier data set, however, an ensemble size of 2,623 ± 1,074 in MI would be required to attain 90% performance as compared with 425 ± 69 in PMd.



View larger version (14K):
[in this window]
[in a new window]
 
FIG. 10. Classification performance as a function of ensemble size. A: classification performance using random but unique subsets of neurons of different sizes from MI (left panel) and PMd (middle panel). Each point in the scatterplots represents the decoding performance from a random but unique subset of all the recorded neurons. Right panel directly compares the performance averaged over all subsets (asterisks) of MI and PMd along with the best-fit power functions (solid lines).

 

    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
We have provided evidence for a double dissociation in function between ensembles of MI and PMd neurons such that primary motor cortex activity is more suitable for reconstructing a dynamically varying endpoint, whereas premotor cortex is more effective in predicting discrete visual targets to be reached. Motor control can be characterized hierarchically as the selection and planning of discrete movement types or goals followed by the execution of dynamic, time-varying motor output. The results of this study are consistent with the prevailing view that PMd neurons represent the early selection of movements to visual targets, whereas MI neurons encode lower-level aspects of movement execution. These previous studies, however, have focused on the encoding problem characterizing how motor behavior affects the firing properties of single neurons. The current study extends this view by examining the inverse problem of decoding motor behavior from single trials of simultaneously recorded ensembles in MI and PMd. This is the perspective the nervous system is faced with when selecting motor actions and controlling the peripheral motor apparatus. Schwartz and colleagues have recently observed a double dissociation in function between MI and ventral premotor cortex (PMv) that is similar in spirit to our findings (Schwartz et al. 2004Go). By changing the gain relating the motion of the arm to the visually observed cursor controlled by the arm, they observed that a population of MI neurons reconstructed the actual movement of the arm more accurately, whereas a population of PMv neurons more accurately reconstructed the visual trajectory of the cursor.

Our findings, however, are inconsistent with previous data from Nicolelis and colleagues (Wessberg et al. 2000Go) who indicated that PMd ensembles could reconstruct hand position more accurately than MI. The reason for this discrepancy is unclear. There are at least 2 possible differences in experimental protocol that may explain the inconsistency. First, they used a different species of monkey (owl monkey vs. rhesus macaque that we used) in their study. Second, the task they used to quantify the functional differences between MI and PMd was one-dimensional, in which the monkey moved a lever either to the left or right. Recently, however, the Nicolelis group published a study using rhesus macaques that performed a 2-D task very similar to our random-sequence task (Carmena et al. 2003Go). In that study, they observed that hand position reconstructions were more accurate using neuronal ensembles in MI versus PMd. Their results were similar to ours, indicating approximately 70% improvement using an MI ensemble as compared with an equally sized ensemble in PMd.

At first glance, the shapes of the optimal decoding filters generated in the random-sequence task for PMd neurons appear to be counterintuitive. Although there was considerable variation in linear filter shape across individual neurons in both MI and PMd, the absolute filter coefficient magnitude tended to decrease from the immediate past to the distant past for both areas (see insets in Fig. 5, A and B). It might have been predicted that the filter magnitudes associated with PMd neurons would be larger for time points in the distant past. The observed filter shapes suggest that PMd activity in the intermediate and distant past does not correlate well with the current kinematics of the hand that are being decoded in the random-sequence task but rather may carry information about the target to be reached as in the center-out task. This is consistent with previous data showing that PMd neuronal activity is generally not related to the on-line control of arm movements as MI is (Crammond and Kalaska 1996Go). We are currently investigating whether PMd activity may be used to predict the position of the current target in the RS task.

The effect of ensemble size on decoding performance demonstrated that less than half the number of neurons in MI would be required to attain ideal performance (i.e., accounting for 90% of the hand position variance) as compared with PMd in the continuous case, whereas a 6th the number of PMd neurons would be required to correctly classify 90% of trials during the preparatory period in the discrete case as compared with MI. These results are based on extrapolation using functions that fit the data. Extrapolation, however, should be viewed with extreme caution because the data may deviate from the best-fit functions (using smaller groups of neurons) as the ensemble sizes grow larger. This was made clear in Fig. 6B in which the hyperbolic function overestimated the continuous decoding performance for the one data set that contained many more neurons. Extrapolation can also amplify small differences, as is evident in the large variability in the predicted number of MI neurons required to attain 90% correct classification in the discrete CO task. In the future, we plan to examine these issues by recording from larger groups of neurons within each cortical area.

The early planning activity to visual targets observed in PMd is similar to findings by Andersen and colleagues who measured early movement "intentional" signals in an area called the parietal reach region (PRR) partially overlapping with the medial intraparietal area (MIP) of the posterior parietal cortex (Batista et al. 1999Go). This is, perhaps, not surprising given the direct anatomical connections between MIP and dorsal premotor cortex (Matelli and Luppino 2001Go). A similar probabilistic classifier was shown to accurately predict upcoming movement direction using spiking activity from serially recorded neurons within PRR (Shenoy et al. 2003Go). There may be differences, however, in the coordinate systems used in each area. Arm movements appear to be encoded in eye-centered coordinates in PRR (Batista et al. 1999Go). Although eye position appears to influence activity in PMd when monkeys are required to fixate (Boussaoud et al. 1998Go), the effects are quite small when monkeys are free to change their gaze (Cisek and Kalaska 2002Go).

Our results have implications for the development of a brain–machine interface. There are discrete control tasks such as selecting from a number of distinct buttons or attaining a particular postural arm or hand configuration that can take advantage of the early movement-selection properties of PMd ensembles. On the other hand, a BMI may also be faced with controlling continuously varying motor variables such as guiding a cursor on a screen, moving a robotic arm in space, or navigating an autonomous robot on the ground, which could be handled using single units from MI. A general-purpose BMI could benefit from the complementary nature of motor and premotor ensemble activity.


    GRANTS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
This work was funded by the Whitehall Foundation, the Brain Research Foundation, and National Institute of Neurological Disorders and Stroke Grant N01-NS-2-2345.


    DISCLOSURE
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
N. Hatsopoulos has stock ownership in a company, Cyberkinetics, Inc., that is commercializing neural prostheses for severely motor disabled people.


    ACKNOWLEDGMENTS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
We thank Z. Haga and D. Paulsen for help with the surgical implantation of the arrays, training of monkeys, and data collection. We also thank M. Fellows and E. Gunderson for help with the surgical procedures.


    FOOTNOTES
 
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Address for reprint requests and other correspondence: N. G. Hatsopoulos, Dept. of Organismal Biology and Anatomy, University of Chicago, Chicago, IL 60637 (E-mail: nicho{at}uchicago.edu).


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 DISCLOSURE
 ACKNOWLEDGMENTS
 REFERENCES
 
Batista AP, Buneo CA, Snyder LH, and Andersen RA. Reach plans in eye-centered coordinates. Science 285: 257–260, 1999.[Abstract/Free Full Text]

Boussaoud D, Jouffrais C, and Bremmer F. Eye position effects on the neuronal activity of dorsal premotor cortex in the macaque monkey. J Neurophysiol 80: 1132–1150, 1998.[Abstract/Free Full Text]

Buneo CA, Jarvis MR, Batista AP, and Andersen RA. Properties of spike train spectra in two parietal reach areas. Exp Brain Res 153: 134–139, 2003.[CrossRef][Web of Science][Medline]

Carmena JM, Lebedev MA, Crist RE, O'Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez CS, and Nicolelis MA. Learning to control a brain–machine interface for reaching and grasping by primates. Publ Library Sci Biol 1: 1–16, 2003.

Cisek P and Kalaska JF. Modest gaze-related discharge modulation in monkey dorsal premotor cortex during a reaching task performed with free fixation. J Neurophysiol 88: 1064–1072, 2002.[Abstract/Free Full Text]

Crammond DJ and Kalaska JF. Differential relation of discharge in primary motor cortex and premotor cortex to movements versus actively maintained postures during a reaching task. Exp Brain Res 108: 45–61, 1996.[Web of Science][Medline]

Kennedy PR, Bakay RA, Moore MM, Adams K, and Goldwaithe J. Direct control of a computer from the human central nervous system. IEEE Trans Rehabil Eng 8: 198–202, 2000.[CrossRef][Web of Science][Medline]

Kostov A and Polak M. Parallel man-machine training in development of EEG-based cursor control. IEEE Trans Rehabil Eng 8: 203–205, 2000.[CrossRef][Web of Science][Medline]

Matelli M and Luppino G. Parietofrontal circuits for action and space perception in the macaque monkey. Neuroimage 14: S27–S32, 2001.[CrossRef][Web of Science][Medline]

Maynard EM, Hatsopoulos NG, Ojakangas CL, Acuna BD, Sanes JN, Normann RA, and Donoghue JP. Neuronal interactions improve cortical population coding of movement direction. J Neurosci 19: 8083–8093, 1999.[Abstract/Free Full Text]

Pesaran B, Pezaris JS, Sahani M, Mitra PP, and Andersen RA. Temporal structure in neuronal activity during working memory in macaque parietal cortex. Nat Neurosci 5: 805–811, 2002.[CrossRef][Web of Science][Medline]

Schwartz AB, Moran DW, and Reina GA. Differential representation of perception and action in the frontal cortex. Science 303: 380–383, 2004.[Abstract/Free Full Text]

Scott SH. Apparatus for measuring and perturbing shoulder and elbow joint positions and torques during reaching. J Neurosci Methods 89: 119–127, 1999.[CrossRef][Web of Science][Medline]

Serruya MD, Hatsopoulos NG, Paninski L, Fellows MR, and Donoghue JP. Instant neural control of a movement signal. Nature 416: 141–142, 2002.[CrossRef][Medline]

Shenoy KV, Meeker D, Cao S, Kureshi SA, Pesaran B, Buneo CA, Batista AP, Mitra PP, Burdick JW, and Andersen RA. Neural prosthetic control signals from plan activity. Neuroreport 14: 591–596, 2003.[CrossRef][Web of Science][Medline]

Taylor DM, Tillery SI, and Schwartz AB. Direct cortical control of 3D neuroprosthetic devices. Science 296: 1829–1832, 2002.[Abstract/Free Full Text]

Taylor DM, Tillery SI, and Schwartz AB. Information conveyed through brain-control: cursor versus robot. IEEE Trans Neural Syst Rehabil Eng 11: 195–199, 2003.[CrossRef][Web of Science][Medline]

Warland DK, Rienagel P, and Meister M. Decoding visual information from a population of retinal ganglion cells. J Neurophysiol 78: 2336–2350, 1997.[Abstract/Free Full Text]

Weinrich M and Wise SP. The premotor cortex of the monkey. J Neurosci 2: 1329–1345, 1982.[Abstract]

Wessberg J, Stambaugh CR, Kralik JD, Beck PD, Laubach M, Chapin JK, Kim J, Biggs SJ, Srinivasan MA, and Nicolelis MA. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408: 361–365, 2000.[CrossRef][Medline]

Wolpaw JR, McFarland DJ, Neat GW, and Forneris CA. An EEG-based brain–computer interface for cursor control. Electroencephalogr Clin Neurophysiol 78: 252–259, 1991.[CrossRef][Web of Science][Medline]




This article has been cited by other articles:


Home page
J. Neurophysiol.Home page
J. P. Cunningham, B. M. Yu, V. Gilja, S. I. Ryu, and K. V. Shenoy
Toward Optimal Target Placement for Neural Prosthetic Devices
J Neurophysiol, December 1, 2008; 100(6): 3445 - 3457.
[Abstract] [Full Text] [PDF]


Home page
J. Neurosci.Home page
C. A. Chestek, A. P. Batista, G. Santhanam, B. M. Yu, A. Afshar, J. P. Cunningham, V. Gilja, S. I. Ryu, M. M. Churchland, and K. V. Shenoy
Single-Neuron Stability during Repeated Reaching in Macaque Premotor Cortex
J. Neurosci., October 3, 2007; 27(40): 10742 - 10750.
[Abstract] [Full Text] [PDF]


Home page
J. Neurophysiol.Home page
L. Srinivasan, U. T. Eden, S. K. Mitter, and E. N. Brown
General-Purpose Filter Design for Neural Prosthetic Devices
J Neurophysiol, October 1, 2007; 98(4): 2456 - 2475.
[Abstract] [Full Text] [PDF]


Home page
J. Neurosci.Home page
E. Stark and M. Abeles
Predicting Movement from Multiunit Activity
J. Neurosci., August 1, 2007; 27(31): 8387 - 8394.
[Abstract] [Full Text] [PDF]


Home page
J. Neurophysiol.Home page
W. Wang, S. S. Chan, D. A. Heldman, and D. W. Moran
Motor Cortical Representation of Position and Velocity During Reaching
J Neurophysiol, June 1, 2007; 97(6): 4258 - 4270.
[Abstract] [Full Text] [PDF]


Home page
J. Neurophysiol.Home page
B. M. Yu, C. Kemere, G. Santhanam, A. Afshar, S. I. Ryu, T. H. Meng, M. Sahani, and K. V. Shenoy
Mixture of Trajectory Models for Neural Decoding of Goal-Directed Movements
J Neurophysiol, May 1, 2007; 97(5): 3763 - 3780.
[Abstract] [Full Text] [PDF]


Home page
J. Neurophysiol.Home page
Z. Chi, W. Wu, Z. Haga, N. G. Hatsopoulos, and D. Margoliash
Template-Based Spike Pattern Identification With Linear Convolution and Dynamic Time Warping
J Neurophysiol, February 1, 2007; 97(2): 1221 - 1235.
[Abstract] [Full Text] [PDF]


Home page
J. Neurophysiol.Home page
J. G. O'Leary and N. G. Hatsopoulos
Early Visuomotor Representations Revealed From Evoked Local Field Potentials in Motor and Premotor Cortical Areas
J Neurophysiol, September 1, 2006; 96(3): 1492 - 1506.
[Abstract] [Full Text] [PDF]


Home page
J. Neurophysiol.Home page
L. E. Sergio, C. Hamel-Paquet, and J. F. Kalaska
Motor Cortex Neural Correlates of Output Kinematics and Kinetics During Isometric-Force and Arm-Reaching Tasks
J Neurophysiol, October 1, 2005; 94(4): 2353 - 2378.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (23)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Hatsopoulos, N.
Right arrow Articles by O'Leary, J. G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Hatsopoulos, N.
Right arrow Articles by O'Leary, J. G.


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Visit Other APS Journals Online
Copyright © 2004 by the The American Physiological Society.