|
|
||||||||
1Department of Organismal Biology and Anatomy, University of Chicago, 60637; and 2Pritzker Institute of Biomedical Sciences and Engineering, Illinois Institute of Technology, Chicago, Illinois 60616
Submitted 22 December 2003; accepted in final form 4 March 2004
| ABSTRACT |
|---|
|
|
|---|
| INTRODUCTION |
|---|
|
|
|---|
| METHODS |
|---|
|
|
|---|
Two macaque monkeys (Macaca mulatta) were operantly trained to perform an instructed-delay 8-direction center-out (CO) task and a random-sequence (RS) task by moving a cursor to targets by contralateral arm movements. Both tasks involved projecting the cursor and targets onto a horizontal, reflective surface in front of the monkey above the monkey's hand. The monkey's arm rested on cushioned arm troughs secured to links of a 2-joint robotic arm (KINARM system; Scott 1999
) underneath the projection surface. The shoulder joint was abducted 90° such that shoulder and elbow flexion and extension movements were made in the horizontal plane.
The CO task involved movements from a center target to one of 8 peripherally positioned targets (57 cm distance). The task consisted of 3 epochs: 1) a 500-ms hold period during which the monkey was required to hold its hand over the center target, 2) a fixed instruction period (600 or 1,000 ms depending on the recording session) during which one of the 8 final targets appeared but the monkey was not allowed to move, and 3) a "GO" period during which the target began blinking. This informed the monkey to begin moving to the peripherally positioned target.
In the RS task, a sequence of 7 targets appeared on the projection surface. At any one time, a single target appeared at a random location in the workspace, and the monkey was required to move to it (Fig. 1A). As soon as the cursor reached the target, the target disappeared and a new target appeared in a new, random location. After reaching the 7th target, the monkey was rewarded with a drop of water. During this task the workspace was sampled thoroughly, which was important in developing a hand position decoder that could generalize adequately (Fig. 1B).
|
Two silicon-based electrode arrays composed of 100 electrodes (1.0 mm electrode length; 400 µm interelectrode separation) were implanted in the arm areas of MI and PMd of each monkey (see Maynard et al. 1999
for more details concerning the electrode array) (Fig. 2A). During a recording session, signals from
128 electrodes were amplified (gain, 150) and recorded digitally (14-bit) at 30 kHz per channel using a Cerebus acquisition system (Cyberkinetics, Salt Lake City, UT). Only waveforms (1.6 ms in duration) that crossed a threshold were stored and spike-sorted using Off-line Sorter (Plexon, Dallas, TX). Interspike interval histograms were computed to verify single-unit isolation by ensuring that <0.05% of waveforms possessed an interspike interval <1.6 ms. Figure 2B shows average (±1 SD) waveforms from one representative data set. Figure 2C shows the distribution of signal-to-noise ratios for this data set. Signal-to-noise ratios were defined as the difference in mean peak-to-trough voltage divided by twice the SD. All isolated single units used in this study possessed signal-to-noise ratios (rounded to the nearest whole number) of 3:1 or higher. A total of 10 data sets (7 using the RS and 3 using the CO tasks) for animal B and 4 data sets (2 using the RS and 2 using the CO tasks) for animal R were analyzed, where a data set is defined as all simultaneously recoded neural data collected in one recording session. Each data set contained between 32 and 143 simultaneously recorded units from MI and PMd (Table 1). The data sets used to decode continuous hand position contained equal numbers of units from MI and PMd by selecting the units with the largest signal-to-noise ratios that made both ensembles of equal size. Thus both ensembles consisted of "randomly" selected units from MI and PMd except for a possible bias for neurons with large cell bodies that would generate higher signal-to-noise ratios. All of the surgical and behavioral procedures were approved by the University of Chicago's IACUC and conform to the principles outlined in the Guide for the Care and Use of Laboratory Animals (National Institutes of Health publication no. 8623, revised 1985).
|
|
The shoulder and elbow joint angles were sampled at 500 Hz by the robotic arm's motor encoders and acquired along with the neural data. The x and y positions of the hand were computed by transforming the joint angles into Cartesian endpoint positions using the forward kinematics equations for a 2-joint arm with the measured lengths of the upper arm and forearm of the monkey.
Decoding algorithms
Continuous reconstruction.
Reconstructions of Cartesian x and y endpoint positions were computed from the neural data using linear and nonlinear regression methods. The endpoint position at time t (pt) was predicted from the neural responses (rtin) of multiple neurons (n) at multiple time points (ti) in the past. In the linear case, the optimal set of linear filter coefficients (fin) was found that minimized the sum squared difference between the predicted position (
t) and the actual position using a closed-form solution (Warland et al. 1997
)
![]() | (1) |
In the nonlinear case, a 2-layer neural network was implemented using the Matlab Neural Network Toolbox. The input layer consisted of N x L input nodes representing the responses of multiple neurons at multiple time points. Sigmoidal activation functions were used between the input and hidden layer. Linear activation functions were used between the hidden and output layer, which consisted of one node representing the predicted position at time t,
t. The number of hidden nodes, the learning algorithm, and the learning rate were selected using cross-validation techniques on one data set and then were used in the networks built for 4 other data sets, 2 from each animal. Ten hidden units, a learning rate of 0.01, and the scaled conjugate gradient (gradient descent) learning algorithm were selected as network parameters. Each data set was randomly partitioned into a training set (80% of the data), a validation set (10% of the data), and a test set (the remaining 10%), and the weights of each network were then trained so that the sum squared difference between the actual and predicted positions for the training set was minimized (although a global minimum was not necessarily reached). To prevent overfitting, the sum squared error for the validation set was computed after each training epoch, and training was stopped when the value began to increase. The test set was then used to estimate the generalization performance of the trained network. Because both the initialization of network weights and the division into training, test, and validation sets are random, the training and testing procedure was performed 10 times for each network.
Discrete classification. Parametric and nonparametric classifiers were developed to assign individual trial responses to one of the 8 target/direction classes in the CO task. In the parametric case, a probability distribution model conditioned on each target/direction class was estimated for the response of a single neuron. Both Poisson and Gaussian distributions were used but the Poisson model generally performed better than the Gaussian, and so results from the Gaussian model are not presented. For the Poisson model, a mean (µ) parameter was estimated based on the sample mean response. Assuming conditional independence among the neurons, a joint probability distribution for the combined responses over all neurons was computed by multiplying the single-neuron distributions for each direction. Conditional independence is an approximation. However, spike count correlation coefficients were generally small between pairs of neurons during a 400-ms window in the instruction period. Average pairwise correlation coefficients (averaged over all pairs and directions in a data set) ranged from 0.002 to 0.02 for all but one data set, which had slightly higher but still small values ranging from 0.05 to 0.11. A maximum-likelihood classification rule was used such that a trial response was assigned to the target/direction class that maximized the likelihood of observing the response. Accuracy of the classifiers was estimated using leave-one-out cross-validation, wherein the class of each trial was predicted from a classifier built using all other trials, and the percentage of trials that were classified correctly is then computed to estimate the model performance.
A nonparametric 2-layer neural network similar to that described above (with sigmoidal activation functions at the output layer instead of a linear activation function) was used to classify individual trial responses into one of 8 target/direction categories. The output layer of the network possessed 8 nodes, one for each category. The training data used for the output layer consisted of an 8-dimensional vector with a value of 1 for the correct category and 0 for the other categories. During testing, the trial response was assigned to the category whose node was maximally activated. Results using the neural network on one data set demonstrated a similar improved performance using PMd versus MI and thus are not reported.
| RESULTS |
|---|
|
|
|---|
We systematically varied the bin size B and the filter length LT, used to build the linear filter decoder to find the optimal set of parameters (Fig. 3). Although there were slight variations with bin size as well as variations depending on the decoded parameter and the cortical area used, we found that performance depended primarily on filter length. A filter length of 1 s was near optimal in terms of accounting for the largest percentage of the hand position variance and was used for all reported analyses.
|
|
|
To examine the effect of ensemble size on decoding accuracy, we used a neuron-dropping procedure in which subsets of neurons were randomly sampled and used to build the decoder and cross-validated on 10% of the trials. Figure 6A (left and middle panels) show scatterplots of reconstruction performance values for one data set (measured in terms of r2) using N unique subsets for each ensemble size, where N is the total number of recorded neurons. We termed these subsets "unique subsets" because they were chosen so that no two subsets of size N were identical. That is, if AiK is the ith subset of K neurons, then at least one neuron that is a member of AiK is not a member of AjK, whenever i
j. Hyperbolic functions [f(x) = ax/(1 + ax)] (Wessberg et al. 2000
) were fit to the mean reconstruction performance (Fig. 6A, third panels) values to extrapolate to near-perfect performance (Fig. 6A, fourth panels). Based on all 9 data sets, we predict that an ensemble size of 256 ± 38 (mean ± SE) in MI could account for 90% of the x hand position variance as compared with 438 ± 32 neurons in PMd. For the y hand position, the difference was even larger, with 222 ± 53 neurons in MI versus 710 ± 168 neurons in PMd accounting for 90% of the variance.
|
Hyperbolic functions have been shown previously to fit the monotonically increasing performance with ensemble size (Wessberg et al. 2000
). However, there were clear deviations from a hyperbolic fit for the one data set that contained many more units in both cortical areas (r031206). Figure 6B shows the decoding performance as a function of ensemble size for this data set. The best-fit hyperbolic functions (third panels in Fig. 6B) overestimated the actual performance of the both cortical areas when the ensemble size was large. The fourth panels show the best-fit logarithmic function to the mean performance. Using this function for this data set to extrapolate in MI suggests that 402 neurons (averaged over x and y) would be required to account for 90% of the hand position variance as opposed to 204 neurons based on the hyperbolic fit. In PMd, 970 neurons would be required as opposed to 331 neurons using the best-fit hyperbolic function.
Discrete decoding
As previous studies have shown, neurons in PMd exhibit robust activity related to movement planning. By aligning perievent histograms on the onset of the instruction signal in the instructed-delay CO task, we often observed transient "signal-related" activity lasting about 300400 ms whose magnitude varied with target direction (Fig. 7A, top) (Weinrich and Wise 1982
). This early instruction-related activity was generally either weakly present or entirely absent in MI (Fig. 7A, bottom). To quantify this difference in "signal-related" activity, Fig. 7B compares the firing rate modulation (i.e., percentage change relative to the baseline firing rate in the first 100 ms after the instruction signal onset) averaged over the ensemble of recorded PMd neurons (solid line) and simultaneously recorded MI neurons (dashed line) for one data set.
|
|
|
|
| DISCUSSION |
|---|
|
|
|---|
Our findings, however, are inconsistent with previous data from Nicolelis and colleagues (Wessberg et al. 2000
) who indicated that PMd ensembles could reconstruct hand position more accurately than MI. The reason for this discrepancy is unclear. There are at least 2 possible differences in experimental protocol that may explain the inconsistency. First, they used a different species of monkey (owl monkey vs. rhesus macaque that we used) in their study. Second, the task they used to quantify the functional differences between MI and PMd was one-dimensional, in which the monkey moved a lever either to the left or right. Recently, however, the Nicolelis group published a study using rhesus macaques that performed a 2-D task very similar to our random-sequence task (Carmena et al. 2003
). In that study, they observed that hand position reconstructions were more accurate using neuronal ensembles in MI versus PMd. Their results were similar to ours, indicating approximately 70% improvement using an MI ensemble as compared with an equally sized ensemble in PMd.
At first glance, the shapes of the optimal decoding filters generated in the random-sequence task for PMd neurons appear to be counterintuitive. Although there was considerable variation in linear filter shape across individual neurons in both MI and PMd, the absolute filter coefficient magnitude tended to decrease from the immediate past to the distant past for both areas (see insets in Fig. 5, A and B). It might have been predicted that the filter magnitudes associated with PMd neurons would be larger for time points in the distant past. The observed filter shapes suggest that PMd activity in the intermediate and distant past does not correlate well with the current kinematics of the hand that are being decoded in the random-sequence task but rather may carry information about the target to be reached as in the center-out task. This is consistent with previous data showing that PMd neuronal activity is generally not related to the on-line control of arm movements as MI is (Crammond and Kalaska 1996
). We are currently investigating whether PMd activity may be used to predict the position of the current target in the RS task.
The effect of ensemble size on decoding performance demonstrated that less than half the number of neurons in MI would be required to attain ideal performance (i.e., accounting for 90% of the hand position variance) as compared with PMd in the continuous case, whereas a 6th the number of PMd neurons would be required to correctly classify 90% of trials during the preparatory period in the discrete case as compared with MI. These results are based on extrapolation using functions that fit the data. Extrapolation, however, should be viewed with extreme caution because the data may deviate from the best-fit functions (using smaller groups of neurons) as the ensemble sizes grow larger. This was made clear in Fig. 6B in which the hyperbolic function overestimated the continuous decoding performance for the one data set that contained many more neurons. Extrapolation can also amplify small differences, as is evident in the large variability in the predicted number of MI neurons required to attain 90% correct classification in the discrete CO task. In the future, we plan to examine these issues by recording from larger groups of neurons within each cortical area.
The early planning activity to visual targets observed in PMd is similar to findings by Andersen and colleagues who measured early movement "intentional" signals in an area called the parietal reach region (PRR) partially overlapping with the medial intraparietal area (MIP) of the posterior parietal cortex (Batista et al. 1999
). This is, perhaps, not surprising given the direct anatomical connections between MIP and dorsal premotor cortex (Matelli and Luppino 2001
). A similar probabilistic classifier was shown to accurately predict upcoming movement direction using spiking activity from serially recorded neurons within PRR (Shenoy et al. 2003
). There may be differences, however, in the coordinate systems used in each area. Arm movements appear to be encoded in eye-centered coordinates in PRR (Batista et al. 1999
). Although eye position appears to influence activity in PMd when monkeys are required to fixate (Boussaoud et al. 1998
), the effects are quite small when monkeys are free to change their gaze (Cisek and Kalaska 2002
).
Our results have implications for the development of a brainmachine interface. There are discrete control tasks such as selecting from a number of distinct buttons or attaining a particular postural arm or hand configuration that can take advantage of the early movement-selection properties of PMd ensembles. On the other hand, a BMI may also be faced with controlling continuously varying motor variables such as guiding a cursor on a screen, moving a robotic arm in space, or navigating an autonomous robot on the ground, which could be handled using single units from MI. A general-purpose BMI could benefit from the complementary nature of motor and premotor ensemble activity.
| GRANTS |
|---|
|
|
|---|
| DISCLOSURE |
|---|
|
|
|---|
| ACKNOWLEDGMENTS |
|---|
|
|
|---|
| FOOTNOTES |
|---|
Address for reprint requests and other correspondence: N. G. Hatsopoulos, Dept. of Organismal Biology and Anatomy, University of Chicago, Chicago, IL 60637 (E-mail: nicho{at}uchicago.edu).
| REFERENCES |
|---|
|
|
|---|
Boussaoud D, Jouffrais C, and Bremmer F. Eye position effects on the neuronal activity of dorsal premotor cortex in the macaque monkey. J Neurophysiol 80: 11321150, 1998.
Buneo CA, Jarvis MR, Batista AP, and Andersen RA. Properties of spike train spectra in two parietal reach areas. Exp Brain Res 153: 134139, 2003.[CrossRef][Web of Science][Medline]
Carmena JM, Lebedev MA, Crist RE, O'Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez CS, and Nicolelis MA. Learning to control a brainmachine interface for reaching and grasping by primates. Publ Library Sci Biol 1: 116, 2003.
Cisek P and Kalaska JF. Modest gaze-related discharge modulation in monkey dorsal premotor cortex during a reaching task performed with free fixation. J Neurophysiol 88: 10641072, 2002.
Crammond DJ and Kalaska JF. Differential relation of discharge in primary motor cortex and premotor cortex to movements versus actively maintained postures during a reaching task. Exp Brain Res 108: 4561, 1996.[Web of Science][Medline]
Kennedy PR, Bakay RA, Moore MM, Adams K, and Goldwaithe J. Direct control of a computer from the human central nervous system. IEEE Trans Rehabil Eng 8: 198202, 2000.[CrossRef][Web of Science][Medline]
Kostov A and Polak M. Parallel man-machine training in development of EEG-based cursor control. IEEE Trans Rehabil Eng 8: 203205, 2000.[CrossRef][Web of Science][Medline]
Matelli M and Luppino G. Parietofrontal circuits for action and space perception in the macaque monkey. Neuroimage 14: S27S32, 2001.[CrossRef][Web of Science][Medline]
Maynard EM, Hatsopoulos NG, Ojakangas CL, Acuna BD, Sanes JN, Normann RA, and Donoghue JP. Neuronal interactions improve cortical population coding of movement direction. J Neurosci 19: 80838093, 1999.
Pesaran B, Pezaris JS, Sahani M, Mitra PP, and Andersen RA. Temporal structure in neuronal activity during working memory in macaque parietal cortex. Nat Neurosci 5: 805811, 2002.[CrossRef][Web of Science][Medline]
Schwartz AB, Moran DW, and Reina GA. Differential representation of perception and action in the frontal cortex. Science 303: 380383, 2004.
Scott SH. Apparatus for measuring and perturbing shoulder and elbow joint positions and torques during reaching. J Neurosci Methods 89: 119127, 1999.[CrossRef][Web of Science][Medline]
Serruya MD, Hatsopoulos NG, Paninski L, Fellows MR, and Donoghue JP. Instant neural control of a movement signal. Nature 416: 141142, 2002.[CrossRef][Medline]
Shenoy KV, Meeker D, Cao S, Kureshi SA, Pesaran B, Buneo CA, Batista AP, Mitra PP, Burdick JW, and Andersen RA. Neural prosthetic control signals from plan activity. Neuroreport 14: 591596, 2003.[CrossRef][Web of Science][Medline]
Taylor DM, Tillery SI, and Schwartz AB. Direct cortical control of 3D neuroprosthetic devices. Science 296: 18291832, 2002.
Taylor DM, Tillery SI, and Schwartz AB. Information conveyed through brain-control: cursor versus robot. IEEE Trans Neural Syst Rehabil Eng 11: 195199, 2003.[CrossRef][Web of Science][Medline]
Warland DK, Rienagel P, and Meister M. Decoding visual information from a population of retinal ganglion cells. J Neurophysiol 78: 23362350, 1997.
Weinrich M and Wise SP. The premotor cortex of the monkey. J Neurosci 2: 13291345, 1982.[Abstract]
Wessberg J, Stambaugh CR, Kralik JD, Beck PD, Laubach M, Chapin JK, Kim J, Biggs SJ, Srinivasan MA, and Nicolelis MA. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408: 361365, 2000.[CrossRef][Medline]
Wolpaw JR, McFarland DJ, Neat GW, and Forneris CA. An EEG-based braincomputer interface for cursor control. Electroencephalogr Clin Neurophysiol 78: 252259, 1991.[CrossRef][Web of Science][Medline]
This article has been cited by other articles:
![]() |
J. P. Cunningham, B. M. Yu, V. Gilja, S. I. Ryu, and K. V. Shenoy Toward Optimal Target Placement for Neural Prosthetic Devices J Neurophysiol, December 1, 2008; 100(6): 3445 - 3457. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. A. Chestek, A. P. Batista, G. Santhanam, B. M. Yu, A. Afshar, J. P. Cunningham, V. Gilja, S. I. Ryu, M. M. Churchland, and K. V. Shenoy Single-Neuron Stability during Repeated Reaching in Macaque Premotor Cortex J. Neurosci., October 3, 2007; 27(40): 10742 - 10750. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Srinivasan, U. T. Eden, S. K. Mitter, and E. N. Brown General-Purpose Filter Design for Neural Prosthetic Devices J Neurophysiol, October 1, 2007; 98(4): 2456 - 2475. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. Stark and M. Abeles Predicting Movement from Multiunit Activity J. Neurosci., August 1, 2007; 27(31): 8387 - 8394. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. Wang, S. S. Chan, D. A. Heldman, and D. W. Moran Motor Cortical Representation of Position and Velocity During Reaching J Neurophysiol, June 1, 2007; 97(6): 4258 - 4270. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. M. Yu, C. Kemere, G. Santhanam, A. Afshar, S. I. Ryu, T. H. Meng, M. Sahani, and K. V. Shenoy Mixture of Trajectory Models for Neural Decoding of Goal-Directed Movements J Neurophysiol, May 1, 2007; 97(5): 3763 - 3780. [Abstract] [Full Text] [PDF] |
||||
![]() |
Z. Chi, W. Wu, Z. Haga, N. G. Hatsopoulos, and D. Margoliash Template-Based Spike Pattern Identification With Linear Convolution and Dynamic Time Warping J Neurophysiol, February 1, 2007; 97(2): 1221 - 1235. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. G. O'Leary and N. G. Hatsopoulos Early Visuomotor Representations Revealed From Evoked Local Field Potentials in Motor and Premotor Cortical Areas J Neurophysiol, September 1, 2006; 96(3): 1492 - 1506. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. E. Sergio, C. Hamel-Paquet, and J. F. Kalaska Motor Cortex Neural Correlates of Output Kinematics and Kinetics During Isometric-Force and Arm-Reaching Tasks J Neurophysiol, October 1, 2005; 94(4): 2353 - 2378. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |