| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Department of Neuroscience, Brown University, Providence, Rhode Island 02912; and 2 Division of Neuroscience, Baylor College of Medicine, Houston, Texas 77030
 |
ABSTRACT |
|---|
 Abstract
 Introduction
Methods
Results
Discussion
References
|
|---|
Gibson, Jay R. and John H. R. Maunsell. Sensory modality specificity of neural activity related to memory in visual cortex. J. Neurophysiol. 78: 1263-1275, 1997. Previous studies have shown that when monkeys perform a delayed match-to-sample (DMS) task, some neurons in inferotemporal visual cortex are activated selectively during the delay period when the animal must remember particular visual stimuli. This selective delay activity may be involved in short-term memory. It does not depend on visual stimulation: both auditory and tactile stimuli can trigger selective delay activity in inferotemporal cortex when animals expect to respond to visual stimuli in a DMS task. We have examined the overall modality specificity of delay period activity using a variety of auditory/visual cross-modal and unimodal DMS tasks. The cross-modal DMS tasks involved making specific long-term memory associations between visual and auditory stimuli, whereas the unimodal DMS tasks were standard identity matching tasks. Delay activity existed in auditory/visual cross-modal DMS tasks whether the animal anticipated responding to visual or auditory stimuli. No evidence of selective delay period activation was seen in a purely auditory DMS task. Delay-selective cells were relatively common in one animal where they constituted up to 53% neurons tested with a given task. This was only the case for up to 9% of cells in a second animal. In the first animal, a specific long-term memory representation for learned cross-modal associations was observed in delay activity, indicating that this type of representation need not be purely visual. Furthermore, in this same animal, delay activity in one cross-modal task, an auditory-to-visual task, predicted correct and incorrect responses. These results suggest that neurons in inferotemporal cortex contribute to abstract memory representations that can be activated by input from other sensory modalities, but these representations are specific to visual behaviors.
Inferotemporal cortex (IT) is part of the ventral pathway in primate visual cortex (Ungerleider and Mishkin 1982 Behavioral tasks
Single units were recorded in two macaque monkeys while they performed delayed match-to-sample tasks. Animal 1 was a 7.5-kg male Macaca mulatta and animal 2 was a 7.2-kg male M. nemestrina. Water intake was controlled 5 days a week during training and recording. The animals worked for apple juice rewards that were delivered at the end of every correctly completed trial.
Surgical procedures
Surgeries were performed under full anesthesia in aseptic conditions. Heart rate, respiration rate, and temperature were monitored throughout the surgery. Antibiotics (Bactrim) were given the day before and for 7 days after surgery. Animals were initially prepared with ketamine (15 mg/kg), atropine (50 g/kg), and diazepam (50 g/kg). After these drugs took effect, the animal was intubated and a venous catheter was inserted, and then the animal was mounted in a stereotaxic head holder and put under isoflurane anesthesia. Analgesic was administered after surgery (Banamine, im).
Single unit recording
During training and recording each monkey sat in a primate chair facing a color video monitor (75 Hz, 40 × 31 cm) ×65 cm away. A small loud speaker was placed below the animals line of sight at a distance of 40 cm. The animals head was stabilized for eye position monitoring and neuronal recording by anchoring the headpost. Eye position was monitored using a scleral search coil system (Judge et al. 1980 Data analysis
Except where noted, data analysis was based on correctly completed trials. Behavioral responses made within 200 ms of test onset were counted as guesses and excluded from analysis. We restricted our analysis of delay-period activity to the 800 ms immediately before test stimulus onset. This period was chosen because it was unlikely to be contaminated by responses to the offset of the sample stimulus, and it was a period when the animal had to remember the sample stimulus. Delay activity was typically determined using 16 repetitions of each condition (minimum 7).
Histology
When recordings were completed, the animal was euthanatized with an overdose of barbiturates (Nembutal, iv), and perfused with a phosphate buffer rinse followed by paraformaldehyde fixative. The brain then was removed, blocked coronally, and equilibrated with 30% sucrose in phosphate buffer. Blocks were sectioned at 40 m on a freezing microtome. Sections were then mounted on slides, stained in cresyl violet, and coverslipped. Recording locations were identified using fiducial pins inserted into the grid shortly before perfusion.
Quantitative data were collected from 230 cells in IT. Of these, 174 cells were recorded while the monkeys performed all four tasks. The remaining 56 units were recorded while they performed only the two cross-modal tasks. Many more isolated units were encountered, but only those whose activity was obviously modulated by some aspect of the task were tested. The percentage of correctly completed trials in each task for animal 1 and animal 2 during data collection were: visual-to-visual, 95 and 99%; visual-to-auditory, 94 and 99%; auditory-to-visual, 89 and 99%; auditory-to-auditory, 80 and 89%.
Sensory responses
As expected, most sensory responses in IT were visual. Figure 2 shows for each task the percent of cells with significant responses to the sample stimulus. In the tasks with visual sample stimuli, ~70% of the neurons responded(125/174 for visual-to-visual and 121/174 for visual-toauditory) compared with 30 and 19% in each task with auditory sample stimuli (52/174 for auditory-to visual and 33/174 for auditory-to-auditory). Sensitivity to one stimulus modality was uncorrelated with sensitivity to the other.
Selective delay activity
Many neurons had selective delay activity that depended on the identity of the sample stimulus. These neurons were significantly more active during those delay periods after particular sample stimuli, and this selective activity generally persisted throughout the delay period. The data in Fig. 3 are from a neuron that showed selective delay-period activity during the two cross-modal tasks. In the visual-to-auditory task, this cell was more active after sample stimulus 1, whereas in the auditory-to-visual task, the cell was more active after sample stimulus 2. This selective activation was not sensory because sensory stimulation (i.e., fixation point, the background, and the visual mask) was identical during the delay period in all trials. Selective delay activity has been documented in IT previously with both simple visual matching and visual association tasks, and it has been suggested to play a role in short-term memory (Fuster and Jervey 1982
Individual differences in delay activity
There was a substantial difference in the incidence of delay selective neurons between the two animals. Animal 1 had many cells with strong delay selectivity. In animal 2, a few neurons had clear selective delay-period activity (Fig. 5), but these were rare (Table 1). The low incidence of delay selectivity in animal 2 was not due to poor performance because it performed
Long-term memory representation
Of the neurons with delay selectivity in at least one of the cross-modal tasks, almost half showed selectivity during both (26/60). For animal 1 there was a clear relationship in delay selectivity between the cross-modal tasks. When visual sample 1 in the visual-to-auditory task elicited more delay-period activity, auditory sample 2 generally produced greater delay-period activity in the auditory-to-visual matching task (e.g., Fig. 3). The converse pairing also was observed (Fig. 7). The overall relationship is illustrated in Fig. 8A, which plots visual-to-auditory delay selectivity against delay selectivity for the auditory-to-visual task for animal 1. There was a strong anticorrelation between the values(R =
Relationship between sensory responses and delay activity
Consistent with earlier studies, sample response selectivity and delay period selectivity were fairly independent of each other (Ferrera et al. 1994
Delay activity before incorrect responses
Both animals occasionally made errors. In a separate analysis, we compared delay-period activity during incorrectly completed trials with that from correct trials. Analysis was restricted to cells that showed significant differences in delay activity in at least one of the cross-modal tasks. If the animal did not make at least two errors in each condition of a particular task while a cell was monitored, it was not used in the analysis of that task. Because animal 2 made so few errors and because there were only a few delay-selective cells found in this animal, only cells from animal 1 qualified for this analysis. The analysis could not be performed on either unimodal task either because of too few errors or lack of delay-period selectivity.
Effects of the mask stimulus on delay activity
A visual mask was presented after every sample stimulus. Although this mask was irrelevant to the matching tasks, it nevertheless evoked strong responses from some neurons in IT. Figure 12 shows the range of responses. Some cells were excited or inhibited by the mask (Fig. 12, A and C), whereas others showed no response (Fig. 12B). Using data from animal 1, where ample time before mask onset permitted analysis, delay-period selectivity was generally evident before and after the mask appeared (R = 0.51, P < 0.001, n = 176, only instances of statistically significant delay activity included).
Sensory response modulation related to short-term memory
Many studies have reported modulation of sensory responses dependent upon recently presented stimuli and thus related to short-term memory (Fuster and Jervey 1982
As in previous studies of IT, we observed selective neural activity during the delay period of delayed match-to-sample tasks that was correlated with short-term memory for specific stimuli (Fuster and Jervey 1982
INTRODUCTION
Abstract
Introduction
Methods
Results
Discussion
References
) and plays an important role in pattern recognition (Cowey and Gross 1970; Dean 1976; Desimone et al. 1984; Gross et al. 1972; Iwai and Mishkin 1969). IT contains neurons that respond selectively to complex stimuli, such as faces and hands (Kobatake and Tanaka 1994), suggesting that behaviorally relevant visual stimuli are represented explicitly in this cortical area.
; Motter 1993; Treue and Maunsell 1996), motor signals (Andersen and Mountcastle 1983; Wilson et al. 1990), and short-term memory (Fuster and Jervey 1982; Miller et al. 1991). These extraretinal signals demonstrate that sensory processing in visual cortex depends on behavior. Characterizing these signals will be important for understanding how sensory information is used to guide behavior. We focus here on activity associated with short-term memory in IT and its relation to long-term memory.
; Miyashita and Chang 1988). This firing, or delay activity, is selective in that activity is high while the animal is remembering one stimulus, but low while remembering another, and hence, it is correlated with short-term memory for specific stimuli. Selective maintained firing has been observed in many brain structures in the context of both stimulus memory and motor planning (Gnadt and Andersen 1988; Hikosaka et al. 1989; Koch and Fuster 1989; Kurata and Wise 1988; Mays and Sparks 1980; Niki 1974). It has not, however, been observed in all studies of short-term memory in visual cortex and thus may not be relevant to all forms of visual short-term memory (Baylis and Rolls 1987; Eskandar et al. 1992; Miller et al. 1993).
; Maunsell et al. 1991). Colombo and Gross (1994) reported that auditory stimuli also can elicit delay activity in IT. In these experiments, the animal always expected, and responded to, a visual test stimulus. Activity in visual cortex has not been examined in a task in which an animal anticipates a nonvisual stimulus. Nor has a thorough analysis been performed on the sensory modality specificity of visual cortical delay activity.
; Sakai and Miyashita 1991). Miyashita and Sakai trained monkeys to recognize randomly assigned pairings of visual stimuli. After the associations were learned, some IT neurons were activated selectively during the delay periods after the presentation of either member of a particular pair. This pattern of activation presumably depended on the animal being trained to associate the particular stimuli and can be considered a correlate of long-term memory. We wanted to determine if such a representation could also exist for cross-modal associations.
).
METHODS
Abstract
Introduction
Methods
Results
Discussion
References
2 s.
View larger version (31K):
[in a new window]
FIG. 1.
Delayed match-to-sample tasks. Top: a schematic shown of a trial in delayed match-to-sample task. 
, when stimuli were present. Fixation point was present throughout the trial. Bottom: stimulus arrangement shown for each task. Sample stimuli are left in each box, and test stimuli are right in each. Matches or associates are aligned horizontally. Although this stimulus arrangement was used by animal 1, animal 2 used an arrangement in which the cross-modal and unimodal task stimuli were swapped. For most cells, delay period was the same in all tasks at all times. For some cells, delay period length varied, but each task still had the same proportion of various delays, and thus, the average delay period was the same in all tasks. Auditory sample stimuli were presented for 500 ms and visual samples for 400 ms. Test stimuli were on for a maximum of 800 ms.
), ~10° across, white on a black background. The auditory stimuli were as follows: low tone, high tone, a repeated tap that sounded like a snare drum, and another repeated tap sound higher in frequency than the snare, which sounded like clicks. These sounds were selected because they proved distinct enough for the animals to distinguish readily and because their repetitive nature allowed them to be presented for arbitrary lengths of time (See D'Amato and Salmon 1984).
; Coltheart 1983; Sperling 1960). The mask permitted us to examine the effect of intervening visual stimuli and accompanying visual sensory processing on neural signals during the delay period (Miller et al. 1993). Auditory masks were not used because auditory sensory processing probably does not occur in IT (Desimone and Gross 1979). The masks were arrays of nine Fourier descriptors centered around the fixation point and with each element the same size as the visual stimuli used in the task.
; Robinson 1963). A computer was used for all data collection, stimulus presentation, and behavioral control.
, ~30-µm tips, Microprobe, Type B). Signals from the electrode were amplified, filtered, and transformed into pulses by a window discriminator. Spike times were recorded with 1-ms resolution, and behavioral events were recorded with5-ms resolution. All time bases were synchronized with the vertical retrace of the video monitor at the beginning of each trial.
) was used to direct the electrode to recording sites. The tip of the guide tube usually was positioned at the ventral bank of the lateral sulcus or in the white matter just dorsal to the superior temporal sulcus.
). The selectivity of responses was determined by comparing both sample responses in a task to see if these values differed significantly.
RESULTS
Abstract
Introduction
Methods
Results
Discussion
References
View larger version (42K):
[in a new window]
FIG. 2.
Sample sensory responses in each matching task. Height of each bar represents percent of responsive cells, whereas the darker shades indicate the subset that was also selective (P < 0.05). Percent of responsive and selective cells for each task are as follows: visual-to-visual, 72 and 30; visual-to-auditory, 70 and 25; auditory-to-visual, 30 and 6; auditory-to-auditory, 19 and 2; Percentages are consistent between tasks in which the same modality was the sample stimulus.
; Gross et al. 1972). It should be noted, however, that activity during the presentation of the sample stimulus may not have been entirely sensory. For example, IT neurons can be activated in a nonspecific way by auditory stimuli (Iwai et al. 1987; Ringo and O'Neill 1993). Consistent with this, the number of neurons reaching statistical criterion for auditory sample selectivity was no more than the number expected by chance, whereas about a quarter of cells discriminated between the visual sample stimuli. Furthermore, any traces of selectivity to auditory sample stimuli may be a result of selective delay activity beginning before the stimuli disappear (see below).
; Miyashita and Chang 1988). Although the neuron was also active during the delay period of the visual-to-visual and auditory-to-auditory tasks, we focus here on activity that was selective, or differential, for the sample stimuli within tasks.
View larger version (23K):
[in a new window]
FIG. 3.
Data from 1 cell displaying selective delay activity. Sample stimuli used for each trial type are presented (left of each histogram). Histograms include both matching and nonmatching trials. Test stimulus duration is truncated to approximately the earliest possible behavioral response during a matching trial. This was done because behavioral control is lost at this point in a trial. Underneath each histogram, indicates presence of the sample (S), the mask (M), and the test (T), and 
indicates time window during which delay activity was analyzed. x-axis ticks are 200 ms, and y-axis ticks are 10 spikes/s. This description of histograms applies to all the following histograms unless otherwise stated. In this cell, delay activity is selective only in cross-modal tasks. All histograms in this figure were derived from 20 to 21 individual trials.
one for each task. The distributions of these differences are plotted in Fig. 4. During the visual-to-auditory task, 26% (46/174) of cells had statistically significant differences in delay activity between the two sample conditions. A similar proportion of significant differences was seen for the auditory-to-visual task (23%, 40/174). Fourteen percent (25/174) of cells showed statistically significant delay selectivity during the visual matching task, a value that falls within the range reported previously in studies using a similar number of stimuli (10%, Fuster and Jervey 1982; 18%, Colombo and Gross 1994). The number of neurons reaching statistical criterion for selective delay period activity during the auditory matching task (5%, 8/174) was no greater than the number expected by chance.
View larger version (34K):
[in a new window]
FIG. 4.
Frequency bargraphs of delay differences. These graphs depict magnitude of delay selectivity observed in both animals for each task by plotting the absolute value of the delay difference. , cells in which delay selectivity was statistically significant. Same number of cells are tabulated in each graph.
), animal 1 had greater selectivity in the two cross-modal tasks than in either of the unimodal tasks. These data suggest that delay activity is more selective in cross-modal tasks than in purely visual matching tasks.
B|/min(A,B)]. Thus tasks involving auditory stimuli produced clear selective delay activity in visual cortex, but only in tasks that required a visual discrimination.
4% better in every task compared with animal 1. An example of a delay-selective cell from animal 2 is provided in Fig. 5. The difference between animals is also unlikely to stem from sampling different regions of IT. Data were collected first from animal 1 in a region in IT between 9 and 20 mm anterior in the right hemisphere (Fig. 6A). Almost all delay-selective cells in animal 1 were in a region of the ventral bank of the STS located 9-12 mm anterior. In this region, we estimate that ~5-10% of the cells encountered displayed delay selectivity, whereas outside of this region, delay-selective cells were not nearly as often encountered. Recordings from IT in animal 2 were made initially in the left hemisphere in the range from 11 to 22 mm anterior. When few units with selective delay-period activity were found after searching this region, we sampled an overlapping, but more posterior, range from the right hemisphere (9-22 mm anterior; see Fig. 6, B and C). A total of 65 electrode penetrations separated by no more than 2 mm were made at 51 sites in the two hemispheres. Given this density of sampling, we think it is unlikely that we failed to sample the region corresponding stereotaxically to that which contained the delay-period activity found in animal 1. The pronounced difference in the number of delay-selective cells observed between the animals may reflect a gross difference in the functional organization of their cortices, or perhaps different behavioral strategies used by each animal might have resulted in different brain structures or neural mechanisms being used.
View larger version (12K):
[in a new window]
FIG. 5.
Cell from animal 2 displaying delay selectivity during cross-modal tasks. First 1,300 ms and last 1,200 ms of a trial are depicted because data were collected from trials with delay periods which varied from 1,600 to 2,100 ms. Delay selectivity in each task was highly significant ( P < 0.001 in each task, n = 55 ± 6 trials in each histogram).
View this table:
TABLE 1.
Percent of delay-selective cells by animal
View larger version (20K):
[in a new window]
FIG. 6.
Recording locations. Recording locations are depicted with shading. Extent of recording region continues into the superior temporal sulcus. These schematics are based on histological reconstruction.
0.72, P < 0.001, n = 111). No such trend was observed in animal 2 (Fig. 8B, R = 0.02, P > 0.50, n = 119), even when analysis was restricted to cells with delay selective in at least one of the two cross-modal tasks (R = 0.08, P > 0.50, n = 21).
View larger version (19K):
[in a new window]
FIG. 7.
Data from 1 cell in animal 1 displaying relationship in selectivity between the 2 cross-modal tasks. For all cells, condition 1 in visual-to-auditory task and condition 2 in auditory-to-visual task tended to show similar levels of delay activity. Likewise, visual-to-auditory condition 2 and auditory-to-visual condition 1 were correlated. n = 26 ± 2 trials in each histogram.
View larger version (19K):
[in a new window]
FIG. 8.
Relationship between delay activity in the 2 cross-modal tasks. Differences in firing rate during delay period between the 2 trials in each cross-modal task are plotted separately for each animal. 
, cells with a statistically significant delay-period differences in either of the cross-modal tasks; 
, cells with no such differences. A: there was a strong correlation in animal 1 (R = 0.72, P < 0.001, n = 111), which suggests a very specific form of long-term memory for the associations the animal learned. B: there was no correlation in animal 2 (R = 0.02, P > 0.50, n = 119).
in the context of visual-visual associations using the same type of associational DMS task. The data here show cross-modal associations also can generate such a representation.
View larger version (14K):
[in a new window]
FIG. 9.
Correlation between sample response selectivity and delay selectivity in cross-modal tasks. Delay differences are plotted against sample response differences for cells with a statistically significant difference in delay firing for auditory-to-visual (A) and visual-to-auditory (B) tasks. There is a weak correlation in both tasks (R = 0.35, P = 0.04, n = 61 for auditory-to-visual and R = 0.37, P = 0.002, n = 69 for visual-to-auditory). Included in this analysis are 56 cells that were tested on only the cross-modal tasks. , animal 1; 
, animal 2.
; Fuster and Jervey 1982; Maunsell et al. 1991; Miyashita and Chang 1988). Many cells were stimulus-response selective while not being delay selective, and many were delay selective while not response selective (Figs. 3, 5, 7, and 11).
View larger version (16K):
[in a new window]
FIG. 11.
Delay activity predicted behavior. A negative correlation in delay rate differences existed between the 2 trial types in each cross-modal task. R = 0.80, P < 0.001, n = 23. Hence, population of cells showed a reversal in delay selectivity with incorrect responses.
similarly noted a strong correlation between sample responses and delay activity in an auditory-to-visual task relative to a visual-to-visual task. Thus there appears to be some connection between sensory and delay activity, at least in certain circumstances.
View larger version (22K):
[in a new window]
FIG. 10.
Delay activity predicts behavior. Data from 1 cell collected during auditory-to-visual task are sorted with respect to correct and incorrect behavioral responses. Using a 2-factor analysis of variance, this cell's delay activity had no significant effects with respect to sample stimulus or response accuracy (correct/incorrect) alone, but interaction term was highly significant (P < 0.0001). This interaction represents the reversal of the delay selectivity between correct and incorrect responses. As in Fig. 5, a gap exists in histograms because spikes were collected from trials of varying delay length. First 1,600 ms and last 1,300 ms of trials are plotted. Rasters show neural activity on a trial-by-trial basis. Only every 7th trial is illustrated for rasters representing correct responses.
0.80, P < 0.001, n = 23), which indicates that delay selectivity over the population predicted whether the animal would respond correctly.
View larger version (15K):
[in a new window]
FIG. 12.
Typical sensory responses to the mask. These are data from 1 sample condition picked from each of 3 different cells. The particular sample stimulus used in each condition is indicated underneath each histogram. These are examples of a cell with an excitatory mask response (A), a delay-selective cell with no response to the mask (B), and a delay-selective cell with an inhibitory mask response (C).
; Eskandar et al. 1992; Miller et al. 1993). Miller et al. (1993) reported selective delay activity immediately after the sample stimulus, but this activity was erased by stimuli that intervened between the sample and the matching stimulus. In that study, the animal had to attend to the intervening stimuli. The visual mask in the current study differed from that of Miller et al. (1993) in that the animal could ignore this intervening stimulus and still successfully perform the task.
; Gross et al. 1979; Vogels et al. 1995). During a sequential DMS task, Miller et al. (1993) found cells in inferotemporal cortex that showed a decreased response to matching test stimuli relative to nonmatching test stimulia match suppression effect. Using a similar task, Ferrera et al. (1994) reported a similar "match suppression" in parietal visual cortex and area V4, although the effect was small.
, this analysis was restricted to cells exhibiting excitatory sensory responses, and a two-way ANOVA was used (stimulus against match/nonmatch). We observed significant match/nonmatch modulation in 14/87 cells in the visual-to-visual task and 17/91 cells in the auditory-to-visual task. Modulation of match stimuli was seldom observed in the visual-to-auditory and auditory-to-auditory tasks (2/30 and 0/20). A match/nonmatch index (Miller et al. 1993) was calculated for cells in the visual-to-visual and auditory-to-visual tasks and then tabulated in frequency histograms (Fig. 13).
This index is a normalized difference between match and nonmatch responses. A positive index value means matching stimuli evoked a higher sensory response. In neither task did mean indices deviate appreciably from zero:
0.014 ± 0.014 (mean ± SE) for the visual matching task and 0.012 ± 0.013 for the auditory-to-visual task.
View larger version (23K):
[in a new window]
FIG. 13.
Frequency bargraph of match/nonmatch indices obtained from visual matching task (A) and auditory-to-visual association task (B). Both suppression and enhancement of matching test responses were observed without bias toward either one. , number of cells with statistically significant values; 
, number of nonsignificant cells. Peaks are the total number of cells. Data are from both animals.
sample/test modulation (Eskandar et al. 1992; Gross et al. 1979; Mikami and Kubota 1980; Riches et al. 1991). Like these earlier studies, we compared sample and test responses for the visual matching task, whereas for the cross-modal tasks, visual sample responses from the visual-to-auditory task were compared with visual test responses in the auditory-to-visual task.
DISCUSSION
Abstract
Introduction
Methods
Results
Discussion
References
; Miyashita and Chang 1988). This activity was observed not only in a simple visual matching task, but also during cross-modal tasks in which auditory and visual stimuli were associated. Stimulus specific delay activity could be evoked by auditory stimuli (in the auditory-to-visual task), confirming earlier studies showing that nonvisual stimuli can trigger memory-related neural activity in visual cortex (Colombo and Gross 1994; Haenny et al. 1988). Stimulus-selective delay activity also existed when animals expected to respond to an auditory stimulus but only when the task involved visual stimuli. Furthermore, we showed that delay activity could reveal long-term representations for learned cross-modal associations, and we showed that delay activity also could predict correct or incorrect responses in one behavioral task.
report a similar phenomenon for unimodal visual associations in IT, we show that it can apply to cross-modal associations. Just as in Sakai and Miyashita, this representation probably did not exist before training.
observed that paired associates tended to elicit correlated levels of delay activity, paired associates in the current experiment produced negatively correlated activity. This was probably a consequence of having only two associated pairs in our tasks. We defined the correct response to associated stimuli (visual A and auditory A, or visual B and auditory B) as a lever release. With only two stimuli for each modality, however, it was possible for the animal to infer that the correct response to associated stimuli was to not release the lever, providing that it paired visual A with auditory B and visual B with auditory A. The task could be correctly completed using either set of rules. Because the delay period activity in animal 1 was consistent with the latter formulation, we infer that the animal settled on it. Based on studies showing consistent mnemonic effects produced by associated stimuli (Maunsell et al. 1991; Sakai and Miyashita 1991), if three or more pairs were used in this experiment, there likely would have been a straightforward correlation between operationally defined pairs.
have reported that delay-selective cells tend to cluster into localized spots, and Fuster and Jervey (1982) reported that most delay-selective cells resided locally in the ventral bank of the STS. We likewise observed such clustering in animal 1 and sampled most delay-selective neurons from that region. It is possible that the sampling in animal 2 may not have been adequate to find a similar focal point. The difficulty in finding delay-selective cells representing associations is further demonstrated in Sakai and Miyashita (1991) who, using 12 pairs of associated visual stimuli, only found that 2% of their sample displayed this delay activity phenomenon. It is also possible that the neural representation of the task was different in animal 2, and did not involve neurons with selective delay-period activity in IT. Neurons with delay activity might have existed in a different brain region or the task might have been solved without delay period selectivity being obvious in the firing rates of individual neurons. A difference in neural representation for almost identical tasks, and its connection with behavioral strategy, has been reported by Miller and Desimone (1994).
; Eskandar et al. 1992; Mikami and Kubota 1980; Miller et al. 1993; Riches et al. 1991). Other reports that describe delay-period activity have involved large numbers of animals, which may be needed to make a sufficient number of observations (Fuster 1990; Fuster and Jervey 1982).
). Task difficulty could play a role in the strength of delay activity observed in various studies. Also, Miyashita (1988) found that stimulus familiarity may be important in determining whether delay activity is seen. He found that although familiar stimuli evoked selective delay activity, novel stimuli evoked little, even though the animal performed the task with the novel stimuli just as well.
reported that the amount of delay activity in IT (both selective and nonselective) correlates with behavioral performance in a short-term memory task. Wilson et al. (1990) showed that delay activity in IT cells can predict behavioral responses, but this was in the context of a simple delayed response task and could have represented activity related to motor set (see Kurata and Wise 1988).
showed, delay activity disappeared with passive viewing of the exact same stimulus sequence that occurred in the tasks. Second, in the auditory-to-visual task, we found that delay activity could predict robustly correct and incorrect responses, a property that has not been previously demonstrated in sensory cortex for motor-independent memory. Studies have shown this property for DMS tasks in prefrontal cortex (Funahashi et al. 1989; Watanabe 1986) with isolated examples but not so clearly and systematically across a large population of cells. Because the passive viewing and incorrect trials data were only possible from one animal subject in this study, how this data applies to all animal subjects is not known.
; Ferrera et al. 1994; Fuster and Jervey 1982; Haenny et al. 1988; Maunsell et al. 1991; Miyashita and Chang 1988). Second, although selective sensory responses remained purely visual, selective delay activity could involve memory for both auditory and visual stimuli and thus represent a form of multimodal sensory integration in visual cortex (Haenny et al. 1988; Maunsell et al. 1991). Finally, we also show that a clear long-term memory representation for cross-modal associations observed in delay activity can exist without any similar representation in sensory responses.
have shown that a sensory response to a visual stimulus correlates with delay activity evoked by the stimulus paired associate in a unimodal visual association task. No such relationship was observed for the cross-modal associations in this study. Thus even if the animal was using a form of imagery to mediate short-term memory, this imagery did not share the same representation as that evoked by viewedstimuli.
; Goldman-Rakic 1991; Watanabe 1990) and for cross-modal associations in the supplementary motor area (Tanji and Kurata 1985). Thus delay activity in IT may just be part of a more distributed brain representation for short-term memory. Our results suggest that short-term memory representations, while potentially diffuse in nature, also can be specific for certain brain areas because no selective delay activity was observed in IT during the purely auditory task. Selective delay activity triggered by auditory sample stimuli in the cross-modal tasks could originate through pathways that include projections from multimodal association areas that border IT or from medial temporal structures that are thought to play a critical role in associational memory (Felleman and Van Essen 1991; Morel and Bullier 1990; Suzuki and Amaral 1994; Van Hoesen 1982).
| |
ACKNOWLEDGEMENTS |
|---|
We thank G. M. Ghose, C. E. Landisman, and C. J. McAdams for helpful comments on preliminary versions of the manuscript and B. Noerager for technical assistance.
This research was supported by National Eye Institute EY-05911 and grants from the Office of Naval Research and the McKnight Foundation.
| |
FOOTNOTES |
|---|
Address for reprint requests: J. R. Gibson, Dept. of Neuroscience, Brown University, Providence, RI 02912.
Received 27 February 1997; accepted in final form 5 June 1997.
| |
REFERENCES |
|---|
|
Abstract
Introduction
Methods
Results
Discussion
References
|
|---|
This article has been cited by other articles:
|
|
 |
|
  T. M. Herrington, N. Y. Masse, K. J. Hachmeh, J. E. T. Smith, J. A. Assad, and E. P. Cook The Effect of Microsaccades on the Correlation between Neural Activity and Behavior in Middle Temporal, Ventral Intraparietal, and Lateral Intraparietal Areas J. Neurosci., May 6, 2009; 29(18): 5793 - 5805. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K.L Hoffman and N.K Logothetis Cortical mechanisms of sensory learning and object recognition Phil Trans R Soc B, February 12, 2009; 364(1515): 321 - 329. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 U. Noppeney, O. Josephs, J. Hocking, C. J. Price, and K. J. Friston The Effect of Prior Visual Information on Recognition of Speech and Sounds Cereb Cortex, March 1, 2008; 18(3): 598 - 609. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Y. Masse and E. P. Cook The Effect of Middle Temporal Spike Phase on Sensory Encoding and Correlates with Behavior during a Motion-Detection Task J. Neurosci., February 6, 2008; 28(6): 1343 - 1355. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y.-D. Zhou, A. Ardestani, and J. M. Fuster Distributed and Associative Working Memory Cereb Cortex, September 1, 2007; 17(suppl_1): i77 - i87. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Y. E. Cohen, F. Theunissen, B. E. Russ, and P. Gill Acoustic Features of Rhesus Vocalizations and Their Representation in the Ventrolateral Prefrontal Cortex J Neurophysiol, February 1, 2007; 97(2): 1470 - 1484. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Spitsyna, J. E. Warren, S. K. Scott, F. E. Turkheimer, and R. J. S. Wise Converging language streams in the human temporal lobe. J. Neurosci., July 12, 2006; 26(28): 7328 - 7336. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Y. E. Cohen, B. E. Russ, and G. W. Gifford III Auditory processing in the posterior parietal cortex. Behav Cogn Neurosci Rev, September 1, 2005; 4(3): 218 - 231. [Abstract] [PDF]
|
|
|
|
|
|
H. C. Tanabe, M. Honda, and N. Sadato Functionally Segregated Neural Substrates for Arbitrary Audiovisual Paired-Association Learning J. Neurosci., July 6, 2005; 25(27): 6409 - 6418. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. J. Zatorre, M. Bouffard, and P. Belin Sensitivity to Auditory Object Features in Human Temporal Neocortex J. Neurosci., April 7, 2004; 24(14): 3637 - 3642. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Falchier, S. Clavagnier, P. Barone, and H. Kennedy Anatomical Evidence of Multimodal Integration in Primate Striate Cortex J. Neurosci., July 1, 2002; 22(13): 5749 - 5759. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Messinger, L. R. Squire, S. M. Zola, and T. D. Albright Neuronal representations of stimulus associations develop in the temporal lobe during learning PNAS, September 19, 2001; (2001) 211431098. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. W. Bisley and T. Pasternak The Multiple Roles of Visual Cortical Areas MT/MST in Remembering the Direction of Visual Motion Cereb Cortex, November 1, 2000; 10(11): 1053 - 1065. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Messinger, L. R. Squire, S. M. Zola, and T. D. Albright Neuronal representations of stimulus associations develop in the temporal lobe during learning PNAS, October 9, 2001; 98(21): 12239 - 12244. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
