An understanding of the neural code in a given visual area is often confounded by the immense complexity of visual stimuli combined with the number of possible meaningful patterns that comprise the response spike train. In the lateral geniculate nucleus (LGN), visual stimulation generates spike trains comprised of short spiking episodes (“events”) separated by relatively long intervals of silence, which establishes a basis for in-depth analysis of the neural code. By studying this event structure in both artificial and natural visual stimulus contexts and at different contrasts, we are able to describe the dependence of event structure on stimulus class and discern which aspects generalize. We find that the event structure on coarse time scales is robust across stimulus and contrast and can be explained by receptive field processing. However, the relationship between the stimulus and fine-time-scale features of events is less straightforward, partially due to a significant amount of trial-to-trial variability. A new measure called “label information” identifies structural elements of events that can contain ≤30% more information in the context of natural movies compared with what is available from the overall event timing. The first interspike interval of an event most robustly conveys additional information about the stimulus and is somewhat more informative than the event spike count and much more informative than the presence of bursts. Nearly every event is preserved across contrast despite changes in their fine-time-scale features, suggesting that—at least on a coarse level—the stimulus selectivity of LGN neurons is contrast invariant. Event-based analysis thus casts previously studied elements of LGN coding such as contrast adaptation and receptive field processing in a new light and leads to broad conclusions about the composition of the LGN neuronal code.
How does the visual system represent information about the visual scene? To handle the inherent complexity of the visual world as well as the potentially complicated patterns present in the neuronal response, visual studies usually rely on crucial simplifications, including 1) testing only a delimited subset of all possible visual stimuli and assuming that the responses to this subset apply more broadly; and 2) implicitly assuming what is important about the neuronal response, such as firing rate or individual spike times. For example, ubiquitous “receptive field mapping” experiments measure the average visual stimulus that evokes neuronal spikes with the implicit assumptions that each spike conveys the same signal and that the neuron is only sensitive to a single feature. On the other hand, to remain agnostic to the role of different spike patterns, such as in applications of information theory (Reinagel and Reid 2000), one must use a simplified stimulus such as spatially uniform noise due to the resulting complexity of the measurements. Naturally, the relationship between stimulus and response can be quite intricate (Mante et al. 2008), and different stimulus classes result in different relationships between stimulus and response (Chander and Chichilnisky 2001; Felsen et al. 2005; Lesica et al. 2007; Sharpee et al. 2006; Zaghloul et al. 2005). At the same time, we expect there to be general organizing principles across stimulus classes that allow the information in each stage of the visual system to be interpreted by the next, comprising its “neural code.”
Here we study the neural code of X and Y cells in the LGN, which is significant both in understanding how visual stimuli are encoded into spike trains and in determining which elements are likely to be important to the visual cortex, which is the downstream target of the LGN. We focus on what is likely a crucial organizing principle to the neural code of LGN neurons: the episodic nature of their spike trains in the context of appropriately time-varying stimuli. The episodic structure refers to sparse clusters of spikes, “events,” separated by relatively long periods of silence, and has been analyzed in RGCs and LGN neurons in previous studies in simplified stimulus contexts (Berry et al. 1997; Fairhall et al. 2006; Kumbhani et al. 2007; Reinagel and Reid 2002). In particular, the majority of previous studies describing event structure have been performed in the context of temporally varying spatially uniform noise stimuli, where responses in retina and LGN can be extremely precise. Although event structure persists in more natural stimulus contexts, the time scales of LGN responses can be more than four-fold greater (Butts et al. 2007). Furthermore, studies at the retinogeniculate synapse, which simultaneously record from LGN neurons and their main retinal ganglion cell (RGC) input, demonstrate that often >50% of RGC spikes do not evoke corresponding LGN spikes (Carandini et al. 2007; Casti et al. 2008; Weyand 2007). As a result, it is not clear whether the neural code of LGN neurons shares the same principles as that of the retina (Carandini et al. 2007; Casti et al. 2008; Mukherjee and Kaplan 1995; Sincich et al. 2009) and whether it is different in more natural visual contexts.
Here we demonstrate that event structure is present across time-varying stimuli with a variety of spatial structure, including spatially uniform noise (SUN), spatiotemporal binary noise (SBN), and natural movies (NAT). It is also extremely consistent from trial to trial and persists across different contrasts. At the same time, we see differences in the details of this event structure with stimulus context, suggesting that some elements of LGN responses that were highlighted in previous studies generalize, whereas others are not relevant in more natural visual contexts. For example, both the temporal jitter of the first spike time and the variability in the number of spikes in each event tend to be much larger in NAT compared with SUN. The coarse event structure is almost completely preserved between high and low contrast conditions although many of the fine-time-scale features of events change in detail. This suggests that at a coarse level, LGN neuron selectivity to stimuli is contrast invariant, and apparent differences in tuning arise from fine-time-scale features.
The episodic structure of LGN spike trains suggests a natural framework for in-depth analysis of the spike train. We use this event-based structure to identify elements of the spike train that carry information about the stimulus and quantify how features such as the number of spikes, relative timing between spikes in the event, and other aspects of events change with stimulus context and contrast. In doing so, we both identify and analyze the elements that comprise the LGN neural code and present a series of new tools based on this event analysis.
Single-cell activity was recorded extracellularly in the LGN of anesthetized and paralyzed cats. Two animals were used for a total of nine electrode penetrations. Surgical and experimental procedures were performed in accordance with United States Department of Agriculture guidelines and were approved by the Institutional Animal Care and Use Committee at the State University of New York, State College of Optometry. As described in Weng et al. (2005), cats were initially anesthetized with ketamine (10 mg kg-1 im) followed by thiopental sodium (20 mg kg-1 iv during surgery and at a continuous rate of 1–2 mg kg−1 h−1 iv during recording; supplemented as needed). A craniotomy and duratomy were performed to introduce recording electrodes into the LGN (anterior, 5.5; lateral, 10.5). Animals were paralyzed with atracurium besylate (0.6–1 mg kg-1 h-1 iv) to minimize eye movements and were artificially ventilated. Geniculate cells were recorded extracellularly from layer A of LGN with a multielectrode matrix of seven electrodes (Eckhorn and Thomas 1993). The multielectrode array was introduced in the brain with an angle that was precisely adjusted (25–30° antero-posterior, 2–5° lateral-central) to record from iso-retinotopic lines across the depth of the LGN. A glass guide tube with an inner diameter of ∼300 μm at the tip was attached to the shaft probe of the multielectrode to reduce the inter-electrode distances to ∼80–300 μm. Layer A of LGN was physiologically identified by performing several electrode penetrations to map the retinotopic organization of the LGN and center the multielectrode array at the retinotopic location selected for this study (5–10° eccentricity). Recorded voltage signals were conventionally amplified, filtered, and passed to a computer running the RASPUTIN software package (Plexon). For each cell, spikes were sampled at 40 kHz, stored as 0.8 ms waveforms, and carefully sorted off-line with OfflineSorter (Plexon) using principal component analysis. We measured the quality of spike isolation in our recordings by calculating the percentage of spikes with interspike interval (ISI) <1 ms as in Chen et al. (2008). This percentage was zero in most of our cells and had an average of 0.04 ± 0.07%.
Cells were classified as X or Y according to their responses to counterphase sinusoidal gratings. We did not record from W-cells, which are only present in deeper layers of the LGN, nor did we record from any lagged cells (Saul and Humphrey 1992; Wolfe and Palmer 1998). Cells were eliminated from this study if they did not have ≥2 Hz mean firing rates in response to all stimulus conditions or if the maximum amplitude of their spike-triggered average in response to spatiotemporal white noise stimuli was not at least five times greater than the amplitude outside of the receptive field area. There were 32 X cells and 22 Y cells in this study with 11 neurons that were presented with all three stimuli, 20 neurons just recorded in the SUN condition, and 23 neurons recorded in the SBN and NAT conditions (but not the SUN condition). Trends present in neurons recorded in the context of all three stimulus classes were consistent with those observed with the larger population, and thus we present all of them together in this paper.
Each neuron in the main set of experiments was exposed to 64 repeated presentations of three different stimulus conditions: SUN, SBN, and NAT. All stimuli were presented with a 120 Hz monitor refresh rate and a pixel size of 0.2°. SUN stimuli consisted a single luminance displayed uniformly across the monitor, updated at 120 Hz with the luminance of each frame sampled from a Gaussian distribution. The widths of these distributions were either 0.55 of the total deviation allowed from mean monitor luminance in the high contrast (HC) condition or 0.20 of the deviation in the low contrast (LC) condition. SBN stimuli were 48 × 48, where each pixel was updated at 60 Hz and independently chosen to be either “black” or “white.” The luminance corresponding to black and white represented either 0.55 contrast deviation in HC and 0.2 in LC. The NAT condition was created from natural movies taken from a “cat-cam” recorded from a small camera mounted on top of a cat's head while roaming in grasslands and forests (Kayser et al. 2004). As in Lesica et al. (2007), to improve temporal resolution, movies were interpolated by a factor of two (from 25 to 50 Hz) using commercial software (MotionPerfect, Dynapel Systems) and then presented at 60 Hz, i.e., at 1.2 times the original speed. Following interpolation, the intensities of each movie frame were scaled to have the same mean luminance and a SD of 0.4 in HC and 0.15 in LC. These values were chosen such that the linear output of a typical LGN receptive field would have the same SD in the HC SBN and NAT conditions (and similarly with LC) (Lesica et al. 2007). Stimulus repeats were 30 s long in the SUN condition, and 10 s in the SBN and NAT conditions. A total of 64 repeats were presented in each case with the first two trials discarded due to initial nonstationarity. In addition to these repeated stimulus presentations, we also recorded the responses to a 5 min unique HC SBN sequence, which was refreshed at 60 Hz. A fraction of the time, we also recorded the response of the same neuron to the same HC SBN stimulus sequence, but updated at 120 Hz instead of 60 Hz (lasting 2.5 min), for the purposes of receptive field comparison (Supplementary Fig. S1),1 Also a different set of neurons was recorded in the context of long (>10 min) SBN and NAT conditions, allowing for receptive field comparisons between these two conditions (Supplementary Fig. S2) as well as to measure the correspondence between the filtered stimulus and event size in these conditions (Fig. 4, F and G).
Receptive field estimation
The receptive field in the SUN condition (e.g., Figs. 3C and 9E) is calculated using the spike-triggered average. In the SBN and NAT conditions, we performed maximum likelihood estimation of the receptive field (e.g., Fig. 3, A and B, and Supplementary Figs. S1 and S2) to separately measure center and surround as well as automatically account for stimulus correlations present in NAT. Briefly, the center and surround were assumed to be radially symmetric two-dimensional Gaussian distributions in space with a common center. For a given choice of center and surround widths: (x,y,wc,ws), there exists a unique choice of a center temporal filter and surround temporal filter that maximizes the likelihood of the observed spike train in the context of a generalized linear model (GLM) (Paninski 2004). Starting with an initial guess of the spatial parameters (x,y,wc,ws) based on the spike triggered average, we performed gradient ascent of the likelihood given these parameters and optimal choice of the resulting temporal kernels until the likelihood could not increase further. In the case of SBN, the resulting spatiotemporal kernel is equivalent to the spike-triggered average except that maximum likelihood estimation allows us to parameterize the spatial aspects of the fit providing more sensitivity to identify the surround, whereas the surround needs to be estimated on a pixel-by-pixel basis in using the spike-triggered average.
Peristimulus time histogram (PSTH) “events” were first defined in the PSTH at HC as times of firing interspersed with periods of silence lasting ≥8 ms. Each resulting event was further analyzed and broken into two events if there were two distinct peaks in its smoothed PSTH, where the firing rate in between the peaks was less than ½ the height of the lower peak. The PSTH was smoothed in this case by convolving the raw PSTH binned at 0.5 ms resolution by a Gaussian with a SD corresponding to the median first spike jitter measured across all events. These extended methods were only necessary for a fraction of cases and resulted in a reasonable division into smaller events as determined by visual inspection. Likewise these particular methods were chosen because they gave the most reasonable results with regards to visual inspection, but the results presented here do not qualitatively depend on the precise methods. Note that although the detailed methods differ, analyses in the same spirit has been previously applied by Berry et al. (1997) and Kumbhani et al. (2007).
Once the events in HC were determined, events at LC were then defined by aligning LC spikes to existing HC events the average spike time of which was within 8 ms, with a preference for an HC event that occurred before rather than after the LC spike (because it is known that spikes tend to be more delayed at LC than HC). If no corresponding HC event was found, a new event was created at LC.
Temporal properties of events
Once events were parsed, we could measure a variety of properties associated with the spikes that comprised them. These measures are divided between quantities that are accessible on a single trial versus those that represent variability across trials. For example, the number of spikes can be measured on a given trial, and its variability can be measured across trials (e.g., Fig. 6). Likewise we can do the same for temporal measurements of event properties. For example, the beginning of each event on a single trial is defined (for trials with at least one spike) and could in principle be measured, although would be meaningless without reference to the stimulus. As a result, only the variability in the absolute event time is reported: the first-spike jitter (Fig. 7A, bottom left), defined as two times the SD of the first spike on each trial of the event. It is multiplied by two to give a measure of the width of the event in time rather than just the SD as with the other measures considered.
Other temporal measures available on a single trial are limited because of the typically small number of spikes in events on a single trial (Fig. 6C). We define the “duration” (DUR) for a given event as the interval on each trial between the time of the first spike and the time of the last spike. By this definition, a single spike event has zero duration, although we only consider events with two spikes or greater when reporting the average quantities in Fig. 7. However, in considering the information measures, such a choice would neglect events with one spike, and thus one-spike events are defined to have a duration of zero for the purposes of the information calculation. The first ISI (ISI1) is handled similarly and is defined as the time difference between the first and second spikes. ISI1 is averaged across all trials with at least two spikes for Fig. 7 and defined as zero for single-spike events in the context of the information calculations.
Figure 7 reports the distribution of these quantities across the population of neurons considered in this study. To summarize each quantity across the population, we first measure its average value for each event of each neuron in a given stimulus condition. The ISI1, for example, will have a distribution across trials associated with a given event, and for each event, we measure the average ISI1 as well as its SD across all trials where it is observable (i.e., there are ≥2 spike). The mean and SD of the ISI1 for each event is simply averaged across events to arrive at a single value of each for a given neuron in a given stimulus condition. Finally, Fig. 7A reports the distribution of these quantities across neurons in the study. While there are many reasonable ways that we could calculate and report these average values, we chose this way to most faithfully represent the differences between event structure across different stimulus conditions. These distributions and variability of course play a much more regimented role in the calculation of information quantities, described below. We saw no significant dependence of these quantities between X and Y cells.
The data are presented in box plots to represent the distribution of each quantity across neurons. The horizontal line shows the median, and the box contains the middle half of the distribution (25–75%). Where shown, the error bars demarcate the full extent of the data, although in many cases they were cut off where one or a few outliers made them extend well beyond the region of interest. Unless otherwise stated, quantities were calculated for each neuron in each stimulus context, and the box plot shows the distribution across neurons. For example, in reporting the average duration in Fig. 7A, the duration of each event is calculated as a single quantity for a given neuron in a given stimulus context—by averaging the duration across trials and events—and then the box plot shows how the average quantity was distributed across the neurons in this study.
Response time scale and precision
As described in previous work (Butts et al. 2007; Desbordes et al. 2008), the response time scale is derived from the autocorrelation of the PSTH by fitting a Gaussian to the area of the autocorrelation above zero and multiplying the resulting SD by . By using a revised method for measuring response time scale (Desbordes et al. 2008), we were able to accurately measure the response time scale in the SBN condition and in the LC conditions for all three stimulus contexts (Fig. 7B). This modified method recovers the response time scales reported in Butts et al. (2007).
Label information, derivation, and application
The episodic structure of LGN spike trains suggests a means for decomposing the mutual information between the stimulus and spike train into two components: one that is associated with the larger-time-scale event structure and one associated with small-time-scale features within each event. Such an approach has the advantage over the full direct method of information calculation (Gaudry and Reinagel 2008; Strong et al. 1998) in that it explicitly limits the relevant spike patterns (or other defined features of the spike train) to those that exist within an event on a single trial. Given the small number of spikes in a given event that is common (Fig. 6C), information can be robustly calculated and attributed to specific event elements. This approach assumes that there are no significant correlations between the properties of successive events, which for the most part appears to be the case. The most general event-based approach would catalogue each spike pattern within the event much like the direct method catalogue spike patterns at every time point independent of event structure. Here, however, to focus on the contribution of particular structural elements of each event, we categorize each instance of an event with a particular event label, based on its particular properties. For example, we begin by classifying events based on the number of spikes in the event on each trial: Fig. 8B.
Consider a spike train that, over multiple repeats, has been broken up into N events, indexed by i. Following the “direct method” formulation for mutual information (Strong et al. 1998), for an experiment that lasts a time T, we assume that the relevant stimulus ensemble is sampled, with different stimuli at different times t. Then, the total mutual information rate between the stimulus ensemble S and response ensemble R is given by We now assume that the response can be broken into a binary variable denoting the presence of an event e at a particular time, and a second variable L representing the “label” associated with the event, i.e., a given response r → (e,L). Because a response is then identified by these two variables, we assume that the probability of a given response at a particular time is given by the probability that the relevant event ei is present (or absent), multiplied by the probability of that particular label given the particular event Because the label will not be defined on some trials, we also define fi as the fraction of trials for which a label is defined. For example, the probability of a particular label over the whole experiment is written as p(L) = Σi fi p(L|ei)/Σi fi.
This approximation allows a variant of the chain rule for mutual information to be applied. Decomposing the spike train into N discrete events (indexed by i), and ignoring contribution of zero-response to the information (as per Brenner et al., 2000), gives with p(L) defined in the preceding text.
The first term in the final expression above is the event information defined by Brenner et al. (2000). In this case, the assumption inherent in this measure—that events are independent of each other—is by design due to the separation in time between events. The second term is what we define as the label information and can be further rearranged in terms of entropies as shown in the main text. Note that the additional factor fi represents the fraction of trials in which the event label is defined because on some trials there are no spikes. This is equivalent to including an extra “null” label when tabulating p(L|ei).
Episodic structure in the neural response across stimulus class
We recorded from X and Y cells in the LGN during three types of stimuli (Fig. 1): SUN, SBN, and NAT. We chose this range of stimuli to study the structure of LGN spike trains present in stimuli approaching natural vision (NAT), while comparing these elements in conditions more commonly used to study LGN neurons (i.e., noise conditions). Making comparisons between stimulus classes was motivated by the substantive differences in time scale of the LGN neural response (often >4 times) between the SUN and NAT (Butts et al. 2007), which implies that conclusions drawn in previous studies that focused on one stimulus context might not generalize. Furthermore, noise stimuli have been used extensively in the past to characterize the nature of visual stimuli that the neuron responds to, but recent studies have highlighted the dependence of the neuron's receptive field on the overall stimulus context (Felsen et al. 2005; Lesica et al. 2007; Sharpee et al. 2006). As a result, we also compared the SBN condition to the NAT condition, which was adapted from movies recorded from the camera on the head of a cat that was freely roaming in the woods (Kayser et al. 2004). Finally, as stimulus contrast is known to affect processing (Lesica et al. 2007; Shapley and Victor 1978), all stimuli were presented at two contrasts for comparison.
We see that in all contexts, the neuron's response is comprised of episodic “events” (Fig. 1), characterized by one or more closely spaced spikes separated by much longer periods where there is no spiking at all. In the HC conditions (top), events were clearly visible by eye and can be simply distinguished by >8 ms gaps in the PSTH. Spikes are color-coded in Fig. 1 to represent their association with a given event. Event membership was similarly distinguishable in LC conditions (bottom), although in some cases, an increased number of events and increased trial-to-trial variability made event distinctions less clear-cut. As a result, we used a simple algorithm (described in methods) to associate each LC spike with an HC event. LC spikes that were >5 ms away from any spike in an HC event were considered new events, and are color-coded accordingly (e.g., Fig. 1, A and B).
The ability to decompose the spike train into events suggests a natural division between two time scales: short time scales relate to patterns of spikes within events and longer time scales correspond to intervals between events (Fig. 2A). Figure 2B shows the ISI distribution (black) of the neuron shown in Fig. 1 in the NAT condition. A majority of the small intervals are between spikes within an event (gray), leaving the larger ISIs corresponding to spike pairs between events. Separating these two types of ISI into different distributions (Fig. 2C) demonstrates this separation of time scales with most intraevent ISIs occurring within 10 ms (top), and a majority of interevent intervals >100 ms (bottom). Short ISIs thus associate spikes in the same event, and long ISIs signal separations between events.
For such an ISI criterion to be useful biologically, one must be able to make such distinctions on a single trial, whereas thus far we have made event distinctions using many repeats of the same stimulus. Because the ISI distribution does not fully separate into a small ISI and large ISI distribution (Fig. 2B), there would be errors in distinguishing event boundaries using only single trial criteria, both because an ISI could be too large within an event or there may not be a sufficiently large ISI in between events. We directly compared the event boundaries determined across trials compared with those estimated from single trials and found almost no errors in all three conditions (<0.1%) in correctly distinguishing event boundaries when they were greater than a given ISI (which was optimal around 8 ms in all 3 stimulus conditions). However, we discovered that there was often larger ISIs present within a given event, leading to error rates ≤20% in distinguishing the spikes that were in the same event. In other words, event classification that is apparent over multiple trials can be disrupted on single trials by large ISIs. This fact explains why the full ISI distribution (Fig. 2B) is not clearly bimodal: the within-event ISIs can extend past 30 ms (Fig. 4D). However, this might not be of practical importance because the signal from single LGN neurons is not considered for each neuron independently by downstream neurons in the cortex, which integrate over many more neurons (Alonso et al. 2001). As a result, spikes from other similarly tuned neurons arriving at the same time will make events whole and likely make the event boundaries more clear. In this sense, aligning events across trials in this analysis serves as a proxy for integrating spikes from related neurons as is likely to happen downstream in the visual pathway.
This separation of ISIs into two time scales was consistent across all three stimulus classes. However, the overall time scales of the ISIs—both within events (Fig. 2D) and between events (Fig. 2E)—differed in detail between stimulus classes with the SUN condition associated with the fastest time scales. The SBN and NAT conditions have much more similar time scales with NAT having the longest duration intervals within events and between events. As described in the following text, the differences in time scales across these stimulus classes suggest a dependence of the neuron's temporal processing on the stimulus context.
Episodic spike trains arise from temporal receptive field properties
The episodic nature of the neuron's spike train arises naturally from the temporal selectivity of LGN neurons to changing stimuli. Such temporal selectivity is evident in the neuron's spatiotemporal receptive field (STRF; Fig. 3A), estimated here using a maximum likelihood approach (see methods), which can directly measure the temporal components of the center and surround (Cai et al. 1997). This STRF is equivalent to the average stimulus to which the neuron responds and can be used to predict the neuronal response from the stimulus as well (Chichilnisky 2001; Simoncelli et al. 2004). The dominant contribution to the response is from the receptive field center (black), although it is also affected by processing of the surround (gray), which is opposite in sign and temporally delayed by ∼8 ms (dashed lines).
The temporal processing performed by the neuron that is captured by the STRF appears to be consistent when measured at different (but appropriately fine) time resolutions. As a control, for a subset of neurons, we measured the STRF using SBN stimuli that updated at both 60 and 120 Hz, and we found no substantive difference (Supplementary Fig. S1A). However, changing the spatial structure of the stimulus does affect temporal processing of the neuron. Because SUN stimuli simultaneously stimulate the center and antagonistic surround, the ∼8 ms delay between them leads to a sharper temporal kernel in this condition (Fig. 3B). The sharpness may also be accentuated by processing that is not captured by the linear receptive field (see discussion). We could not separately verify that the temporal processing in SUN was identical in the 60 Hz SUN condition because this slower refresh rate (with 16 ms in between each refresh) cannot adequately sample the finer temporal features of the SUN receptive field (Fig. 3B). However, comparing the temporal processing during 120 Hz SUN and SBN stimuli suggests that such differences in the temporal processing of the RF do not arise from the different time scales of the stimulus itself (Supplementary Fig. S1B).
The temporal selectivity of the receptive field, combined with the nature of the stimuli used in this study, underlies the episodic nature of the neuron's spike train. First, the biphasic nature of the temporal receptive fields (Fig. 3, A and B) is selective to transitions in luminance rather than constant luminance. Second, because time-varying stimuli naturally involve both increases and decreases in luminance over a particular range of time scales, a particular location in the visual field cannot get brighter without then getting darker (and vice versa). Thus a preferred stimulus for a given neuron will always be followed by a stimulus to which the neuron does not respond, resulting in gaps where the neuron does not fire in between times that it does, separated by intervals that are determined by both those of the stimulus and the temporal filter (Butts et al. 2007). This can be demonstrated by a linear comparison between the neuron's receptive field k and the stimulus as a function of time s(t): the filtered stimulus g(t) = ∫dt s(t − τ) k(τ). For example, in the context of SUN stimuli (Fig. 4A), the neuron considered in Fig. 3 has a simple temporal receptive field that selects for an off-to-on transition. Such a transition is highlighted (top, dashed line) to demonstrate the associated large value of the filtered stimulus with the appropriate latency (#1). However, this transition is naturally followed by an on-to-off transition—opposite from the receptive field selectivity—leading to a correspondingly negative value of the filtered stimulus (#2).
A large value for the filtered stimulus signifies the presence of a stimulus feature to which the neuron is tuned, and the peaks of the filtered stimulus are generally associated with response events (Fig. 4C). This can be demonstrated by a histogram of the values of g(t) at the average spike time of each event (Fig. 4D) compared with the overall distribution of g(t). As expected, events tend to occur for large positive values of the filtered stimulus, and this relationship is stronger for events with more spikes (Fig. 4E). We found equivalent results in SBN and NAT conditions (Fig. 4, F and G) except that the stimulus and spatiotemporal processing of the neuron result in longer time scales.
This analysis is quite similar to a standard approach to evaluating how well receptive-field-based models can predict the stimulus selectivity of retina and LGN neurons based on single spikes, formalized by the linear-nonlinear (LN) model of visual processing (Chichilnisky 2001). However, it specifically relates the time scales associated with the receptive field itself to the timing between events, offering an explanation of the differences in these time scales between stimulus classes (Fig. 2E). Furthermore, this highlights two important points that set the basis for further analysis: 1) single-neuron signaling is not a continuous representation of the stimulus but rather offers a snapshot only when preferred stimuli are present in the receptive field; and 2) during this snapshot—over the collection of spikes that comprise the event—the stimulus is relatively constant from the context of the receptive field filtering, raising the possibility that structural elements of the event itself (such as the number of spikes) might collectively signal a given stimulus rather than each spike signaling something independently.
Composition of events in terms of spike count and spike timing
Which structural elements of the event might signal additional information about the stimulus? Although the event itself might look quite complicated when considered across trials (Fig. 5), the neural code can only be based on structures present on single trials, leading to a much smaller number of possible elements that could be relevant. First and foremost, the structure of the event depends on the overall number of spikes. Then given a particular number of spikes, the timing of different spikes on a given trial, relative to one another, can carry additional information. While the absolute timing of the event is also important, absolute time refers to the explicit stimulus dependence (which defines the timing of each trial) and thus falls into consideration in the preceding text of the receptive field, which describes this relationship between the stimulus and response, either at a coarse level here or with more involved models of stimulus processing (Berry and Meister 1998; Gaudry and Reinagel 2007a; Keat et al. 2001; Mante et al. 2008).
The spike count within an event can be quite variable from trial to trial, such that the overall distribution of spike count resembles a Poisson distribution (Fig. 5, right). To make this comparison explicit, the variance of spike count is compared with the mean spike count (Fig. 6A), with the ratio between the two defined as the Fano factor. A Poisson process has a Fano factor of unity, such that the variability can be described by a single Poisson rate: in such a case, it is generally assumed that the number of spikes is simply useful as an estimate of the underlying rate (Rieke et al. 1997). The relationship between variance and mean can vary between the minimum-possible (Fig. 6A, solid lines), up through that of Poisson (i.e., variance equals the mean, dashed lines) and beyond. Overall the SBN condition has the most variability associated with it (Fig. 6A, middle) with most events clustered around a Fano factor of unity. Across all stimulus classes, there is more variability in LC compared with HC (Fig. 6B), and in general, events with more spikes have less variability than smaller events. The wide range of these distributions across events suggests that while LGN neurons can be quite variable in their response to some stimuli, their response to other stimuli can be relatively reliable.
For the most part, there are usually only a small number of spikes in an event on a given trial (Fig. 6C); this limits the number of possible timing comparisons that can exist within an event on a single trial. Because coding elements must be present on a large number of trials to be useful, here we only consider timing comparisons between first and second spike (the ISI1) and between the first and last spike (the event duration or “DUR”). Note that the fact that most events have relatively few spikes (Fig. 6C) suggests that even these differently defined measures will be carrying similar information.
Figure 7A (top) shows how the mean values of DUR and ISI1 across neurons vary with the stimulus type and contrast. As with the spike count, each quantity varies from trial to trial, although for timing measures, the amount of variability of each tends to be lower than the value of the median (Fig. 7A, bottom). In contrast to the relative timing measures within the event (ISI1 and DUR), the variability in absolute timing variability of the event, the first spike jitter (Fig. 7A), dominates the overall time scale of the event. Previous work studying the precision of LGN responses (Butts et al. 2007; Desbordes et al. 2008) defines an overall response time scale of the PSTH from its correlation function (see methods). This “response time scale” is likely important for representing the slower time scales of the stimulus (Butts et al. 2007) and, as demonstrated here (Fig. 7B), most closely follows the first-spike jitter rather than the timing of the structural elements of the event available on a single trial. While the jitter roughly matches the other time scales of the event in HC, it increases in LC, mirroring the overall response time scale. At the same time, the underlying timing structure within the event remains relatively consistent (Fig. 7A) (Desbordes et al. 2008).
Information conveyed by event structure about the stimulus
The measurements in the preceding text imply that elements of the event structure present on a single trial might be used to signal additional information about the stimulus. To determine whether this is the case, one needs to compare the magnitude of systematic variation in these elements from stimulus to stimulus (thereby identifying different stimuli) with the trial-to-trial variability in signaling the same stimulus. Consider a cartoon example (Fig. 8A) where a neuron responds to just four stimuli (vertical lines) of the entire stimulus ensemble during an experiment (rectangular box). In every case, the neuronal response is either one or two spikes. (For the purposes of illustration, we assume that the time scales between spikes within events are sufficiently small compared with the interevent spacing such that it can be ignored.) In this instance, can the number of spikes in the event convey additional information about the stimulus?
The answer depends on how the one- and two-spike responses are distributed within the events, and Fig. 8A shows two extremes. In the first example (left), the one- and two-spike responses are distributed randomly between the four stimuli, such that each event has the same proportion of each associated with it. In this case, a two-spike response is just as likely with any of the four stimuli, and as a result, there is no information in the spike number. By comparison, consider the other extreme (right), where there is the same number of one- and two-spike responses, but two stimuli (labeled with “a”) reliably evoke two-spike responses, and the other two (“b”) reliably evoke one-spike responses. In this case, knowing how many spikes were in a given response can increase one's knowledge from knowing that the stimulus was one of four possibilities to one of two possibilities, meaning that spike number provides an extra bit of information about the stimulus.
Following this intuition, we define the label information, which is the amount of information gained by labeling each event using the value of a particular event property, such as spike count. Following the formalism of Shannon's information theory (see methods), we define the label information IL as where T is the duration of the experiment, fi is the fraction of trials in which a neuron fired at least one spike during a given event ei, and p(L) = Σi fi p(L|ei)/Σi fi. Note that this expression is much like the mutual information between the label and the identity of specific events except that labels are not present on every trial, necessitating the additional fi terms. (For simplicity, in Fig. 8A we illustrate the case where fi = 1 for all events, but this is usually not the case for real data.)
Following the intuition of what mutual information represents, it is possible to re-express the label information into entropy-like terms: where FE = Σi fi, H[L] = -ΣL p(L) log p(L), and H[L|ei] = −ΣL p(L|ei) log p(L|ei). Here, the “label entropy” H[L] represents how much information a particular type of label could carry. In both the examples of Fig. 8A, this number is 1 bit because the neuron is equally likely to signal with one or two spikes. However, as this label could be either reliably signaling different stimuli (Fig. 8A, right), or alternatively represent trial-to-trial variability (left), there must be a second term in the equation for IL that adjusts for this noise, called the “label noise entropy” H[L|ei], which is averaged across all events. In the left example, each event has an equal number of one- and two-spike responses, meaning that the noise entropy is also 1 bit. This means the label information is zero because all the potential information in spike count results from trial-to-trial variability. Another way to say this is that the one- versus two-spike events are not stimulus locked, and the probability of getting either does not depend on the stimulus. By comparison, there is no noise entropy in the right example because there is no variability of the response within each event: as a result, all the variability of the label is systematic, and the label information is 1 bit.
Of course, there is also information about the stimulus contained in the overall timing of events, independent of their labeling. The presence of an event itself carries a significant amount of information because it signals four stimuli out of the entire stimulus ensemble, and it carries the same amount of information in the left and right examples, as the timing of events themselves are the same. A measure of the information contained in events has been previously proposed for more general contexts (Brenner et al. 2000), and is given by where rE(t) is the event firing rate over time. Like the label information, this information also depends on how reliably each event is signaled, which is implicit in the rate rE(t).
In fact, the label information defined in the preceding text is derived such that the total information in the spike train is given by the sum of the event information IE and label information IL. This decomposition of the mutual information relies on two approximations already discussed: 1) separate events convey independent information and 2) features within an event are conveying information about single stimuli (rather than time variations in the stimuli on small time scales), both of which are suggested by the observed event-based structure of LGN spike trains. Thus this event-based analysis allows us to directly compare the information in different event features as well as calibrate this to the information conveyed by the presence of events themselves.
What are informative features of events?
Using the label information, we now can measure which intraevent elements are best at signaling. We start with the event feature that is the most intuitive: the number of spikes in an event. As demonstrated in Fig. 4, the average number of spikes is correlated with the degree to which the stimulus matches the receptive field. Simply put, the stronger the stimulus, the more vigorous the response of the neuron. However, it is unclear how much the average differences in spike count are disguised by the often Poisson-like trial-to-trial variability described in the preceding text.
Figure 8B shows the amount of label information conveyed by spike count across the different stimulus conditions, averaged across the population of LGN neurons recorded. The label information for each condition is a particular quantity in bits/second, but this number is not meaningful on its own. Rather we are interested in the fraction of information available using within-event properties (i.e., the label information IL) compared with the event information IE (Fig. 8C). These information measures are roughly proportional, such that neurons with the least information in their event timing also have the least to gain in labeling events.
Because of their proportionality, we will express label information as a quantity normalized to the event information, representing the fraction of information gained by labeling the events, compared with ignoring them. In addition to calibrating the label information to be more meaningful, this also provides a more consistent measure of label information across neurons (Fig. 8D, left, compared with B). It also makes the label information relatively invariant to changes in the overall reliability of the response, which is known to change with contrast (Lesica et al. 2007) but does not correspond to a change in the elements of the spike train that convey information.
Which structural elements of events are most informative? We calculate the label information fraction for the following event properties (Fig. 8D): the spike count (1st column), event duration (2nd column), ISI1 (3rd column), and presence of bursts (4th column) (Alitto et al. 2005; Denning and Reinagel 2005; Lesica et al. 2006; Wang et al. 2007). We see that duration and first ISI contain about the same amount of information, as might be expected given that they are largely equivalent due to the fact that a large fraction of the events have only one or two spikes (Fig. 6C). In the meantime, spike count conveys less label information and bursts convey the least.
What factors explain the differences in label information between the different event elements? For example, spike count generally carries much less information than event duration, which is most striking in the context of the SUN stimulus (Fig. 8D). Here most events contain only one or two spikes on a given trial, meaning that spike count cannot vary nearly as much as event duration, and as a result duration can represent a richer signal to represent different stimuli. For spatiotemporal stimulus conditions, spike count can have a wider range of values, and the difference in label information between spike count and duration is not as striking. However, the event duration is more consistent from trial to trial than spike count: as a result event duration conveys more information in these cases as well.
Given the larger number of spikes in the SBN and NAT conditions, it is somewhat surprising that first ISI can convey as much information as event duration. However, first ISI is equivalent to duration for one- and two-spike responses. For larger events, there is an apparent correlation with first ISI and duration with both having roughly the same amount of variability.
Finally, bursts convey the least amount of label information. As with previous work (Alitto et al. 2005; Denning and Reinagel 2005; Lesica et al. 2006; Ramcharan et al. 2000; Wang et al. 2007), we use an extracellular signature of bursting: consecutive spikes with ISIs <4 ms following a gap of more than 50 or 100 ms. (Both 50 and 100 ms were used with 50 ms giving slightly better performance.) Burst information was previously measured in the SUN condition using a different (though effectively similar) information measure (Denning and Reinagel 2005; Gaudry and Reinagel 2008), and here we find a similar magnitude for the information in bursts (between 1 and 6 bit/s, average of ∼2 bit/s). Bursting conveys the least amount of information of these event-label measures, both because bursting only results in a binary signal (presence of absence of burst) and because in many cases, bursting is less consistent from trial to trial than these other measures. We should note that the cellular mechanism underlying bursting—an inactivating low threshold calcium current that makes the neuron respond more robustly following hyperpolarization (Lesica et al. 2006; Sherman 2001)—likely contributes to the consistency of other measures (such as ISI1). But in this light, it is clear that the underlying mechanisms do not make the extremely small ISIs associated with a burst a reliable signal on their own, relative to the other measures considered here.
Direct correspondence between events in HC and LC
Manipulating the contrast of a visual scene is one way to test the robustness of particular coding elements because lowering the contrast changes the strength of the signal while leaving the same temporal sequences (and in NAT, the same objects). This manipulation of the visual stimulus can thus be used to identify elements of the spike train that are present across contrasts and thus robustly encode the content of the scene.
The ability to decompose spike trains into discrete events allows in-depth analysis of the structural elements that comprise the spike train and measurement of the degree to which each element changes with contrast. Figure 9A shows a raster plot of a typical neuron's response to multiple repeats of the NAT stimulus in HC and LC conditions similar to that shown in Fig. 1. At this resolution, it is clear that there is a correspondence between events in HC and LC, but the events clearly have differences apparent at finer time scales.
Although most HC events have a corresponding event in LC, there are also a number of events that are not shared between HC and LC, which we call orphan events. Orphan events are only present in a fraction of the trials and generally consist of single spikes, such that in all three stimulus conditions, the most common orphan event is a single spike occurring on only 1 of 62 repeats (Fig. 9B, 1st bar on either side of dotted line), and nearly all orphan events have an average of <0.1 spike (equivalent to there being <6 spike distributed across the 62 trials). As a result, if we define an event to be “reliable” if it occurs on >10% of the trials, then orphan events represent only 2% of the total number of events in all three stimulus conditions (Fig. 9C).
Due to the small number of spikes in orphan events, they only make up a small fraction of the total number of spikes in a given neuron's spike train, and in all conditions but LC-SBN constituted <5% of the total number of spikes (Fig. 9D). As a result, it is unlikely that they have any significant effect on the overall tuning (i.e., receptive field) of the neuron. To verify this, we compare the spike-triggered average (STA) from all spikes (Fig. 9E, solid line) with the STA excluding all spikes that were part of orphan events (dashed line). Because orphan events can only be identified by looking across many repeats, this analysis could only be performed in the SUN condition, where there is a long enough stimulus (30 s) to achieve a statistically reliable estimate of the receptive field and enough repeats to distinguish orphan events. As expected, the receptive fields are completely overlapping, suggesting that orphan events have negligible effect on the characterization of the average stimulus to which the neuron responds.
This event analysis thus demonstrates that the neuron responds to the same stimuli in HC and LC, such that every significant response of the neuron in HC has a corresponding response in LC. In this way, the events themselves are contrast invariant, suggesting that—at least at coarse time scales—the neuron responds to the same stimuli in HC and LC. On the surface, this appears to contradict earlier studies that demonstrated shifts in the receptive field of the neuron at different contrasts (Chichilnisky 2001; Lesica et al. 2007; Zaghloul et al. 2005), suggesting that the stimulus selectivity of the neuron had changed. Similar shifts were observed in our data (Supplementary Fig. S2): for example that lower contrast is associated with receptive fields have a longer latency to peak. As a result, this analysis implies instead that such shifts are a result of changes in the details of the structure and timing within events themselves with contrast as analyzed in the following text.
Are structural elements of events contrast invariant?
The contrast invariance of the coarse event structure, combined with the shift of the LGN receptive fields with contrast, implies that at least some event properties themselves shift with contrast. One of the most apparent differences between HC and LC events is the difference in their absolute timing (e.g., Fig. 9A, vertical bars). Rather than being systematic—as suggested by studies that either used repetitive stimuli like gratings (Shapley and Victor 1978) or focused on average properties (Chander and Chichilnisky 2001; Lesica et al. 2007; Zaghloul et al. 2005)—such latency differences vary from event to event, creating the distributions shown in Fig. 10A for a typical neuron. These distributions are all biased toward a positive latency shift (i.e., lower contrast has greater latency), and this average shift (Fig. 10B) matches the latency to peak of the temporal receptive fields in each stimulus condition (Supplementary Fig. S2). Such latency changes likely result in the observed changes in the temporal sensitivity of the neuron and the overall shape of the receptive field because, in addition to delaying the receptive field in LC, the change in latency can spread it out over the larger width of the latency-change distribution for each neuron (Fig. 10C). In the meantime, because both increased latency and variability in LC involve timing shifts, in that are smaller than the effective trial-to-trial variability (Fig. 7A), this suggests that the changes in receptive field properties associated with contrast shifts may not have much practical consequence.
We also analyzed the degree to which other average properties of events (i.e., those indexed in Fig. 7A) change with contrast (Fig. 11A). For reference, the overall change in contrast (−64%) is shown as a dashed line, which serves as a basis for comparison to other event property shifts. For example, the much smaller change in firing rate (−35%) is commonly cited as evidence that responses properties are relatively preserved across contrast (Gaudry and Reinagel 2007b; Meister and Berry 1999). This change in overall firing rate is reflected in the change of the within-event property of spike count (2nd column), which is of similar magnitude. Many of the temporal properties of events change even less, at least on average, from HC to LC, and the changes in the average duration and first ISI were distributed around zero (Fig. 11A, 3rd and 4th columns).
These small changes in average properties again suggest a degree of contrast invariance. However, as the preceding analysis of latency changes suggests, larger changes present from event to event might partially or mostly cancel in the average across events. Thus to rigorously address the degree to which each structural element of the event considered earlier might be contrast invariant, we calculate the label information of each element using “combined contrast” events that include trials from both the HC and LC experiment. By pooling trials at different contrasts within events, any changes in event properties with contrast are implicitly treated as additional trial-to-trial variability. If such variability exceeds that already present within events, there will be less “contrast-invariant information” than if each contrast is considered separately. For example, consider spike count. Any change in spike count with contrast would add ambiguity to the meaning of a particular number of spikes because a smaller number of spikes might correspond to either the same stimulus at lower contrast or a different stimulus at the same contrast. If the same stimulus was represented by different numbers of spikes at different contrasts, then spike count would be a less meaningful signal in stimulus contexts with changing contrasts (like NAT) and convey less contrast-invariant information.
We found that the contrast-invariant information still conveys >10% more information than event timing for most event properties considered (Fig. 11B). However, it is ∼60% less than the label information conveyed using contrast-specific codes (Fig. 8D) for each event property and across all stimulus conditions. This decrease results from a systematic shift with contrast for spike count. While there is little systematic shift in ISI1 and DUR with contrast, DUR and ISI1 can both increase and decrease in different events (with the average shifts across events mostly canceling out), making for a more ambiguous signal on an event-by-event level.
Thus these within-event properties are partially influenced by contrast but can still represent a significant fraction (10–15%) of the total information in the spike train if considered in a contrast-invariant way.
We observed episodic (or “event-based”) structure of LGN spike trains in every LGN X and Y cell recorded across a range of time-varying stimuli. This structure consists of small groups of spikes with short ISIs (<10 ms) separated from each other by longer interevent intervals. The same episodic structure persists across contrast, such that nearly all of the individual events in HC had a direct analog in the LC condition. The robust presence of episodic spike trains across the conditions tested implies that it is an important element in the LGN neural code (at least for X and Y cells) and suggests the utility of event-based analyses to provide insight into the important elements of the code.
By breaking up the spike train into events, the many elements of the spike train that might comprise the LGN neural code can be catalogued and quantitatively compared. For example, how consistent they are across trials and the degree that they are preserved across contrast is related to the degree that each carries information about the stimulus, and we have introduced new methods of analysis (the label information) to reflect this. In applying these analyses, we uncover the ways in which information is represented by LGN neurons, which has important implications for downstream cortical representations of the visual scene.
Dependence of event properties on stimulus class and brain region
The presence of episodic structure of spike trains has been previously studied in the context of SUN in both the retina (Berry et al. 1997) and LGN (Kumbhani et al. 2007; Reinagel and Reid 2000, 2002). SUN stimuli (and temporally modulated gratings used by Kumbhani et al.) are appealing both because the stimulus is low dimensional and can be adequately sampled in a relatively short experiment and because responses in the LGN are generally very reliable and temporally precise. However, studies in more natural stimulus contexts (Butts et al. 2007) have revealed that the time scales of responses in the LGN can be dramatically different, raising the possibility that conclusions derived from these simpler contexts might not apply in more natural contexts.
These previous studies have laid the groundwork for the parsing of neural responses into events and distinguishing between information at coarse and fine time scales. However, the most extensive study to date (Berry et al. 1997) found that the first spike time and spike number are the most salient elements of the neural code in the retina; this does not appear to be the case in the LGN, particularly in NAT and SBN conditions. First, we find that the first-spike jitter is on the same scale or larger than the overall event duration (Fig. 7). This observation is consistent with previous studies for the SUN condition because events most often consist of single spikes, and thus the jitter is still quite small. However, in the context of the much longer events in SBN and SUN, the first spike time is much less precise, especially relative to other temporal aspects of the event such as ISI1.
Second, we found that spike number was also not nearly as reliable as shown previously (Berry et al. 1997) nor did it match expectations given a much broader study of neuronal variability in the retina, LGN, and cortex (Kara et al. 2000). The latter study, where slowly moving grating stimuli evoked long episodes containing high firing rates, points to a possible explanation for this discrepancy. At such high firing rates, the neuron's refractory period makes spiking much more regular and reliable with much lower Fano factors (Berry and Meister 1998; de Ruyter van Steveninck et al. 1997; Kara et al. 2000). We also see this trend in our data, where events with a larger numbers of spikes have a proportionally smaller variability (Fig. 6A). However, for the most part, events still consisted of a small number of spikes (Fig. 6C) and thus would be expected to be less influenced by refractoriness and have a more variable spike count.
The amount of spike count variability of the LGN observed in this study is also higher than that observed in the retina. Additional processing at the retinogeniculate synapse can lead to increased variability because LGN neurons are driven by only a fraction of the spikes from their main RGC input (Carandini et al. 2007; Casti et al. 2008; Weyand 2007). This transmission can be modulated by the state of alertness (Bezdudnaya et al. 2006; Cano et al. 2006; Hartveit and Heggelund 1994; Weyand 2007), and the amount of LGN variability may be lower in the awake state that under anesthesia, and thus be closer to that reported in the retina. Nevertheless any noise in the process of retinogeniculate transmission would result in less reliability in first spike time (because the 1st spike might sometimes be dropped) as well as the total spike count in the event. Such a possibility might also explain, in part, the increasing Fano factor at each stage of the visual pathway (Kara et al. 2000) as spikes are dropped at each synapse and event durations become correspondingly shorter. In any case, although there are many qualitative similarities between event structure in the LGN and those described in past studies of the neural code in retina, it is clear that many particular aspects of event structure do not generalize.
Temporal scales of events
The decomposition of the spike train into events manifests as a clear division between the time scales within events and between events. The separation of these two time scales recalls the familiar debate about the importance of spike timing versus firing rate in neural coding (Theunissen and Miller 1995). In considering events as single units, the overall event time is related to receptive field-based coding the stimulus (Fig. 4) to a precision approximately given by their jitter (Butts et al. 2007). Within events, the number of spikes itself can be quite variable (Fig. 6), suggesting that the spike count on a given trial may simply be conveying an underlying estimate of firing rate at these finer time scales within the event. In this sense, the time scale of firing rate fluctuations relates to the degree that spike timing matters, the event structure described here identifies this relevant time scale where this equivalence occurs.
Because most time scales within events—encompassing measures ranging from jitter to overall event duration—are of similar magnitude and generally consistent across contrast, we have defined an overall response time scale for each stimulus condition (Fig. 7B). While the response time scale clearly tracks the overall trial-to-trial variability in absolute spike time (Desbordes et al. 2008), it is also in the same range as the underlying structural elements of the events (Fig. 7A). Response time scales are nearly identical between the two spatiotemporally varying stimuli (NAT and SBN), but they are significantly faster for the SUN stimulus condition (Fig. 5B). This mirrors the stimulus tuning of LGN neurons in these conditions: the spatiotemporal receptive fields measured in SBN and NAT have almost identical temporal structure, while the temporal receptive field of the SUN stimulus has much finer structure (Fig. 3), owing to the simultaneous activation of center and mostly canceling surround in this stimulus condition.
At the same time, the similar response time scale of SBN and NAT is at odds with the dramatically different time scales contained in these two stimuli: temporal correlations in the NAT condition lead to much longer time scales, both in the stimulus itself and the stimulus filtered by the receptive field (Butts et al. 2007). In the SUN and NAT conditions, response time scales are two to four times more precise than the stimulus filtered by the receptive field, implying the presence of nonlinear mechanisms that contribute to spiking output. Surprisingly, here we found that such precision is not maintained in the SBN condition, which has the same response time scales as NAT (Fig. 7B), but smaller filtered stimulus time scales compared with NAT. The lack of precision in the SBN condition also mirrors the apparently increased variability in spike count relative to the other conditions (Fig. 6A). Together, this implies that SBN does not drive the nonlinear mechanisms in LGN neurons that result in more reliable responses and precise time scales, and questions the use of spatiotemporal noise stimuli to draw general conclusions about properties of the neural code.
Information in the fine-time-scale structure of spike trains
While the larger between-event time scales of the LGN spike train are well predicted by receptive field filtering (e.g., Fig. 4), the finer time scale features of the event themselves are not directly predicted by the receptive field as demonstrated by the fact that the filtered stimulus is relatively constant across these much shorter time scales. The smaller time-scale event measures are instead likely related to more complex interactions between stimulus properties and properties of the neuron itself (Berry and Meister 1998; Desbordes et al. 2010), and we observed some direct relationships between event properties and the stimulus. For example, the number of spikes in the event is related to the overall magnitude of the filtered stimulus (Fig. 4E). Likewise, there is a strong relationship between the magnitude of the filtered stimulus and the ISI1, such that “stronger” stimuli have shorter ISIs associated with them (Mainen and Sejnowski 1995).
One can imagine that either of these event features could be decoded through simple biophysical mechanisms in the cortex. Temporal integration at the thalamocortical synapse will yield a graded response with spike count, whereas shorter time scale coincidence detection might result in a larger signal with smaller ISIs. It has been shown that LGN spikes that are close together in time have higher cortical efficacy (Alonso et al. 1996; Kara and Reid 2003; Usrey et al. 2000), suggesting the plausibility of cortical neurons being sensitive to such extra information.
Contrast gain control and the contrast invariant response
The changes of response properties of RGCs and LGN neurons with contrast have been extensively studied, and we expect that many of the properties of adaptation to contrast in the LGN are likely inherited from the retina. The most dominant aspect of contrast adaptation is the change in gain (termed “contrast gain control”), such that the neuron's firing rate is maintained at nearly the same level despite dramatic decreases in contrast (Meister and Berry 1999; Shapley and Victor 1978). It is often postulated that adaptation removes aspects of the visual image that are not useful in object identification (e.g., contrast), while preserving those that are useful. At the same time, other aspects of contrast adaptation do not fit easily into this picture: contrast changes also result in a change in latency and an apparent change in temporal sensitivity of the neuron (Chander and Chichilnisky 2001; Lesica et al. 2007; Shapley and Victor 1978; Zaghloul et al. 2005). We observed these contrast-dependent changes in receptive field properties (Supplementary Fig. S2), as well as analogs of these changes in changing event structure, in the form of increased jitter in the event timing (Fig. 7A) and systematic latency differences (Fig. 10B).
Our findings thus potentially unite these two apparently disparate perspectives on contrast adaptation. We found that the structure of spike trains on longer time scales was highly insensitive to contrast changes, such that events present in HC had a direct analog in LC. The relative invariance of event structure has also been observed in limited contexts in the retina (Meister and Berry 1999). In this sense, the overall tuning of the neuron appears to be relatively contrast invariant, and the observed changes in receptive field properties with contrast thus likely arise from changes of within-event properties.
We developed a “contrast-invariant” measure of the information carried in structural elements of events, which we could compare with the information carried in a contrast-dependent way. Our conclusions are mixed. On the one hand, using the same “neural code” (i.e., particular event properties such as duration or ISIl) irrespective of contrast can still convey >10% more information than the event timing alone (Fig. 11B), which suggests the presence of a contrast-invariant code. At the same time, this represents a ∼60% decrease in the amount of information that could be carried in these structures in a contrast-dependent way (Fig. 8C), suggesting that some elements of this code might not be contrast invariant. In either case, the event itself appears to be the unit of communication of LGN neurons as events signal in a contrast invariant manner and are generally robust across trials.
Events and the LGN neural code
The episodic structure of LGN spike trains suggests something very specific about the nature of the LGN neural code and how to interpret it. Rather than providing a continuous visual signal to cortex, LGN neurons mark specific times when a specific subset of visual stimuli is present in their receptive field. In this sense, an LGN neuron can only signal in time, and such timing is robust over the coarse between-event time scales (Fig. 4, B–D), which accounts for a majority of the information in the LGN spike train (Fig. 8C). Additional information that can further identify the particular stimulus being signaled is present on shorter time scales and may be represented by a number of different event elements such as spike count or ISI.
Because the LGN comprises the visual input into cortex, such observations have strong implications for what neurons in the input layers of visual cortex perform computation on and ultimately what they represent about the visual world.
This work was supported by National Science Foundation Collaborative Research in Computational Neuroscience (CRCNS) Grant IIS-0904630 to G. B. Stanley and J.-M. Alonso and the National Eye Institute Grant EY-005253 to J.-M. Alonso.
No conflicts of interest, financial or otherwise, are declared by the author(s).
↵1 The online version of this article contains supplemental data.
- Copyright © 2010 the American Physiological Society