Neurons are notoriously noisy devices. Although the traditional view posits that noise degrades system performance, recent evidence suggests that noise may instead enhance neural information processing under certain conditions. Here we report that random channel and synaptic noise improve the ability of a biologically realistic computational model of the Hermissenda eye to encode light intensity. The model was created in GENESIS and is based on a previous model used to examine effects of changes in type B photoreceptor excitability, synaptic strength, and network architecture. The network consists of two type A and three type B multicompartmental photoreceptors. Each compartment contains a population of Hodgkin–Huxley-type ion channels and each cell is stimulated via artificial light currents. We found that the addition of random channel and synaptic noise yielded a significant improvement in the accuracy of the network's encoding of light intensity across eight light levels spanning 3.5 log units (P < 0.001, modified Levene test). The benefits of noise remained after controlling for several consequences of randomness in the model. Additionally, improvements were not confined to perithreshold stimulus intensities. Finally, the effects of noise are not present in individual neurons, but rather are an emergent property of the synaptically connected network that is independent of stochastic resonance. These results suggest that noise plays a constructive role in neural information processing, a concept that could have important implications for understanding neural information processing or designing neural interface devices.
Neurons are notoriously noisy devices, and problems of random variation, noise, and reliability arise almost universally in the nervous system (Perkel and Bullock 1968). The traditional view, partially influenced by decades of signal processing research, is that noise lowers the signal-to-noise ratio (SNR) and thus degrades performance. Another traditional view, more relevant to neuroscience, is that noise reduces spike-timing precision and therefore lowers the rate of information transfer. If the traditional views are true, then decreasing noise in the nervous system should improve performance. However, biological systems perform quite well in the presence of noise, often easily outperforming their human-engineered counterparts. Here we present evidence for an alternate interpretation based on a suite of computational experiments: the nervous system uses randomness to its advantage such that noise paradoxically improves, rather than degrades, performance. Our experiments demonstrate such an effect in the Hermissenda photoreceptor network. The purpose of this study is to examine the ability of the eye to encode light intensity in the presence of random noise. A companion paper takes this analysis a step further to look for mechanisms of noise-induced improvement (Butson and Clark 2008).
The marine mollusk Hermissenda crassicornis has served as a prominent preparation for the investigation of cellular mechanisms of learning, particularly conditioned suppression of phototaxis (Alkon 1974; Crow 1983; Crow and Alkon 1978; Crow and Offenbach 1983; Farley and Alkon 1982; Lederhendler and Alkon 1987). Naïve Hermissenda instinctively locomote toward a light source. This positive phototactic response is suppressed after classical conditioning with a light-conditioned stimulus (CS) and a rotation unconditioned stimulus (US). Animals also exhibit a new conditioned response, foot shortening, that resembles the response to the US (Lederhendler et al. 1986; Matzel et al. 1990a). Nonassociative suppression of phototaxis also occurs (Alkon 1974; Crow 1983; Crow and Alkon 1978; Farley and Alkon 1980, 1982; Grover et al. 1987; Matzel et al. 1990b; Rogers et al. 1994). Increases in excitability (Crow and Alkon 1980; Farley and Alkon 1982; Goh and Alkon 1984; Goh et al. 1985) and synaptic strength of type B cells (Frysztak and Crow 1994; Gandhi and Matzel 2000; Schultz and Clark 1997; Schuman and Clark 1994) constitute an important neural mechanism for these forms of learning.
The eye of Hermissenda provides an advantageous preparation for the investigation of mechanisms underlying neural information processing, as well as information acquisition, storage, and retrieval (“learning” and “memory”). The large, identifiable neurons and relatively simple circuitry of this preparation permit the construction of biologically realistic computational models that facilitate quantitative analyses (Blackwell 2006; Fost and Clark 1996b,c; Mo and Blackwell 2003; Sakakibara 1989; Smith and Farley 2006; Werness et al. 1992). Each of the two Hermissenda eyes contains five cells interconnected in a stereotyped manner (Alkon 1984) (Fig. 1). Three type B cells form inhibitory synaptic contacts both with two type A cells as well as with the other type B cells. The type A cells form reciprocally inhibitory connections with the type B cells but do not communicate with each other. Both types of photoreceptors show large depolarizing generator potentials and vigorous spiking in response to light (Alkon and Fuortes 1972; Alkon and Grossman, 1978; Dennis 1967; Detwiler 1976). Several possible pathways between the photoreceptors and distal motor neurons have been identified; one commonly identified pathway is the excitatory connection between the type A cells and a pool of interneurons that drive the pedal musculature and are believed to mediate positive phototaxis (Crow and Tian 2003a,b, 2004; Goh and Alkon 1984; Goh et al. 1985). Thus one consequence of the architecture within the eye is that, via the B–A inhibitory pathway, the type B cells are capable of inhibiting type A cells and the motor response that type A cells control.
Although Hermissenda have historically been used as a system to study learning and memory, here we take advantage of the large knowledge base available for this preparation to examine its performance in the presence of ion channel and synaptic noise. Several groups have already begun to examine the effects of noise in other model (Chow and White 1996; Horikawa 1991; Liu et al. 2001; Schneidman et al. 1998; Steinmetz et al. 2000) and physiological (Bialek et al. 1991; Holt et al. 1996; Keeler et al. 1989) systems. Evidence from experimental, theoretical, and computational studies indicates that noise from voltage-gated ion channels can have important effects at the cellular level (Chialvo et al. 2000; Roddey et al. 2000; Schneidman et al. 1998) and that such noise plays an important, if poorly understood, role in the functioning of the nervous system (White et al. 1998). Additionally, the variability of input spike arrival times, which can result from synaptic noise, plays a much greater role in facilitating SNR increases than does intrinsic (channel) noise (Plesser and Gerstner 2000). In this paper we examine the ability of the Hermissenda eye to encode light intensity under varying levels of channel and synaptic noise and find that noise paradoxically improves, rather than degrades, the encoding of light intensity. In a companion paper we look more deeply at the mechanisms for noise-induced performance improvement, and demonstrate that this improvement arises from changes in contextual spike-timing relationships, rather than from stochastic resonance, DC-bias effects, or other possible mechanisms.
The large knowledge base available for the Hermissenda eye, in conjunction with the relative simplicity of the system itself, has made it possible for us to develop a biologically realistic computational model on a Hodgkin–Huxley level using empirically derived parameters. Our previously published computational studies investigating neural mechanisms of learning in Hermissenda used a custom program written explicitly for those investigations (Fost and Clark 1996b,c). This computational model has proven to be informative and has made several predictions regarding physiology that were initially controversial but have subsequently been confirmed, including the impacts of synaptic strengthening and input frequency on postsynaptic targets (Fost and Clark 1996a) and the role of the IA current in spike broadening and synaptic facilitation (Cai et al. 2006; Gandhi and Matzel 2000; Han et al. 2001). We have since ported the model to GENESIS (Bower and Beeman 1998; Butson and Clark 2001, 2002), which provides a simulation environment used by a larger community. Thus the use of GENESIS will facilitate dissemination, confirmation, and extension of our results, as well as dissemination of the model itself.
Simulations were performed using a multicompartmental model with Hodgkin–Huxley current-based descriptions (Hodgkin and Huxley 1952) of type B and type A photoreceptors (Fig. 1). Each cell has seven compartments and nine currents, including light-induced and fast sodium currents, so both types of model cells exhibit spiking responses to light stimulation. As in the original Hodgkin–Huxley formulation, each current has the general form Ik = gk × mi × hj × (V − Ek), where Ik is the current, gk is the peak conductance, mi and hj are the activation and inactivation state variables, respectively; V is the absolute voltage of the cell; Ek is the reversal potential for that current; and i and j are constants. A variety of methods have been proposed for parameter estimation in single-neuron and ion-channel models (Tabak et al. 2000; Vanier and Bower 1999; Willms et al. 1999). We chose to use biologically based parameters that were derived from voltage-clamp or other physiological experiments wherever possible. Equations for each current and sources for comparisons to physiology are given in Fost and Clark (1996b). Amplitudes of the generator potential in type B cells elicited by unattenuated light were 47 and 19.3 mV (peak and plateau, respectively). Calculations were performed using Crank–Nicholson implicit numerical integration with a time step size of 0.01 ms (∼one order of magnitude smaller than the smallest time constant in the neural parameters). This method is more than adequate for our stability and accuracy requirements, as confirmed by evaluation with other time steps and by comparison with previous results.
To make the network models more biologically realistic, we introduced various amounts of ionic current noise and synaptic noise into the simulations. To mimic channel noise, Gaussian noise was injected into each compartment of each cell at each time step in the form of an ionic current. Unless otherwise noted, the noisy condition contained an ionic current drawn from the N[0, 0.33 nA] distribution, where the noise variance was estimated from physiological recordings. During synaptic transmission, the quantal force parameter (Q) was calculated to determine the “quanta” released from the presynaptic cell to the postsynaptic cell, which in turn was used to calculate the postsynaptic current (Fost and Clark 1996b). Q was calculated once per spike and, in each instance, it was multiplied by a factor drawn from the N[1, 0.2] distribution. To create a set of models that could be compared in the presence and absence of noise, the membrane resistance of each cell in the network was multiplied by a scaling factor drawn from the N[1, 0.025] distribution. This step was repeated 11 times, yielding 11 model eyes. In previous work, this sample size proved sufficient to detect differences of about 0.1 Hz (Fost and Clark 1996c). One particular advantage of this approach is that it allows statistical comparisons of effects in the deterministic, noise-free condition (which would otherwise always be identical, by definition, and thus represent a sample size of 1). This approach also ensures that results are general, rather than idiosyncratic, to one particular set of parameters. Previous studies have shown evidence for (Alkon and Fuortes 1972) and importance of (Mar et al. 1999; Read and Siegel 1996; Werness et al. 1992) heterogeneity of individual cell responses to light stimuli.
Firing rates from the type A cells in the plateau region (the last 25 s of a 30-s simulated light step, except where explicitly noted) were used to measure output over eight light intensities spanning approximately 3.5 log units. Previous studies of Hermissenda have shown a log-linear increase in cell response with light intensity (Detwiler 1976). Additionally, Akaike and Alkon (1980) found that all five photoreceptors depolarize and increase their firing frequency in response to light when stimulated over a range of about 3.5 log units with a maximum intensity of 105 ergs·cm−1·s−1.
Eleven artificial eyes were subjected to eight light levels in noisy and noise-free conditions to determine whether noise, light, or noise × light had an effect on average type A cell firing frequencies using the general linear model (GLM) ANOVA (SPSS, Chicago, IL), where P values <0.05 indicate significance. Performance was evaluated using a modified Levene test on residuals. Specifically, the performance of each eye was determined by measuring the residuals between the actual performance of the system and the expected performance of an ideal decoder under varying levels of channel and synaptic noise. An example that steps through the analysis process is summarized in Fig. 2. Simulations were run for 30 s postlight, and the average firing frequency of the type A cells from the last 25 s of the simulation was recorded at eight light levels. These relatively long integration times follow from the 1- to 2-min behavioral response time of the animal. To find the optimal decoder for each individual eye, firing rate data were tested for significant curve fits of polynomial orders 1 to 7 (within-subject contrasts from GLM ANOVA). Of the statistically significant curves, the one with the lowest Akaike information criterion (AIC) was selected. AIC is calculated from (1) where n is the sample size, σ is the sum of squares of the residuals, and p is the number of model parameters (in this case p refers to the number of polynomial terms included). With this approach, we gave each individual eye the chance to find an optimal decoder, rather than enforcing one a priori. Once a decoder was determined, residuals between the actual firing rate and the optimal decoder were calculated for each eye and analyzed using a repeated-measures ANOVA for 8 light levels, 11 model eyes, and 2 conditions (noisy and noise-free). This performance measure was designed to take into account the known correlation between light intensity and firing frequency, although we considered other more general performance measures that examined the difference in mean frequency between light levels in relation to the variance at each light level (see discussion).
Spike times for each neuron were recorded using an action potential detector in GENESIS and stored in a Microsoft Access database. Database queries were designed to extract spike times for specific cells and windows of time for firing rate calculations.
Noise improves performance of light-intensity encoding
Our principal finding is that noise improves the performance of light-intensity encoding. For the 11 model networks, the average firing rates of the type A cells over the last 25 s of a 30-s light response are plotted versus log (light intensity) (Fig. 3A). ANOVA results indicate a significant difference for light (P < 0.001) and noise × light (P < 0.001), but no significant difference for noise level (P < 0.461). Most importantly, the noisy network outperformed the noise-free network: residuals were smaller and more evenly distributed in the noisy condition (Fig. 3B). Repeated-measures ANOVA on the residuals (modified Levene test) showed significant differences for noise (F = 37.127, P < 0.001), light (F = 5.427, P < 0.001), and noise × light (F = 7.012, P < 0.001), with smaller residuals in the noisy condition than in the noise-free condition. The residuals represent the differences (errors) between the eye's best-fit optimal decoder and the photoreceptor's actual firing frequencies for the various light intensities. Thus these results indicate that the response of the photoreceptor is more systematic and monotonic in the noisy condition, compared with the noise-free condition. Given that one role of this system is to determine light intensity, the network will make this measurement more accurately in the noisy condition. Thus we conclude that, for a wide range of stimulus intensities, the model Hermissenda photoreceptor network encodes light intensity more systematically in the noisy condition compared with the noise-free condition.
Encoding performance is modulated by noise amplitude
Previous investigators have observed an optimal noise intensity for signal-to-noise ratio (SNR) in neurons with Hodgkin–Huxley channels (Wang et al. 2000). In an attempt to see whether such an optimum existed in the Hermissenda photoreceptor, we characterized the sensitivity of network performance to noise type and amplitude. Although some noise appears beneficial, system performance could be modulated by noise amplitude and type. As a result, certain noise levels would provide greater benefit. To explore this, we varied channel and synaptic noise across a wide range of levels (channel noise varied from N[0, 0 nA] to N[0, 1 nA]; synaptic noise varied from N[1, 0] to N[1, 0.25]) as shown in Fig. 4. We found that there is a floor effect for both channel and synaptic noise. Some noise is good, but high levels of noise do not have any added benefit. Although either noise type can confer performance benefits at sufficient magnitudes, the residuals for channel noise are smaller than those for synaptic noise.
Noise enhances encoding accuracy in longer epochs
Having shown that noise improves performance, we next sought to characterize the conditions under which noise improvement occurs. To examine the effect of noise on neural response time (i.e., how quickly can the intensity of the light stimulus be determined?), we measured light encoding performance with different duration time epochs from 5 to 25 s postlight onset. We found that in the noisy condition the accuracy of light-intensity encoding was correlated with the length of the time by comparing the average residuals in the 5-, 10-, and 25-s light epochs (Fig. 5). In contrast, the noise-free condition did not improve, regardless of epoch length.
Noise-induced performance requires ongoing presence of noise
In a second experiment to characterize the conditions for noise-induced improvement, we examined the performance of the network as noise was turned on and off during the light response (Fig. 6) at the eight light levels explained earlier. In this case, we conducted an 80-s light-duration experiment during which noise was off for the first 30 s, then on for 25 s, then off for the final 25 s. Each epoch (5–30, 30–55, and 55–80 s) was analyzed independently to determine firing rates and residuals. There are two important results from this experiment. First, it confirmed our initial observation that network performance is modulated by the ongoing presence of noise. It is not sufficient to “seed” the network with noise and retain any lasting benefit when noise is turned off. Second, while analyzing these results we observed something intriguing about the cell firing patterns. In the absence of noise we discovered that the networks tended to adopt spike timing patterns where the type A cells become artificially synchronized, an effect that warrants further exploration (see discussion).
Noise-induced improvement is an emergent property of the photoreceptor network
In the absence of synaptic connections, the individual cells in the noise-free and noisy conditions perform similarly (Fig. 7), with a slight improvement in the noise-free condition compared with the noisy condition. In the single-cell case, the two conditions have relatively similar monotonic increases in firing rate with light intensity, and have relatively comparable residuals when compared with an ideal decoder. ANOVA results indicate significant differences in firing frequency for light (P < 0.001), noise (P < 0.005), and noise × light (P < 0.006); ANOVA results on the residuals indicate significant effects for noise (P < 0.002) but not light (P > 0.4) or noise × light (P > 0.9). Given that the mean residuals in the noise-free condition are lower, these results suggest that in the absence of synaptic connections, the individual cell performs slightly better in the noise-free condition, which is the exact opposite of the effect of noise in the synaptically connected network, for which noise improved encoding. However, this effect is small. Thus the ability of noise to improve encoding of light intensity is an emergent property of the inhibitory network arising from synaptic connections among its neurons, rather than a property of the individual neurons themselves.
Noise and neural systems
In this paper we demonstrate the paradoxical improvement that random noise exerts on the performance of the Hermissenda photoreceptor network. One important component of the modeling approach used in this paper is the nature of noise sources. Physiologically, intracellular recordings capture the sum of signal and noise, keeping in mind that the definitions of these two sources are not strictly defined. Herein we view channel noise as voltage deviations from the moving average value and synaptic noise as variability in IPSPs. Major noise sources in physiological recordings result from the stochastic nature of ion channel openings and closings, as well as changes in the magnitude of postsynaptic potentials during steady-state conditions. We attempted to mimic these noise sources by adding ionic noise currents on a per-compartment basis and by varying the magnitude of postsynaptic potentials. These sources have not been previously characterized in the Hermissenda photoreceptor network, but have been examined in similar systems. For example, in Hermissenda statocyst hair cells, noise variance is caused by mechanotransduction and shares a common origin with the generator potential (DeFelice and Alkon 1977; Grossman et al. 1979). They measured noise variance in the range 1e-8 to 12e-8 V for cells under relevant conditions, values that are somewhat larger than those that we measured from the photoreceptor cells. They also observed that removal of sodium from the extracellular bath drastically decreased noise variance, suggesting that sodium ion channels are a primary source of channel noise. Indeed, a small number of persistent Na+ channels can cause a relatively high coefficient of variation in induced current as measured in entorhinal cortex (White et al. 1998), where the effects of channel noise are insensitive to changes in the specific form of the noise: changing the threshold, bandwidth, or voltage dependence of the noise altered details but not the basic properties of the results. Consistent with these findings, channel stochasticity is likely to be a key player in setting neuron's firing patterns, and thus it should be incorporated in models that explore the firing variability and spike timing of cortical neurons (Schneidman et al. 1998). In our approach we mimic the microscopic effects of channel noise with macroscopic noise injection, an approach that has been shown to be relevant for the distribution of thresholds for generation of action potentials (Chow and White 1996; Steinmetz et al. 2000).
Synaptic noise has also been investigated. Noise can play a constructive role in sensory processing in neuronal systems, specifically that the SNR of a weak sinusoidal signal can be increased in the presence of Gaussian noise in a Hodgkin–Huxley neuronal model (Liu et al. 2001). This effect was present in single cells, but much more pronounced in a network of synaptically connected cells. In another set of experiments, the variability of input spike arrival times may play a much greater role in facilitating SNR increase than intrinsic noise (Plesser and Gerstner 2000).
There are several possible noise sources such as changes in intracellular or extracellular ion concentrations that are not captured by the model used herein, and their effects are not known. However, we observed that the paradoxical performance improvement in the Hermissenda photoreceptor network was not strongly dependent on the type (channel vs. synaptic) or amplitude of the noise. Rather, once noise levels reached a certain level as shown in Fig. 4, either type of noise was sufficient to confer performance improvement. Thus our results suggest that providing additional noise sources would not change the primary results.
Effects of performance measure
Several different measures of photoreceptor network performance are available besides the modified Levene test used in this study (see methods). Among them, mutual information has been attractive to some investigators because it requires no assumptions about the underlying model. This has proven to be very useful in certain situations where little is known about the encoding or decoding mechanisms. For example, information theoretic approaches have been used to accurately decode angular velocity of a moving bar in fly H1 neurons (Bialek et al. 1991). However, this is precisely why this measure is not useful for our experiments: mutual information places no bounds on the complexity of the underlying encoder and decoder. From a practical standpoint in Hermissenda, this means that the average type A cell firing rates could be randomly rearranged with respect to light level, yet the mutual information would not change. However, physiological experiments provide evidence for a rate code for light intensity: the higher the light intensity, the higher the firing frequency (Akaike and Alkon 1980; Detwiler 1976). To accurately measure performance, our metric had to take this systematic relationship into account. Our statistical tests use this relationship by comparing the actual firing rate to that of an optimal decoder that is chosen on a per-eye basis, which is a more biologically realistic approach to the problem than assumption-free methods like mutual information. Similar methods have been proposed to assess the performance of neural encoding models in the presence of noise (Roddey et al. 2000), using a theoretically optimal, signal-averaged encoder for the neural system rather than an optimal encoder for each case.
The design of our model is consistent with observations that the existence of repeatable spike patterns and the reliability of their timing change not only from neuron to neuron, but even for the same neuron under different circumstances (Cecchi et al. 2000). They also showed that neural output noise is dependent on different stimuli and that this effect is dependent on the network architecture. This is in contrast to assumptions about neurons as communication channels from information theory, where the noise is assumed to be independent of the signal.
Simple mechanisms cannot account for results
Before looking for detailed mechanisms for noise-induced improvement, we searched for trivial mechanisms that could explain these results. First, we considered stochastic resonance (SR) as a mechanism. SR is a phenomenon described using a variety of definitions with subtle distinctions. One common description is a system whose subthreshold dynamics exceed threshold in the presence of noise. However, SR is normally used to describe neural dynamics of perithreshold stimuli, which does not apply to the experiments presented in this study. Second, we examined the effects of DC bias on the cell firing rates. Due to the rectifying properties of ion channels, even zero-mean channel noise can cause the cell to depolarize (Fig. 8A). Indeed, we found that our Gaussian noise distribution of N[0, 0.33 nA] caused the average cell firing rate to increase about 0.1 Hz. To compensate for this DC bias, we added a small hyperpolarizing current such that the firing rates were matched to within 0.1 Hz (power = 0.9) between the noisy and noise-free condition. Even with this correction the noise-induced performance improvement persisted, indicating that noise-induced improvements in light-intensity encoding were not due to a DC bias. Last, we examined the effects of type A cell synchronization. In the absence of any biophysical difference between type A cells, the noise-free condition collapses to what is essentially a two-cell network (Fig. 8B). The two type A cells fire in unison, as do the three type B cells. The addition of cellular heterogeneity within each eye abolished this artificial synchronization but does not account for the differences between the noisy and noise-free conditions. Thus noise-induced improvements did not arise simply from temporal dispersion of spike firing times among the photoreceptors.
This observation led to a new set of experiments designed to look for mechanisms for noise-induced performance improvement. In particular, we began to examine the relationships among photoreceptor firing times in the Hermissenda eye. In many past theoretical studies, inhibitory inputs were considered to simply lower firing rates of postsynaptic neurons rather than affect precise firing times. However, Luk and Aihara (2000) showed that IPSPs can play a functional role in realizing synchronization of neuronal firing. They also showed that these effects are peculiar to the biologically realistic Hodgkin–Huxley neuron models. This and other evidence led us to look for contextual spike-timing codes. Briefly, contextual spike-timing codes are distinct from rate or labeled line codes in that they encode information through the firing of one neuron relative to others. Here we have demonstrated enhanced light-encoding performance of the Hermissenda photoreceptor network in the presence of noise, relative to the noise-free condition. Alternate explanations have been considered for this effect. However, the phenomenon has persisted after controlling for each alternate mechanism. In contrast to noise-induced effects observed during stochastic resonance, this effect occurs with suprathreshold stimuli.
We have shown that the effects of noise are an emergent property of the Hermissenda network. In the absence of synaptic connections, noisy and noise-free cells show comparable performance in encoding light intensity. The addition of either channel or synaptic noise (or both) in a synaptically connected eye drastically improves performance: there is a closer relationship between light intensity and photoreceptor firing rate, allowing light intensity to be inferred more accurately. Synaptic connections also confer other advantages, such as network interactions contributing to learning and memory (Alkon 1974; Crow 1983; Crow and Alkon 1978; Crow and Offenbach 1983; Farley and Alkon 1982; Frysztak and Crow 1994; Lederhendler and Alkon 1987; Schuman and Clark 1994). Having eliminated several trivial explanations for the noise-induced improvements, we now turn to a set of hypotheses on mechanisms for noise-induced improvement that are explored in a companion paper (Butson and Clark 2008). We specifically examine the effects of contextual spike timing and its implications as a mechanism for noise-induced improvement.
This work was supported by the Whitaker Foundation and National Institute of Mental Health Grant R01-MH-068392.
We thank C. Johnson, G. Jones, J. Wiskin, R. Normann, and D. Beeman for comments on an earlier version of this manuscript.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2008 by the American Physiological Society