Abstract
Many phenomenological models of the responses of simple cells in primary visual cortex have concluded that a cell's firing rate should be given by its input raised to a power greater than one. This is known as an expansive powerlaw nonlinearity. However, intracellular recordings have shown that a different nonlinearity, a linearthreshold function, appears to give a good prediction of firing rate from a cell's lowpassfiltered voltage response. Using a model based on a linearthreshold function, Anderson et al. showed that voltage noise was critical to converting voltage responses with contrastinvariant orientation tuning into spiking responses with contrastinvariant tuning. We present two separate results clarifying the connection between noisesmoothed linearthreshold functions and powerlaw nonlinearities. First, we prove analytically that a powerlaw nonlinearity is the only inputoutput function that converts contrastinvariant input tuning into contrastinvariant spike tuning. Second, we examine simulations of a simple model that assumes instantaneous spike rate is given by a linearthreshold function of voltage and voltage responses include significant noise. We show that the resulting average spike rate is well described by an expansive power law of the average voltage (averaged over multiple trials), provided that average voltage remains less than about 1.5 SDs of the noise above threshold. Finally, we use this model to show that the noise levels recorded by Anderson et al. are consistent with the degree to which the orientation tuning of spiking responses is more sharply tuned relative to the orientation tuning of voltage responses. Thus neuronal noise can robustly generate powerlaw inputoutput functions of the form frequently postulated for simple cells.
INTRODUCTION
Responses of visual cortical simple cells are commonly described by simple phenomenological models in which a linear filtering of the stimulus is followed by an expansive powerlaw nonlinearity to determine an instantaneous firing rate (Albrecht and Geisler 1991; Albrecht and Hamilton 1982; Anzai et al. 1999;Carandini et al. 1997, 1999; Emerson et al. 1989; Gardner et al. 1999; Heeger 1992,1993; Murthy et al. 1998; Sclar et al. 1990). By a powerlaw nonlinearity, we mean that a cell's firing rate r depends on its input or voltage Vas r =k([V]^{+})^{n}for constants k and n, where the 0 voltage is set equal to the mean voltage at rest and [V]^{+} = max (V,0).1 By an expansive nonlinearity, we mean that n > 1. The linear filtering aspect of this model receives support from a number of studies showing that voltage responses of simple cells are remarkably linear functions of the visual stimulus (Anderson et al. 2000; Jagadeesh et al. 1993, 1997;Lampl et al. 2001) (but seediscussion).
Despite the success of using an expansive power law nonlinearity in phenomenological models, direct experimental investigations have shown that the transformation from instantaneous voltage to instantaneous spike rate is well approximated by a linearthreshold function r = k([V −T]^{+}), where V is the voltage after removal of spikes and lowpass filtering, r is the lowpassfiltered spike train, and T is an effective spike threshold (Anderson et al. 2000; Carandini and Ferster 2000). In this paper, we show a simple and surprisingly robust connection between linearthreshold models and expansive powerlaw nonlinearities: if the voltage trace includes significant stimulusindependent noise and if the conversion from instantaneous voltage to instantaneous firing rate is a linear threshold function, then the conversion from trialaveraged voltage to trialaveraged firing rate will be well described by an expansive power law (cf. Suarez and Koch 1989).
This work is inspired by the recent results of Anderson et al. (2000). They studied the intracellular basis for the observation that orientation tuning of visual cortical neurons is contrast invariant, i.e., changing stimulus contrast simply scales the magnitude of a neuron's response, without changing the shape of its orientation tuning curve (Sclar and Freeman 1982;Skottun et al. 1987). Anderson et al. (2000)'s results can be separated into three main findings. First, they found that a cell's trialaveraged voltage responseV̄ showed contrastinvariant orientation tuning (both the size of the mean voltage response and the amplitude of voltage modulation had this property). Second, they found that spiking responses also showed contrastinvariant tuning (as expected) and that a given neuron's spiking response was more narrowly tuned for orientation than its intracellular voltage response. Finally, they used computer simulations to demonstrate that a linear threshold model for converting instantaneous voltage to instantaneous spike rate could account for this data, but that the noise in the actual voltage traces was critical for this result: if a linear threshold model was applied to the actual noisy voltage traces, or to the trialaveraged voltage traces with the addition of random noise, then contrastinvariant spiking tuning was attained; but if a linear threshold model was simply applied to the trialaveraged voltage traces without additional noise, the spiking tuning became contrast dependent.
Here we present two basic results that serve to clarify the connection between the work of Anderson et al. (2000) and the many successful phenomenological descriptions of simple cell responses that assume linear input plus an expansive power law nonlinearity. First, while others have noted that a power law nonlinearity converts contrastinvariant input into contrastinvariant output (e.g.,Carandini et al. 1997; Heeger 1992;Heeger et al. 1996), here we prove that a power law is the only function that achieves this. This result, in combination with the results of Anderson et al. (2000), implies that adding noise to a linear threshold functionmust yield power law behavior to a good approximation. Second, we quantify the degree to which a noisesmoothed linear threshold function can indeed be approximated by a power law, finding that the approximation holds over a wide range of parameters. We also show that the exponent in the bestfit power law decreases with increasing noise level and that the sizes of signal and noise measured by Anderson et al. (2000) predict an exponent that accounts well for the observed sharpening of spiking orientation tuning relative to voltage tuning.
RESULTS
Contrastinvariant tuning
A response function is contrast invariant if changes in stimulus contrast simply scale responses without affecting the tuning to other parameters. For example, let V(c, θ) describe the voltage response of a given neuron as a function of contrast levelc and orientation θ. Letg_{V} (θ) be the orientation tuning function at maximal contrast. If the cell displays contrastinvariant orientation tuning, then changing to a different contrast csimply scales the response, i.e., V(c, θ) =f_{V} (c)g_{V} (θ), where f_{V} (c) is the cell's contrast response function (Fig.1). Therefore saying that orientation tuning is contrast invariant is equivalent to saying that the contrast response function is orientation invariant.
We will work in units in which V = 0 represents the mean voltage at rest, in the absence of a stimulus, so thatV represents the stimulusinduced voltage. It is easy to see that if the voltage response is contrast invariant, and the stimulusinduced spike rate R(V) (i.e., the spike rate after subtracting off the background rate) is equal to the stimulusinduced voltage raised to a power n, then this spike rate is contrast invariant:
Only a pure power law yields contrast invariant responses. Power law functions with nonzero threshold, i.e., r(V) =k([V −T]^{+})^{n} forT > 0, do not yield contrastinvariant responses, instead they typically lead to an “iceberg” effect—tuning widens with increasing contrast as more orientations receive suprathreshold input.
Accuracy of powerlaw approximation
Now we turn to the question of whether noisesmoothed threshold linear functions can be approximated by powerlaw nonlinearities. We let the instantaneous spike rate r be a threshold linear function of voltage V:r_{T}
(V) =k[V −T]^{+}, where T > 0 is a threshold and again V = 0 represents the mean voltage at rest. We assume that the voltage can be written as a sum of a trialaveraged voltage V̄ and zeromean Gaussian noise υ with SD ς: V = V̄ + υ, where (letting an overbar represent an average over trials) ῡ = 0 and
Figure 2 B, top, shows R̄_{T} versusV̄ as continuous lines for a range of values of threshold T. To determine how well this function might be approximated by a power law, for each T, we found the bestfit power lawkV̄^{n} (leastmeansquares fit;shown as dashed lines) over the range 0 ≤ V̄ ≤ T +V̄ _{hi} (upper limit shown as vertical dotted line segments). We illustrate, and initially consider, the case V̄ _{hi} = 1.5. The power law gives an excellent fit to V̄ over the fitted range for all values of T. Fig. 2 B, bottom,shows the same fits on a loglog plot. This shows that the power law fails for small inputs. However, these small inputs correspond to very low output rates (R̄_{T} = 0.1 shown as thin horizontal lines) that occur only for values of V̄well below threshold and result in negligible absolute differences between actual and fitted functions.
The bestfit exponent [n in the power lawR̄_{T} ≈k([V̄]^{+})^{n}] is always greater than one and increases with increasing thresholdT (Fig. 3). Intuitively, a larger exponent means that the responseV̄^{n} remains small for larger values of V̄, consistent with a higher threshold (cf. Carandini et al. 1997). Note that because the threshold T is expressed in units of the noise, increasingT for fixed noise is equivalent to decreasing the noise for fixed T. Thus the exponent is expected to be a decreasing function of noise level, i.e., higher noise leads to lower exponents (Anderson et al. 2000).
We quantified the robustness of the power law approximation toR̄ _{T}(V̄) in two ways. First, for each value of the threshold T, we calculated the size of the error at a given voltage V̄relative to the size of the responseR̄_{T} (V̄), i.e., we plot ‖R̄_{T} (V̄) − kV̄^{n} ‖/R̄_{T} (V̄) for a range of values of V̄ and T(Fig. 4). (Recall that kand n are determined from optimizing the fit for 0 ≤V̄ ≤ T + 1.5). Not surprisingly, the power law breaks down for large V̄.In this range, all values of V̄ + υ are above threshold, and the inputoutput function becomes the underlying linear threshold function. For low thresholds, the best fit powerlaw is more nearly linear (exponents near 1.0, see Fig. 3) and a good fit extends well beyond T + 1.5 (T + 1.5 is indicated by the upper dashed line in Fig. 4). For T > 2, however, accurate powerlaw fits do not extend much beyond this upper bound of the fitting range. The power law also breaks down for small values ofV̄. This indicates that the powerlaw fit does not capture the exact shape of the transfer function as it bends away fromR̄_{T} = 0 (see Fig. 2 B). However, errors at small V̄ correspond to very low firing rates (R̄_{T} <0.1 shown as lower dashed line in Fig. 4). At such low rates, these large relative errors reflect small absolute differences betweenR̄_{T} (V̄) and the bestfit power law (see Fig. 2 B, top).
To determine the range of voltages over which the powerlaw can give a good fit, we varied V _{hi}, the upper voltage cutoff of the range of fit [0, T +V _{hi}]. For eachV _{hi}, we calculated the average absolute error of the approximation (Fig.5 A), and the average error relative to the response (Fig. 5 B), for integral values of threshold T from 1 to 5. As T increases beyond 2 SDs of the noise, good powerlaw fits are only obtained for ranges that extend about 1.5 SDs above threshold.
In summary, both methods of assessing the accuracy of power law fits reveal that, across a wide range of thresholds, a power law gives a good fit in the range [0, T + 1.5].
Comparison to experimental data
We can compare these results to data as follows. The noise in the recordings of Anderson et al. (2000) was generally ς = 3–4 mV (rms). Their thresholds were roughly 10 mV from rest, yielding T = 2.5–3.3 (expressed in units of the noise). Stimulusinduced voltage changes (DC + F1) at the highest contrast studied at the preferred orientation were in the range of 8–12 mV, yielding V̄ = 2–4 (again in units of the noise) or no more than 1.5 above T. (Only contrasts ≤64% were studied, but responses of most cat V1 cells are nearly or entirely saturated at that contrast, e.g., Albrecht 1995.) Thus they found that even the strongest visual cortical voltage responses remain in the range in which a linear threshold function yields a power law as the averaged inputoutput function.
A power law with exponent n is expected to sharpen tuning by a factor of √n. This is based on consideration of a Gaussian tuning curve: if such a curve has SD ς, raising it to the power n produces a curve with SD ς/√n. Thus we can compare independent estimates of the exponent n, one obtained from measures of sharpening of tuning, and the other obtained by using the relationship of noise levels to exponents shown in Fig. 3. The range T = 2.5–3.3 obtained from the recordings ofAnderson et al. (2000) yields exponentsn = 2.9–3.7 (Fig. 3), corresponding to a sharpening of tuning by factors of √n = 1.7–1.9. Carandini and Ferster (2000) found that voltage orientation tuning had a halfwidthhalfheight (HWHH) of 38 ± 15° (mean ± SD, averaged over cells), while spiking orientation tuning had a HWHH of about 23 ± 8° [see also Volgushev et al. (2000), who also found spike tuning to be sharper than tuning of intracellular potentials]; however, their spiking tuning estimate was almost certainly overly broad, because their methods did not allow resolution of spiking HWHHs <20°. These mean values represent a sharpening by a factor of 1.65, the square of which suggests an exponent n = 2.72, which is attained in our model whenT = 2.3. Given that this is almost certainly an underestimate of the true sharpening, this agrees well with the estimate n = 2.9–3.7. Gardner et al. (1999) examined the same issue using extracellular recording by comparing the tuning predicted from a cell's noisemapped linear receptive field to that observed in response to gratings; they found sharpening corresponding to powerlaw exponents that had a geometric mean across cells of 3.15. Under the assumption that the linear receptive field approximates the transformation of stimuli into membrane voltage, this degree of sharpening agrees well with the noisebased estimate.
DISCUSSION
Here we have shown two things. First, a power law is the only inputoutput function that converts contrastinvariant voltage tuning into contrastinvariant spiking tuning. Second, given a linearthreshold function relating instantaneous voltage to instantaneous firing rate, addition of Gaussian noise to the voltage yields a relationship between trialaveraged voltage and trialaveraged firing rate that is well approximated by an expansive power law. This approximation seems quite good provided that the trialaveraged voltage does not exceed the threshold by more than about 1.5 SDs of the noise. We have gone on to compare these results to existing data. The voltage responses reported by Anderson et al. (2000) remained within the range in which a power law gives a good approximation, even at high contrasts. The exponents predicted from their reported noise and threshold levels predict a sharpening of spiking tuning, relative to voltage tuning, that agrees well with published data on the degree of such sharpening (Carandini and Ferster 2000;Gardner et al. 1999).
Mechanisms yielding contrastinvariant tuning
Anderson et al. (2000) used numerical simulations to demonstrate that neural noise and a thresholdlinear transfer function could transform contrastinvariant voltage responses into contrastinvariant spike responses. Our results, both theoretical (appendix ) and computational (Figs. 2 and 4), indicate that the invariance of spike tuning was due to the fact that a noisesmoothed thresholdlinear function is well approximated by a power law. Thus the approach of Anderson et al. (2000)to contrastinvariant orientation tuning, based on noisesmoothed linear threshold models, resembles phenomenological descriptions in which a linear filtering of the stimulus is followed by an expansive power law nonlinearity (Carandini et al. 1997;Heeger 1992; Heeger et al. 1996). It would be interesting to see if similar mechanisms explain contrastinvariance of other response properties, such as spatial frequency tuning (Albrecht and Hamilton 1982).
However, a noiseinduced powerlaw nonlinearity only explains half the problem of contrastinvariant tuning of V1 simple cells, namely how voltage tuning that scales with contrast is converted into spiking tuning that scales with contrast. The mechanisms by which the input to simple cells from neurons in the lateral geniculate nucleus (LGN) is converted into voltage tuning that scales with contrast remain to be elucidated. The voltage responses do not result from a simple linear filtering of the input as postulated by the phenomenological models. If the voltage resulted from a linear filtering of the drifting sinusoidal grating stimulus, then the mean voltage would be independent of orientation and contrast because changes in these parameters do not change mean luminance. The experiments of Anderson et al. (2000) found, instead, that the mean voltage responses to drifting sinusoidal grating stimuli were welltuned for orientation and grew with contrast. From a more neural point of view, one might expect the voltage response of a simple cell to arise from a linear filtering of the firing rates of the LGN neurons that are presynaptic to that cell. Because LGN firing rates have a rectification nonlinearity—their firing rates can greatly increase but can decrease only to zero—their mean rate of firing increases with contrast. However, the output of such a filter should again be untuned for orientation because the mean responses of LGN cells are largely untuned for orientation, and the mean voltage response under a linear filtering of the LGN would be obtained by a weighted sum of the mean responses of each LGN input (Ferster and Miller 2000; Troyer et al. 1998). Thus the observed voltage responses do not arise from a linear filtering either of the stimulus or of the LGN firing rates.
This raises the question as to how this mean LGN input, which is untuned for orientation and grows with contrast, is converted into a tuned mean voltage response. We have suggested that feedforward inhibition (inhibition from interneurons driven by LGN input) can suppress the untuned mean input from the LGN and hence explain the lack of contrastdependent responses to stimuli oriented perpendicular to the preferred orientation (Ferster and Miller 2000;Miller et al. 2001; Troyer et al. 1998; see also McLaughlin et al. 2000; Wielaard et al. 2001, who propose a similar feedforward inhibitory mechanism but in the context of a somewhat different circuit). The increases in mean voltage for preferred stimuli may arise through the interaction between cellular or circuit nonlinearities and the large voltage modulations experienced at these orientations: both the reversal potential nonlinearity and intracortical excitation from other cells with a threshold nonlinearity will cause stimuli that yield larger voltage modulations to be accompanied by larger voltage mean responses. That is, the tuning of the voltage mean may largely be inherited from the tuning of the voltage modulation. More generally, the results ofAnderson et al. (2000) point away from the nonlinearities involved in converting intracellular voltages to spikes (e.g., the threshold nonlinearity) as being the key issue related to contrastinvariant tuning, and toward an investigation of mechanisms that contribute to the tuning properties of the intracellular voltage.
Comparison to previous theoretical work
It is widely known that noise can smooth and in certain respects linearize a threshold nonlinearity (e.g., Knight 1972; Spekreijse 1969;Stemmler 1996), making otherwise subthreshold inputs become “visible.” However, the present work is showing something far more specific, namely that the specific smooth function that results closely approximates a power law. Furthermore it is key that it is the voltage deviations from background that are raised to a power [i.e.,R̄_{T} (V̄) =k([V̄]^{+})^{n}]. Had the form of the output function instead been, say,k([V̄ −T]^{+})^{n} forT >0, this would not yield contrastinvariant tuning—more and more of the input would be suprathreshold at higher contrasts. Because the exact equation for R̄_{T} (Eqs. 2 and EB1 ) depends on V̄only through its dependence on V̄ − T,it is surprising that this is well approximated bykV̄^{n} over a significant range—we know of no simple analytic reason why this empirical finding should be true.
We are aware of two other works that relate thresholdlinear functions and power laws. First, Carandini et al. (1997) showed graphically that a power law can roughly approximate a threshold linear function with higher thresholds corresponding to larger exponents. This suggests that for some response properties, models based on linear threshold and power law nonlinearities may yield similar predictions. However, contrastinvariant orientation tuning requires that where responses are significantly larger than zero, the ratio of responses between various orientations must remain constant at different levels of contrast. Even though absolute differences between a power law and an unsmoothed linear threshold function might be moderate, the relative error between the functions is very large near threshold, and as a result linear threshold models do not yield contrastinvariant tuning (Anderson et al. 2000). The key point we are making is not simply that a linear threshold function resembles a power law but rather that noise converts a linear threshold function into a different function that approximates a power law sufficiently closely to achieve contrastinvariant tuning. Second, in a finding close in spirit to the present work, Suarez and Koch (1989) showed that, given a linear threshold model, adding noise to the input that is uniformly distributed over some range (or, having a population of cells receiving identical input but with a uniform distribution of thresholds) acts like taking the integral of the linear threshold function, yielding a quadratic inputoutput function. However, the argument is not robust. It only yields a power law of the formk([V̄]^{+})^{n}[rather than k([V̄ −T]^{+})^{n}] when the upper bound of the noise (at rest) is equal to spike threshold and can only yield power law exponents exactly equal to 2.
Conclusions
Neural noise can convert an instantaneous linearthreshold inputoutput function into a powerlaw relationship between mean input and mean output. Given reports suggesting that spontaneous voltage fluctuations (“noise”) in neocortex in vivo are large and of comparable size to stimulusinduced voltage modulations (Arieli et al. 1996; Azouz and Gray 1999; Ho and Destexhe 2000; Paré et al. 1998;Tsodyks et al. 1999), it will be of great interest to determine if the response properties of cells in other regions of the neocortex are best modeled by an expansive power law nonlinearity. In particular, it will be interesting to see if such a power law might be related more generally to tuning for stimulus form that is invariant to changes in stimulus magnitude.
Acknowledgments
We thank B. Bialek, J. Anderson, and D. Ferster for useful conversations.
We gratefully acknowledge support by National Eye Institute Grant EY11001 (K. D. Miller).
After this work was completed, we became aware that the independent work of D. Hansel and C. van Vreeswijk reached similar conclusions.
Footnotes

Address for reprint requests: K. D. Miller, Dept. of Physiology, UCSF, San Francisco, CA 941430444 (Email:ken{at}phy.ucsf.edu).

↵1 We use the rectified voltage [V]^{+} in our definition of a power law to ensure that responses are an increasing function of voltage, i.e., we assume that hyperpolarization cannot increase response.
 Copyright © 2002 The American Physiological Society
Appendix
Here we show that a rectified power lawR̄(V̄) =k([V̄]^{+})^{n}is the only static inputoutput function that converts contrastinvariant input tuning into contrastinvariant output tuning, assuming that R̄(V̄) is nondecreasing (an increase in voltage cannot give a decrease in response) and nonnegative. The more general expression without the latter assumptions is also a form of power law.
We let c be contrast, θ be the other parameters (such as orientation) that show contrastinvariant tuning, V̄ be the input and R̄(V̄) be the output. Contrastinvariant input tuning implies that
Differentiating both sides of Eq. EA2
with respect tof and g yields
We obtained these results on the assumption that f, F, g,and G were all nonzero. Combining these results with the continuity of G and F, however, we can conclude that 1) G(0) = 0 and F(0) = 0 and 2) F and G are either strictly zero or strictly nonzero (and equal to a power law) on each open halfinfinite interval (−∞, 0) and (0, ∞). Finally, since we have assumed that f is nonnegative andR̄(V̄) is nondecreasing and nonnegative, G(g) = 0 for g< 0. Thus, over the full range of V̄
Appendix
Let the voltage V = V̄ + υ where overbar represents an average, ῡ = 0 and