Schematic illustration of functional models for early visual cortical neurons. A: functional model for simple cells. Response of a simple cell is described by a linear filter whose output is half-wave rectified and squared. B: minimalist energy model for complex cells, which respond to the visual stimuli of both contrast polarities in the receptive fields. Response invariance to contrast polarity is parsimoniously explained by a quadrature pair (i.e., different in spatial phase by 90°) of linear filters whose outputs are squared and summed. This operation calculates the energy of the preferred spatial frequency components of visual scene in the receptive fields. Because the number of linear filters is minimized in the parsimonious energy model, the receptive field of a complex cell has the same spatial extent as that of the preceding linear filters. In model C, however, spatial pooling and energy computation are important components to generate the receptive fields of complex cells. In this case, the receptive field of a complex cell is substantially larger than that of the individual linear filters.
Analysis of second-order interaction to uncover the internal spatial structure of receptive fields for complex cells. A: a spike train was recorded during the presentation of a sequence of 2-dimensional (2D) dynamic noise stimuli. B: spike-triggered stimuli were picked up for a particular correlation delay for each spike. C and D: second-order interaction was analyzed as follows. Reference was fixed at a particular position, and the local stimulus patch was selected for the second-order interaction analysis (C). For a dark reference, the contrast of all dots in the patch was reversed (D, left). For a bright reference, the contrast of all dots in the patch was retained (D, center). In the case of a gray reference, the contrast of all dots in the patch was nullified (D, right). These operations correspond to multiplication by −1, 1, and 0 for dark, bright, and gray reference stimuli, respectively. E: second-order interaction map for the reference location was obtained by accumulating all spike-triggered local stimulus patches that were modified on the basis of the contrast polarity of reference stimuli. Second-order interaction at the reference location cannot be measured, but is required for subsequent analyses such as Fourier transform. Thus interaction strength at the reference location was interpolated from neighboring values (spline interpolation). F: partial Hilbert transform (HT) of E was obtained by Fourier analysis. G: envelope of the second-order interaction map was computed as the root-mean-square sum of E and F. Second-order interaction maps and their envelopes were obtained for all reference locations in visual stimuli. H: overall receptive field of the complex cell was obtained by taking sum of E2 + F2 of all reference locations and then computing its square root. (·)2 denotes a squaring operation for each matrix element. (·)1/2 indicates taking square root for each matrix element.
Definition of length and width for the subunits and receptive fields of complex cells. First, the bounds of second-order interaction map envelopes (gray dashed ellipse enclosing subregions) and receptive fields (black solid ellipse) were defined by the level that was 5% of the peak of the 2D Gaussian function fitted to them. Then, the length and the width were evaluated for the elliptic region along the parallel and orthogonal axes to the optimal orientation, respectively. Note that the length and the width are independent of the axis of the elliptic elongation for subunit envelopes and receptive fields.
Representative example of the second-order interaction analysis for a complex cell. A second-order interaction map indicates contributions to responses of a second dot placed near the reference stimulus. This may be calculated from responses to dynamic dense noise stimuli (see methods for details). A: matrix of second-order interaction maps obtained by selecting a reference position within the stimulated area at 2-dot intervals, and arranged according to the relative order of reference positions. Each map consists of 21 × 21 pixels, and the position of the reference stimulus is at the center. Bright pixels represent excitatory responses to stimuli with the same contrast polarity as stimulus at reference (bright–bright or dark–dark), and dark pixels indicate excitatory responses to stimuli with the opposite contrast polarity to stimulus at reference (bright–dark or dark–bright). Interaction map in the center of A, outlined by a solid frame, had the strongest interactions among the maps for all reference locations. This interaction map covered a portion of the stimulus area shown by the solid square in B. (Contrast of noise stimuli in this illustration is reduced for clarity.) Position of the top left interaction map in A is also shown by the dashed square in B. A scale bar in B corresponds to 5°. C: illustration of the bounds of interactions (dashed white ellipse) for the central interaction map in A (solid frame). Solid white ellipse indicates the bounds of the complex cell receptive field determined from F. D: partial Hilbert transform of the interaction map shown in C. C and D form a quadrature pair and were used to obtain the envelope of the interaction map. E: envelope of the interaction map shown in C. F: overall receptive field for the complex cell. Although the map in F is defined over the area covered by the dynamic noise stimuli, it is truncated to the same spatial extent as that of C, D, and E for ease of comparison. Center of the receptive field is defined as the origin in F, and also corresponds to the origin in C, D, and E. Correlation delay was 53 ms. Optimal spatial frequency, orientation, and areal pooling ratio were 0.25 cycles/deg, 72°, and 1.08, respectively.
Another example of the second-order interaction analysis. Format is identical to that of Fig. 4; see Fig. 4 legend. For this complex cell, the strongest second-order interaction was observed at a correlation delay of 39 ms. Optimal spatial frequency and orientation were 0.53 cycles/deg and 22°, respectively. Areal pooling ratio was 1.69, which was significantly >1 (P < 0.05, resampling).
Another example of the second-order interaction analysis. Format is basically the same as that of Fig. 4; see Fig. 4 legend. However, the quadrature counterpart of the interaction map shown in C is omitted for this example cell because the orientation tuning of the subunit is not sufficiently strong to define the axis of the partial Hilbert transform. D: envelope of the interaction map shown in C. E: overall receptive field. Strongest second-order interaction was observed at a correlation delay of 39 ms. Optimal spatial frequency and orientation were 0.86 cycles/deg and 14°, respectively. Areal pooling ratio was 4.73, which was significantly >1 (P < 0.05, resampling).
Examples of the second-order interaction analysis for 5 additional complex cells. For each analysis, the left panel shows the second-order interaction map that exhibited the strongest interactions, the center panel shows its envelope, and the right panel shows the envelope of the receptive field. Dashed ellipse represents the bounds of the interactions; solid ellipse delineates those of the receptive field. Origin of the receptive field map corresponds to its center. Optimal correlation delay, spatial frequency (SF), orientation (OR), and areal pooling ratio (PR; *P < 0.05, different from one, resampling) are shown to the right of the receptive field for each analysis. E and F: responses of a single complex cell measured for the right and left eye, respectively.
Relationships between length and width. A: complex cell subunits. B: complex cell receptive fields. C: simple cell receptive fields. In the scatterplots, the width is plotted along the horizontal axis, and the length is plotted along the vertical axis. Diagonal line of unity slope indicates a perfect correspondence between the 2 values. Histograms depict the distributions of the length (right) and the width (top). Arrows in the histograms indicate the median values for each distribution.
Distribution of number of subregions. A: within complex cell subunits. B: within simple cell receptive fields. Number of subregions was computed from the width and optimal spatial frequency parameters as described in Number of subregions. Arrows indicate the medians of each distribution.
Comparison of length-to-width ratios. A: comparison of the length-to-width ratios on a log–log axis between the subunits and the receptive fields of complex cells. Length-to-width ratios of subunits are plotted along the horizontal axis, and the length-to-width ratios of receptive fields are plotted along the vertical axis. Histograms depict the distributions of the length-to-width ratios of subunits (top) and receptive fields (right). B: distribution of the length-to-width ratios for simple cell receptive fields. Arrows in the histograms indicate the medians for each distribution.
Comparison of aspect ratios. A: comparison of the aspect ratios between the subunit envelopes and the receptive fields. Aspect ratios for the subunit envelopes are plotted along the horizontal axis; those for the receptive fields are plotted along the vertical axis. Comparison is depicted on a log–log coordinate. Histograms illustrate the distributions of the aspect ratios for the subunit envelopes (top) and the receptive fields (right). B: distribution of the aspect ratios for the receptive field envelopes of simple cells. Arrows in the histograms indicate the medians for each distribution.
Analysis of anisotropic elongation for the subunit envelopes and the receptive fields of complex cells. Orientation of elongation was measured from the absolute horizontal axis, and clockwise and counterclockwise rotations were not discriminated. Left: results of analysis for the subunit envelopes. Right: results of analysis for the receptive fields of complex cells. In the scatterplots, the aspect ratios are plotted against the orientation of the major axis. Filled symbols denote data from cells for which subunit envelopes and receptive fields were sufficiently elliptic (aspect ratio >1.2; n = 66 for subunit envelopes in the left; n = 56 for receptive fields in the right). Open symbols are used for complex cells with nearly circular (aspect ratio ≤1.2) subunit envelopes (n = 20) and receptive fields (n = 30). Histograms depict the distributions of the orientation of the major axis for complex cells with sufficiently elliptic subunit envelopes and receptive fields. Criterion ratio is denoted by a solid horizontal line in the scatterplots.
Analysis is shown of possible relationships between the axis of receptive field elongation and the optimal orientation for simple cells. Scatterplot compares the optimal orientation and the orientation of the major axis of receptive field envelopes for simple cells. Angle was gauged counterclockwise from the horizontal axis (horizontal orientation and elongation correspond to 0°, whereas vertical correspond to 90°). In the scatterplot, filled symbols denote data from cells for which receptive field envelopes were sufficiently elliptic (aspect ratio >1.2; n = 114); open symbols represent data from neurons of which receptive field envelopes were not elongated, i.e., nearly circular (aspect ratio ≤1.2; n = 38). Histograms depict the distributions of optimal orientation (right) and the orientation of the major axis for the receptive field envelopes (top). Only cells with sufficiently elliptic receptive field envelopes were counted for the histogram at top.
Relationship with the length-to-width ratio is examined for the optimal orientation and spatial frequency for simple cells. A: length-to-width ratio against the optimal orientation as a scatterplot. Circles, triangles, and square symbols indicate neurons which prefer low (<0.25 cpd), middle (≥0.25 cpd and <0.5 cpd), and high (≥0.5 cpd) spatial frequencies, respectively. Cutoff values are arbitrary. Mean length-to-width ratios were calculated for each group of cells categorized based on the optimal orientation, and were superimposed in the scatterplot by a solid curve. Error bars indicate the SDs. Receptive fields of cells pointed to by arrows (C–J) are shown in the bottom corresponding panels. B: plot of the length-to-width ratio against the optimal spatial frequency. Circles, triangles, and square symbols represent neurons tuned to horizontal (within 30° of horizontal), oblique, and vertical (within 30° of vertical), respectively. A solid curve and bars indicate the mean ± SD values of the length-to-width ratios for each population of neurons grouped on the basis of the optimal spatial frequency. Cutoff values are the same as in A. Neurons labeled in A are also indicated by arrows in B except for cells D and E, which are located in the dense cluster of symbols near (0.2, 1). C–J: receptive fields of simple cells that are indicated by arrows in A and B. Bright regions indicate on subregions, whereas dark regions denote off subregions. Solid ellipses indicate the extents of receptive fields determined as 5% of the peak of the fitted Gaussian envelopes. Center of each receptive field is defined as the origin. Optimal correlation delay, spatial frequency (SF), orientation (OR), and length-to-width ratio (LWR) for C–J were as follows, respectively: τ = 55, 42, 43, 54, 52, 51, 46, and 49 ms; SF = 0.32, 0.17, 0.25, 0.17, 0.98, 1.00, 0.33, and 0.50 cycles/deg; OR = 5, 38, 54, 82, 86, 135, 158, and 178°; LWR = 2.43, 1.03, 1.01, 0.73, 1.92, 1.15, 0.93, and 1.24.
Analysis of spatial pooling for subunits in complex cells and the receptive field size of simple cells. A: distribution of pooling ratios used to evaluate the degree of the spatial pooling of subunits. An areal pooling ratio was defined as the receptive field area divided by the subunit envelope area. Black bars highlight neurons for which the areal pooling ratios exhibit statistically significant deviation from one (P < 0.05, resampling). Minimalist energy model for complex cells predicts the areal pooling ratios close to one. Median of the distribution is indicated by an arrow. B: distributions of the areas of complex cell subunits (top) and receptive fields (bottom). Black and gray bars indicate complex cells with and without significant spatial pooling, respectively (P value = 0.05, resampling). C: distribution of the areas of simple cell receptive fields. Arrows in the histograms indicate the medians for each distribution.
Analysis of radial anisotropy of spatial pooling for subunits in complex cells. Degree of spatial pooling was evaluated directionally along radial axes that were at various angles with the major axis of the subunit envelope. A directional pooling ratio was defined as the receptive field size divided by the subunit size, measured along a particular radial axis, as schematically illustrated in C. A: scatterplots show relationships between the maximal (left) or the minimal (right) pooling ratios, and directional angles at which they were obtained. Black symbols represent complex cells for which areal pooling ratios were significantly >1 (P < 0.05, resampling). Gray symbols denote neurons without significant pooling. Filled circles indicate neurons with sufficiently elliptic subunit envelopes, which ensure reliable determination of their major axes (aspect ratio >1.2). Remaining cells are shown by open symbols. Histograms in A illustrate the distributions of directional angles to obtain the maximal and the minimal directional pooling ratios. To examine the distributions, complex cells with sufficiently elliptic subunit envelopes were selected. Following the convention used in the scatterplots, black bars indicate neurons with significant spatial pooling; gray bars represent neurons without significant pooling. B: plot of the geometric mean values of directional pooling ratios across neurons against angle from the subunit major axis. Error bars indicate SEs. C: illustration of definition of directional pooling ratio, which was measured at an angle with the major axis of the subunit envelope. By changing the angle parameter, the maximal and minimal size ratios were searched between the subunit and the receptive field.
Examples of bright-minus-dark maps for 4 complex cells. Bright-minus-dark maps were calculated separately for dark- and bright-reference stimuli. For each cell, the left panel shows the map for a dark reference, the center panel shows that for a bright reference, and the right panel shows the second-order interaction map, which is the difference between the center and left maps. These maps were obtained at the same reference location at which the strongest second-order interaction was observed. Reference location is indicated by the cross hairs. Optimal correlation delay, spatial frequency (SF), orientation (OR), and areal pooling ratio (PR) for second-order interaction maps are indicated to the right of the interaction maps for each cell. Cells shown in B and C are identical to those in Figs. 7C and 4, respectively.
Analysis of center position for the envelopes of bright-minus-dark maps for dark (left) and bright (right) reference stimuli. Center position was defined with respect to the reference location, and plotted along the horizontal axis as spatial phase angle for the optimal spatial frequency. Envelope and the center were computed for each 2D map, but only the offset perpendicular to the optimal orientation (obtained by projecting the center onto the width axis) is shown in these figures. Optimal spatial frequency and orientation were extracted by the Fourier analysis from the second-order interaction map. Amplitude was divided by the SD of values on the edges (one-pixel wide) of the bright-minus-dark map envelope, and plotted along the vertical axis. Maps for complex cells pointed to by arrows (A–D) are shown in the corresponding rows in Fig. 17.
Distributions of center position difference and amplitude ratio between the maps for dark and bright references. Position difference is expressed in terms of a phase angle for the spatial frequency of the subunit. Values were computed relative to the parameters for the dark-reference map. Therefore the amplitude ratios >1 indicate that the bright-reference maps are stronger than the dark-reference map, whereas those <1 indicate the opposite. For this analysis, only those cells were selected for which normalized amplitudes exceeded 3 for both dark- and bright-reference stimuli (n = 78). See also Fig. 18 legend.
Summary of results of comparisons between complex cell receptive fields and subunits (A) and comparisons between complex cell subunits and simple cell receptive fields (B)
A. Comparisons of complex cell receptive fields and their subunits
Receptive fields are slightly larger (mean area ratio = 1.2)
Receptive fields are slightly more circular than subunits
Response maps for bright and dark references
Response strengths can be imbalanced (e.g., Fig. 17, C and D)
Center positions can be different (e.g., Fig. 17D)
B. Comparisons of complex cell subunits and simple cell receptive fields
Number of subregions
Complex cell subunits have more subregions than simple cells
Envelopes of simple cell receptive fields tend to be elongated more horizontally than vertically (e.g., Fig. 14, C–F and J)
Subsection A summarizes comparisons between subunits and receptive fields to examine the minimal model for complex cells. Subsection B summarizes comparisons between complex cell subunits and simple cell receptive fields to examine whether they are equivalent to each other. The comparison of spatial extents is based on length, width, and areal size, whereas the comparison of spatial shape is based on aspect ratios, length-to-width ratios, and the orientation of elongation. Brief comments and relevant examples are given in the right column.
Cover: Electrophysiological and morphological measurements were obtained simultaneously from a single corticospinal neuron. These data served as constraints on evolutionary optimization, generating a family of corticospinal models. A three-dimensional reconstruction serves as the backbone for a pseudo-color visualization of synaptic efficiency as a function of dendritic location, simulated in a single biophysical model selected from the family of optimal individuals. Excitatory synapses at yellow dendritic locations resulted in the largest depolarizations at the soma, while the same synaptic activation at purple locations generated only weak somatic depolarizations. This visualization is surrounded by scatter plots representing the evolutionary optimization: biophysical models optimized across different fitness functions demonstrate tradeoffs between full high-dimensional error (y-axis) and individual error scores (individual x-axes; clockwise order from top
left: subthreshold error, instantaneous firing rate error, spike-shape error, average firing rate error). Color based on 5 error percentiles in increasing instantaneous firing-rate error (purple, red, dark orange, light orange, yellow). From Neymotin SA, Suter BA, Dura-Bernal S, Shepherd GMG, Migliore M, Lytton WW. Optimizing computer models of corticospinal neurons to replicate in vitro dynamics. J Neurophysiol; doi:10.1152/jn.00570.2016.