A possible neural encoding scheme for 3-dimensional (3D) surface slant is illustrated. A: difference of spatial frequency across the eyes (dif-frequency) produces a perception of slant-in-depth. Angle of perceived surface slant depends on the interocular ratio of spatial frequencies. B: disparity energy model is modified to allow encoding for interocular spatial frequency difference. All subunits (S) share the same receptive field position, orientation, and size for left and right eyes as in the standard model, but their preferred spatial frequencies differ across the eyes. C–F: prediction of dif-frequency version of disparity energy model. Equal preferred spatial frequencies for the 2 eyes produces a frontoparallel binocular receptive field (RF) (C, E). Unequal preferred spatial frequencies cause a tilt of its binocular receptive field (D, F). Interocular SF ratio of 1.5 (left:right = 3:2) represents binocular RF tilt of about 10°, that corresponds to a slant in real space of about 70° from the frontoparallel plane at 57 cm viewing distance.
Experimental setup and binocular reverse correlation analysis are illustrated. A: one-dimensional sparse bar noise stimuli were presented to left and right eyes simultaneously by a mirror haploscope setup. B: all possible combinations of left and right eye stimulus position are included for each left–right permutation of contrast sign (dark–dark, bright–bright, dark–bright, bright–dark). Spike trains were cross-correlated with stimulus sequences, and results are displayed as binocular receptive field maps for the 4 permutations (only the map for bright–dark is depicted for clarity).
Binocular receptive fields and their Fourier spectra are shown for simple (A, B) and complex (C, D) cells, respectively. Binocular RF of simple cells tended to be separable in the (XL, XR) domain with 4 peaks in the spectrum, whereas those for complex cells tended to be inseparable and with 2 peaks in the frequency domain. There is a small but apparent tilt (θ) of the binocular RF in C.
Procedures are shown for computing binocular separability index (BSI) and binocular RF tilt angle (θ) from the spectra (see text). A: BSI is determined by a ratio of 2 spectral peak amplitudes, RF and RD, taken from a cross-sectional profile through the highest peak of the spectrum. When BSI is >0.73, neurons are classified as separable, and inseparable otherwise. BSI and θ for this cell (same as that for Fig. 3, A and B) are 0.92 and 3°, respectively. B: BSI for this complex cell (same as that for Fig. 3, C and D) is low (BSI = 0.37), indicating clear inseparability. C: tilt angle (θ) of the binocular RF is calculated from the angular position of the peak in the frequency domain, as the arctangent of the ratio of the peak frequencies for left and right eyes. Tilt angle was −4° for this cell. Same procedure is used for both separable and inseparable types. Cross sections going through the spectral peak that are parallel to the left and right frequency axes depict monocular spatial frequency-tuning curves, as estimated from the binocular RF data. Monocular spatial frequency tuning for left and right are drawn as solid and dashed curve, respectively. Line that goes through the spectral peak and the origin is defined as the cardinal disparity axis for the neuron.
Comparison of conventional simple/complex-type classification and classification by separability of binocular RF is illustrated. A: correlation is shown between left and right F1/F0 ratios. This ratio is used in conventional classification of simple (F1/F0 > 1) and complex cells based on the degree of response modulation to drifting sinusoidal gratings. Ratios for left and right eyes showed significant correlation (Pearson’s r = 0.87, P < 0.01). However, for some neurons, classified types were mismatched between the eyes. B: scatterplot is shown of F1/F0 ratio vs. binocular separability index (BSI). These 2 parameters showed a significant correlation (Pearson’s r = 0.76 and 0.78 for left and right eyes, respectively, P < 0.001, n = 135). F1/F0 ratios were obtained from responses to drifting sinusoidal gratings of optimal spatial frequency for each eye. Open and filled symbols depict data from left and right eyes, respectively. Therefore each cell has 2 symbols for F1/F0 ratios, connected by a line segment for indicating paired data. C: F1/F0 ratios show a bimodal distribution. Filled and open bars indicate right and left eyes, respectively. D: distribution of BSI is shown. Majority of neurons have inseparable binocular RF (as indicated by BSI <0.73), many of which are classified as simple based on the F1/F0 ratio.
Examples are shown of 3 separable type neurons that have different frequency tunings across the eyes. A, left: separable binocular RF of a simple cell is depicted. A, middle: left and right Fourier spectra of the binocular RF are shown as solid and dashed curves, respectively. Although the tilt of binocular RF is small (binocular RF tilt = −7.0), it is statistically significant (P < 0.05, bootstrap test). Predicted disparity gradient is −0.25. A, right: spatial frequency-tuning curves obtained by drifting sinusoidal grating stimuli are illustrated. Open and filled symbols depict responses for the left and right eyes, respectively. Error bars depict SEs. Horizontal dashed line indicates the spontaneous firing rate. Gaussian functions were fitted to the tuning curves. Vertical thin lines indicate the peaks of fitted Gaussian function for left and right eyes, respectively. Preferred spatial frequencies are significantly different between the eyes (bootstrap test, P < 0.05). Spatial frequency ratio is 0.71. B: data from another separable binocular RF is shown for a simple cell in the same format as A. Binocular RF is significantly tilted from frontoparallel plane (binocular RF tilt = −6.3; P < 0.05, bootstrap test). Predicted disparity gradient is −0.22. Spatial frequency ratio is 0.69. C: additional example of separable binocular RFs is shown (binocular RF tilt = −8.5). Predicted disparity gradients and spatial frequency ratio are −0.3 and 0.7, respectively.
Examples are shown of 6 inseparable type neurons. A and B and C and D are pairs of neurons recorded simultaneously, and the data represent responses to the same stimuli. Binocular receptive fields are shown in the left column and their Fourier spectra are shown in the middle column. Dotted and white lines in the left panels indicate tilt angle of binocular RF and frontoparallel line, respectively. Right column depicts monocular spatial frequency-tuning curves obtained by drifting sinusoidal grating stimuli. Format of the figure is otherwise identical to that of Fig. 6. A and B: binocular RFs of both neurons are tilted significantly from frontoparallel plane (P < 0.05, bootstrap test) but their tilts are in the opposite directions (tilt angles are A: −4.0°, B: 7.7°). Spatial frequency ratio for A is 0.62. C: binocular RF of this neuron is not significantly tilted. D: binocular RF of this neuron is tilted significantly from frontoparallel plane (10.2°; P < 0.05, bootstrap test). Spatial frequency-tuning data for this cell are not available. E and F: 2 additional examples are shown of inseparable type neurons. Both of these neurons exhibited significant tilt from the frontoparallel plane (tilt angles are E: −3.8°, F: 6.8°; P < 0.05, bootstrap test). Preferred spatial frequencies are different for the 2 eyes for both neurons. Spatial frequency ratios are 0.76 and 1.28 for E and F, respectively. All of the neurons in this figure were classified as complex, except for B and D for which spatial frequency tunings are not available.
Binocular RFs are illustrated for a pair of neurons. Format of figure is equivalent to Figs. 6 and 7. A: a separable neuron is shown for which the left peak spatial frequency is significantly higher than that for the right (P < 0.05, bootstrap test). Spatial frequency ratio is 1.23. B: an inseparable neuron is depicted similarly. Left and right spatial frequencies are significantly different, and the binocular RF tilted significantly from frontoparallel (tilt angle is −3.7°; P < 0.05, bootstrap test). Spatial frequency ratio is 0.87. Preferred spatial frequencies for the left and right eyes are significantly different for both neurons (bootstrap test, P < 0.05). Note that the directions of interocular spatial frequency shifts are opposite between the 2 neurons. Data from these neurons are recorded about 100 min apart, but without any optical disturbances such as contact lens or eye manipulations or display movements across the measurements.
Distributions of disparity gradients are shown of separable (A) and inseparable neurons (B). Most neurons had disparity gradients within a range of ±0.5. SD was 0.19 for the separable and 0.14 for the inseparable neurons. Black bars indicate cells for which the binocular RF showed significant tilt from a frontoparallel plane; white bars indicate those for which tilt were not significant.
Potential contributions are examined of artifacts that produce apparent tilt of binocular RF and estimated disparity gradient. These factors include mismatch between the distances from the 2 eyes to the display arising from errors in positioning the display and haploscope mirrors, and image magnification difference arising from refractive errors. A: relationship between display positioning error (abscissa) and predicted disparity gradient (ordinate) is shown. Limits of possible positioning errors are estimated to be well below ±5 cm, which translate into disparity gradient errors of less than ±0.1. B: comparisons are shown of distribution of disparity gradients generated by artifacts and those of our data. Data distributions are shown for separable and inseparable binocular RFs by dashed and dotted curves, respectively. Disparity gradient distributions of actual data are significantly wider than that of experimental artifact (test for equal variance, inseparable: F = 13.4, P < 0.001; separable: F = 7.42, P < 0.001).
Goodness of predictions of binocular RF tilts (expressed as disparity gradients) based on the ratios of left and right optimal spatial frequencies are examined. A and B: illustration of correlations of left and right optimal spatial frequencies for separable and inseparable binocular RFs, respectively. Optimal frequencies are obtained from the peak of fitted Gaussian functions. Dotted lines indicate 1 octave difference in the optimal frequencies for the 2 eyes. C and D: disparity gradients from separable and inseparable binocular RFs are plotted against the frequency ratios obtained from A and B. Frequency ratios are defined as fL/fR, where fL and fR, respectively, are left and right optimal spatial frequencies obtained from drifting sinusoidal grating tests. For inseparable neurons shown in D, there is a significant correlation (all data: r = 0.26, n = 90, P < 0.05, Spearman’s correlation coefficient). Correlation analysis on the subset of neurons that exhibited significant tilt (see Fig. 9B) showed somewhat improved correlation coefficient (r = 0.5, n = 29, P < 0.01). Correlation for separable type was not significant. Black and gray symbols indicate data from neurons that exhibited significant and nonsignificant tilts of binocular RFs. Circles and triangles depict cells recorded from areas 17 and 18, respectively. Solid line represents prediction of the dif-frequency version of disparity energy model, as derived from Eq. A8 (see appendix). Labeled data points represent the example cells shown in Figs. 6 and 7. E and F: comparisons of spatial frequency and disparity frequency are illustrated for both separable (E) and inseparable RFs (F). Spatial frequency and disparity frequency are highly correlated in separable RFs. Dashed lines represent regression lines through the data and their slopes are 0.92 and 0.71 for E and F, respectively.
Diagram of a possible alternative hierarchical organization is shown, whereby the tilt of binocular RF is generated by convergence from multiple neurons with untilted binocular RFs. A: each subunit is constructed according to the original disparity energy model. Each subunit’s binocular RF is not tilted, but the optimal disparity progressively shifts depending on their RF’s frontoparallel position. B: a neuron in the next stage will have a highly elongated and tilted binocular RF. Angle of the tilt depends on the rate at which subunits’ preferred disparities shift with frontoparallel position. Such an organization predicts a substantial elongation of the binocular RF in the frontoparallel dimension. Degree of pooling may be quantified by an aspect ratio. Aspect ratio of a binocular RF is defined by the ratio of SDs along the major (a) and minor (b) axes of the RF envelope. C: frequency analysis of binocular RF. Sizes of binocular RFs are proportional to the inverse of SDs of spectral amplitude profiles.
A and B: distributions of aspect ratios for separable (A) and inseparable (B) binocular RFs are illustrated, respectively. Disparity energy model predicts an aspect ratio of 1 (solid vertical line). However, most inseparable RFs had aspect ratios ≫1. Mean aspect ratio was 1.16 for separable RFs and 1.76 for inseparable RFs (vertical dashed line). C and D: relationships are shown between the aspect ratio and disparity gradient of separable (C) and inseparable (D) binocular RFs. Disparity gradients tended to be variable for neurons with low aspect ratios but were relatively small for those with high aspect ratio for inseparable RFs (separable: Mann–Whitney U test, P = 0.09, n = 45; inseparable: Mann–Whitney U test, P < 0.05, n = 90). E and F: relationships are illustrated between the sampling noise quantified in terms of SD of the disparity gradient and the aspect ratio. SDs of the disparity gradients are obtained from the distribution of disparity gradients by bootstrap resampling of 1,000 times. Same vertical scaling is used for C–F for comparison.
A and B: illustration of relationships between the aspect ratio of binocular RFs and their sizes in the frontoparallel and disparity directions, respectively. (A: separable: r = 0.09, P > 0.05, n = 45; inseparable: r = −0.02, P > 0.05; n = 90; B: separable: r = −0.1, P > 0.05, n = 45; inseparable: r = −0.48, P < 0.001, n = 90.) C and D: illustration of relationships between the preferred spatial frequency of neurons and their binocular RF sizes in the frontoparallel and disparity directions, respectively. (C: separable: r = −0.88, P < 0.001, n = 45; inseparable: r = −0.68, P < 0.001, n = 90; D: r = −0.86 P < 0.001, n = 45 for separable, r = −0.82, P < 0.001, n = 90 for inseparable.) E: relationship between preferred spatial frequency and aspect ratio is illustrated (Pearson’s separable: r = −0.1, P > 0.05, n = 45; inseparable: r = 0.5, P < 0.001, n = 90). Circles and triangles indicate cells recorded from areas 17 and 18, respectively. Open and filled symbols depict separable and inseparable RFs, respectively.
Relationship between the preferred orientation and the disparity gradient is illustrated. Preferred orientations are evaluated by the peak of orientation tuning measured by drifting grating test. Average of left and right preferred orientations is used. Each preferred orientation is normalized from 0 to 90°, for horizontal and vertical orientations. There are no correlations between two parameters (Pearson’s r = 0.09, P > 0.05, n = 168).
Relationships are shown of binocular RF characteristics and ratio of spatial frequency to disparity frequency (SF/DF ratio). Because the disparity energy model predicts SF/DF = 1, the ratio reflects discrepancy between the data and predictions by the model. A: relationship between aspect ratio and SF/DF ratio is illustrated. Significant correlation (separable: r = 0.09, P > 0.1, n = 45; inseparable: r = 0.55, P < 0.001, n = 90) is observed for inseparable RFs. B and C: relationships between the SF/DF ratio and the binocular RF sizes in frontoparallel and disparity directions are illustrated, respectively. Circles and triangles indicate data for neurons recorded from areas 17 and 18, respectively. Open and filled symbols depict separable and inseparable RFs, respectively.
A: schematic diagram is shown for illustrating the definition of disparity gradient. Because disparity gradient is a slope in the depth space, the horizontal distance between objects A and B is set arbitrarily equal to the distance between the eyes without losing generality for simplifying the derivation. B: diagram illustrates the geometrical relationship between the sizes of binocular RF as viewed by the 2 eyes (α, β) and the tilt angle θ of binocular RF.
Relationships between the xL, xR domain and the disparity-frontoparallel dimension are shown. A: geometry of binocular viewing condition is illustrated. Gray diamond-shaped area is the region of real space that is jointly covered by left and right receptive fields. B: area corresponding to the diamond region in A is illustrated as the Cartesian xL, xR domain. When the monocular viewing angle is α, the disparity axis spans 2α and the frontoparallel axis spans α. Because of this asymmetry, the xL, xR domain and the disparity-frontoparallel domain cannot be transformed directly by standard coordinate rotations.
Cover: Multiregional Brain-on-a-Chip in vitro model. Neurons (green) from different brain regions (here prefrontal cortex, left, and amygdala, right) are restricted to distinct areas, but are connecting controllably to the other brain region through axons which are guided by microcontact printed lines. This is a zoom in image from the entire brain-on-a-chip model, which consists of three brain regions. This multiregional brain-on-a-chip is electrically active and reacts to drug applications in a brain region dependent way. Astrocytes are labelled in red. The smallest distance between the different brain areas is 1 mm. From Dauth S, Maoz BM, Sheehy SP, Hemphill MA, Murty T, Macedonia MK, Greer AM, Budnik B, Parker KK. Neurons derived from different brain regions are inherently different in vitro: a novel multiregional brain-on-a-chip. J Neurophysiol 117: 1320 –1341, 2017; doi:10.1152/jn.00575.2016.