Columnar organization of spatial phase in visual cortex

Binocular receptive-field construction in the primary visual cortex

ON and OFF thalamic afferents from the two eyes converge in the primary visual cortex to form binocular receptive fields. The receptive fields need to be diverse to sample our visual world but also similar across eyes to achieve binocular fusion. It is currently unknown how the cortex balances these competing needs between receptive-field diversity and similarity. Our results demonstrate that receptive fields in the cat visual cortex are binocularly matched with exquisite precision for retinotopy, orientation/direction preference, orientation/direction selectivity, response latency, and ON-OFF polarity/structure. Specifically, the average binocular mismatches in retinotopy and ON-OFF structure are tightly restricted to 1/20 and 1/5 of the average receptive-field size but are still large enough to generate all types of binocular disparity tuning. Based on these results, we conclude that cortical receptive fields are binocularly matched with the high precision needed to facilitate binocular fusion while allowing restricted mismatches to process visual depth.

Dynamic Recruitment of the Feedforward and Recurrent Mechanism for Black-White Asymmetry in the Primary Visual Cortex

Black and white information is asymmetrically distributed in natural scenes, evokes asymmetric neuronal responses, and causes asymmetric perceptions. Recognizing the universality and essentiality of black-white asymmetry in visual information processing, the neural substrates for black-white asymmetry remain unclear. To disentangle the role of the feedforward and recurrent mechanisms in the generation of cortical black-white asymmetry, we recorded the V1 laminar responses and LGN responses of anesthetized cats of both sexes. In a cortical column, we found that black-white asymmetry starts at the input layer and becomes more pronounced in the output layer. We also found distinct dynamics of black-white asymmetry between the output layer and the input layer. Specifically, black responses dominate in all layers after stimulus onset. After stimulus offset, black and white responses are balanced in the input layer, but black responses still dominate in the output layer. Compared with that in the input layer, the rebound response in the output layer is significantly suppressed. The relative suppression strength evoked by white stimuli is notably stronger and depends on the location within the ON-OFF cortical map. A model with delayed and polarity-selective cortical suppression explains black-white asymmetry in the output layer, within which prominent recurrent connections are identified by Granger causality analysis. In addition to black-white asymmetry in response strength, the interlaminar differences in spatial receptive field varied dynamically. Our findings suggest that the feedforward and recurrent mechanisms are dynamically recruited for the generation of black-white asymmetry in V1.SIGNIFICANCE STATEMENT Black-white asymmetry is universal and essential in visual information processing, yet the neural substrates for cortical black-white asymmetry remain unknown. Leveraging V1 laminar recordings, we provided the first laminar pattern of black-white asymmetry in cat V1 and found distinct dynamics of black-white asymmetry between the output layer and the input layer. Comparing black-white asymmetry across three visual hierarchies, the LGN, V1 input layer, and V1 output layer, we demonstrated that the feedforward and recurrent mechanisms are dynamically recruited for the generation of cortical black-white asymmetry. Our findings not only enhance our understanding of laminar processing within a cortical column but also elucidate how feedforward connections and recurrent connections interact to shape neuronal response properties.

Luminance Contrast Shifts Dominance Balance between ON and OFF Pathways in Human Vision

Human vision processes light and dark stimuli in visual scenes with separate ON and OFF neuronal pathways. In nature, stimuli lighter or darker than their local surround have different spatial properties and contrast distributions (Ratliff et al., 2010; Cooper and Norcia, 2015; Rahimi-Nasrabadi et al., 2021). Similarly, in human vision, we show that luminance contrast affects the perception of lights and darks differently. At high contrast, human subjects of both sexes locate dark stimuli faster and more accurately than light stimuli, which is consistent with a visual system dominated by the OFF pathway. However, at low contrast, they locate light stimuli faster and more accurately than dark stimuli, which is consistent with a visual system dominated by the ON pathway. Luminance contrast was strongly correlated with multiple ON/OFF dominance ratios estimated from light/dark ratios of performance errors, missed targets, or reaction times (RTs). All correlations could be demonstrated at multiple eccentricities of the central visual field with an ON-OFF perimetry test implemented in a head-mounted visual display. We conclude that high-contrast stimuli are processed faster and more accurately by OFF pathways than ON pathways. However, the OFF dominance shifts toward ON dominance when stimulus contrast decreases, as expected from the higher-contrast sensitivity of ON cortical pathways (Kremkow et al., 2014; Rahimi-Nasrabadi et al., 2021). The results highlight the importance of contrast polarity in visual field measurements and predict a loss of low-contrast vision in humans with ON pathway deficits, as demonstrated in animal models (Sarnaik et al., 2014).SIGNIFICANCE STATEMENT ON and OFF retino-thalamo-cortical pathways respond differently to luminance contrast. In both animal models and humans, low contrasts drive stronger responses from ON pathways, whereas high contrasts drive stronger responses from OFF pathways. We demonstrate that these ON-OFF pathway differences have a correlate in human vision. At low contrast, humans locate light targets faster and more accurately than dark targets but, as contrast increases, dark targets become more visible than light targets. We also demonstrate that contrast is strongly correlated with multiple light/dark ratios of visual performance in central vision. These results provide a link between neuronal physiology and human vision while emphasizing the importance of stimulus polarity in measurements of visual fields and contrast sensitivity.

Thalamocortical boutons cluster by ON/OFF responses in mouse primary visual cortex

In higher mammals, the thalamic afferents to primary visual cortex cluster according to their responses to increases (ON) or decreases (OFF) in luminance. This feature of thalamocortical wiring is thought to create columnar, ON/OFF domains in V1. We have recently shown that mice also have ON/OFF cortical domains, but the organization of their thalamic afferents remains unknown. Here we measured the visual responses of thalamocortical boutons with two-photon imaging and found that they also cluster in space according to ON/OFF responses. Moreover, fluctuations in the relative density of ON/OFF boutons mirror fluctuations in the relative density of ON/OFF receptive field positions on the visual field. These findings indicate a segregation of ON/OFF signals already present in the thalamic input. We propose that ON/OFF clustering may reflect the spatial distribution of ON/OFF responses in retinal ganglion cell mosaics.NEW & NOTEWORTHY Neurons in primary visual cortex cluster into ON and OFF domains, which have been shown to be linked to the organization of receptive fields and cortical maps. Here we show that in the mouse such clustering is already present in the geniculate input, suggesting that the cortical architecture may be shaped by the representation of ON/OFF signals in the thalamus and the retina.

Cortical mechanisms of visual brightness

The primary visual cortex signals the onset of light and dark stimuli with ON and OFF cortical pathways. Here, we demonstrate that both pathways generate similar response increments to large homogeneous surfaces and their response average increases with surface brightness. We show that, in cat visual cortex, response dominance from ON or OFF pathways is bimodally distributed when stimuli are smaller than one receptive field center but unimodally distributed when they are larger. Moreover, whereas small bright stimuli drive opposite responses from ON and OFF pathways (increased versus suppressed activity), large bright surfaces drive similar response increments. We show that this size-brightness relation emerges because strong illumination increases the size of light surfaces in nature and both ON and OFF cortical neurons receive input from ON thalamic pathways. We conclude that visual scenes are perceived as brighter when the average response increments from ON and OFF cortical pathways become stronger.

Mazade et al. find that the visual cortex encodes brightness differently for small than large stimuli. Bright small stimuli drive cortical pathways signaling lights and suppress cortical pathways signaling darks. Conversely, large surfaces drive response increments from both pathways and appear brightest when the response average is strongest.

ON/OFF domains shape receptive field structure in mouse visual cortex

In higher mammals, thalamic afferents to primary visual cortex (area V1) segregate according to their responses to increases (ON) or decreases (OFF) in luminance. This organization induces columnar, ON/OFF domains postulated to provide a scaffold for the emergence of orientation tuning. To further test this idea, we asked whether ON/OFF domains exist in mouse V1. Here we show that mouse V1 is indeed parceled into ON/OFF domains. Interestingly, fluctuations in the relative density of ON/OFF neurons on the cortical surface mirror fluctuations in the relative density of ON/OFF receptive field centers on the visual field. Moreover, the local diversity of cortical receptive fields is explained by a model in which neurons linearly combine a small number of ON and OFF signals available in their cortical neighborhoods. These findings suggest that ON/OFF domains originate in fluctuations of the balance between ON/OFF responses across the visual field which, in turn, shapes the structure of cortical receptive fields.

Neurons in the early visual system respond preferentially to the onset or offset of light. Here the authors show that ON/OFF responses cluster in the mouse primary visual cortex, shaping the receptive fields of cortical cells.

A theory of cortical map formation in the visual brain

The cerebral cortex receives multiple afferents from the thalamus that segregate by stimulus modality forming cortical maps for each sense. In vision, the primary visual cortex maps the multiple dimensions of the visual stimulus in patterns that vary across species for reasons unknown. Here we introduce a general theory of cortical map formation, which proposes that map diversity emerges from species variations in the thalamic afferent density sampling sensory space. In the theory, increasing afferent sampling density enlarges the cortical domains representing the same visual point, allowing the segregation of afferents and cortical targets by multiple stimulus dimensions. We illustrate the theory with an afferent-density model that accurately replicates the maps of different species through afferent segregation followed by thalamocortical convergence pruned by visual experience. Because thalamocortical pathways use similar mechanisms for axon segregation and pruning, the theory may extend to other sensory areas of the mammalian brain.

Najafian et al. introduce a developmental theory of map formation in the cerebral cortex. The theory proposes that increases in the density of thalamic afferents sampling sensory space make cortical maps to segregate more stimulus dimensions.

Age-related decline in function of ON and OFF visual pathways

Purpose

A simple psychophysical paradigm is available as a digital application in iOS devices such as iPad to measure the function of ON and OFF visual pathways. However, an age-matched normative database is not readily available. The purpose of the study is to evaluate the response of ON and OFF visual pathways as a function of age.

Methods

158 normal healthy adults (84 males and 74 females) whose age ranged 18–80 years participated in the study. None of them had any ocular disease (except cataract of grade II or less) and visual acuity of ≤ 20/25. Monocular testing (only one eye) was performed on the ‘EyeSpeed’ application on an iPad at 40cm distance. The targets ranged between 1 to 3 light or dark squares presented randomly in a noise background and participants responded by indicating the number of squares by touching the screen as fast as possible. The main outcome variables are reaction time, accuracy and performance index (1 / speed * accuracy).

Results

The median reaction time was shorter (Median (IQR): 1.53s (0.49) [dark] Vs 1.76s (0.58) [light], p < 0.001) and accuracy was higher (97.21% (3.30) [dark] Vs 95.15% (5.10) [light], p < 0.001) for dark targets than the light targets. Performance index and reaction time for both target types significantly correlated with age (ρ = -0.41 to -0.43; p < 0.001).

Conclusions

This normative database will be useful to quantify disease-specific defects. More importantly, the ON pathway function can potentially serve as a surrogate for rod photoreceptor function.

ON and OFF Signaling Pathways in the Retina and the Visual System

Visual processing starts at the retina of the eye, and signals are then transferred primarily to the visual cortex and the tectum. In the retina, multiple neural networks encode different aspects of visual input, such as color and motion. Subsequently, multiple neural streams in parallel convey unique aspects of visual information to cortical and subcortical regions. Bipolar cells, which are the second order neurons of the retina, separate visual signals evoked by light and dark contrasts and encode them to ON and OFF pathways, respectively. The interplay between ON and OFF neural signals is the foundation for visual processing for object contrast which underlies higher order stimulus processing. ON and OFF pathways have been classically thought to signal in a mirror-symmetric manner. However, while these two pathways contribute synergistically to visual perception in some instances, they have pronounced asymmetries suggesting independent operation in other cases. In this review, we summarize the role of the ON-OFF dichotomy in visual signaling, aiming to contribute to the understanding of visual recognition.

Natural environment statistics in the upper and lower visual field are reflected in mouse retinal specializations

Pressures for survival make sensory circuits adapted to a species' natural habitat and its behavioral challenges. Thus, to advance our understanding of the visual system, it is essential to consider an animal's specific visual environment by capturing natural scenes, characterizing their statistical regularities, and using them to probe visual computations. Mice, a prominent visual system model, have salient visual specializations, being dichromatic with enhanced sensitivity to green and UV in the dorsal and ventral retina, respectively. However, the characteristics of their visual environment that likely have driven these adaptations are rarely considered. Here, we built a UV-green-sensitive camera to record footage from mouse habitats. This footage is publicly available as a resource for mouse vision research. We found chromatic contrast to greatly diverge in the upper, but not the lower, visual field. Moreover, training a convolutional autoencoder on upper, but not lower, visual field scenes was sufficient for the emergence of color-opponent filters, suggesting that this environmental difference might have driven superior chromatic opponency in the ventral mouse retina, supporting color discrimination in the upper visual field. Furthermore, the upper visual field was biased toward dark UV contrasts, paralleled by more light-offset-sensitive ganglion cells in the ventral retina. Finally, footage recorded at twilight suggests that UV promotes aerial predator detection. Our findings support that natural scene statistics shaped early visual processing in evolution.

Image luminance changes contrast sensitivity in visual cortex

Accurate measures of contrast sensitivity are important for evaluating visual disease progression and for navigation safety. Previous measures suggested that cortical contrast sensitivity was constant across widely different luminance ranges experienced indoors and outdoors. Against this notion, here, we show that luminance range changes contrast sensitivity in both cat and human cortex, and the changes are different for dark and light stimuli. As luminance range increases, contrast sensitivity increases more within cortical pathways signaling lights than those signaling darks. Conversely, when the luminance range is constant, light-dark differences in contrast sensitivity remain relatively constant even if background luminance changes. We show that a Naka-Rushton function modified to include luminance range and light-dark polarity accurately replicates both the statistics of light-dark features in natural scenes and the cortical responses to multiple combinations of contrast and luminance. We conclude that differences in light-dark contrast increase with luminance range and are largest in bright environments.

Dynamics of Contrast Decrement and Increment Responses in Human Visual Cortex

Purpose

The goal of the present experiments was to determine whether electrophysiologic response properties of the ON and OFF visual pathways observed in animal experimental models can be observed in humans.

Methods

Steady-state visual evoked potentials (SSVEPs) were recorded in response to equivalent magnitude contrast increments and decrements presented within a probe-on-pedestal Westheimer sensitization paradigm. The probes were modulated with sawtooth temporal waveforms at a temporal frequency of 3 or 2.73 Hz. SSVEP response waveforms and response spectra for incremental and decremental stimuli were analyzed as a function of stimulus size and visual field location in 67 healthy adult participants.

Results

SSVEPs recorded at the scalp differ between contrast decrements and increments of equal Weber contrast: SSVEP responses were larger in amplitude and shorter in latency for contrast decrements than for contrast increments. Both increment and decrement responses were larger for displays that were scaled for cortical magnification.

Conclusions

In a fashion that parallels results from the early visual system of cats and monkeys, two key properties of ON versus OFF pathways found in single-unit recordings are recapitulated at the population level of activity that can be observed with scalp electrodes, allowing differential assessment of ON and OFF pathway activity in human.

Translational Relevance

As data from preclinical models of visual pathway dysfunction point to differential damage to subtypes of retinal ganglion cells, this approach may be useful in future work on disease detection and treatment monitoring.

Functional Specialization of ON and OFF Cortical Pathways for Global-Slow and Local-Fast Vision

Visual information is processed in the cortex by ON and OFF pathways that respond to light and dark stimuli. Responses to darks are stronger, faster, and driven by a larger number of cortical neurons than responses to lights. Here, we demonstrate that these light-dark cortical asymmetries reflect a functional specialization of ON and OFF pathways for different stimulus properties. We show that large long-lasting stimuli drive stronger cortical responses when they are light, whereas small fast stimuli drive stronger cortical responses when they are dark. Moreover, we show that these light-dark asymmetries are preserved under a wide variety of luminance conditions that range from photopic to low mesopic light. Our results suggest that ON and OFF pathways extract different spatiotemporal information from visual scenes, making OFF local-fast signals better suited to maximize visual acuity and ON global-slow signals better suited to guide the eye movements needed for retinal image stabilization.

Dynamic Patterns of Spontaneous Ongoing Activity in the Visual Cortex of Anesthetized and Awake Monkeys are Different

Ongoing internal cortical activity plays a major role in perception and behavior both in animals and humans. Previously we have shown that spontaneous patterns resembling orientation-maps appear over large cortical areas in the primary visual-cortex of anesthetized cats. However, it remains unknown 1) whether spontaneous-activity in the primate also displays similar patterns and 2) whether a significant difference exists between cortical ongoing-activity in the anesthetized and awake primate. We explored these questions by combining voltage-sensitive-dye imaging with multiunit and local-field-potential recordings. Spontaneously emerging orientation and ocular-dominance maps, spanning up to 6 × 6 mm2, were readily observed in anesthetized but not in awake monkeys. Nevertheless, spontaneous correlated-activity involving orientation-domains was observed in awake monkeys. Under both anesthetized and awake conditions, spontaneous correlated-activity coincided with traveling waves. We found that spontaneous activity resembling orientation-maps in awake animals spans smaller cortical areas in each instance, but over time it appears across all of V1. Furthermore, in the awake monkey, our results suggest that the synaptic strength had been completely reorganized including connections between dissimilar elements of the functional architecture. These findings lend support to the notion that ongoing-activity has many more fast switching representations playing an important role in cortical function and behavior.

Mechanisms of Feature Selectivity and Invariance in Primary Visual Cortex

Visual object identification requires both selectivity for specific visual features that are important to the object's identity and invariance to feature manipulations. For example, a hand can be shifted in position, rotated, or contracted but still be recognized as a hand. How are the competing requirements of selectivity and invariance built into the early stages of visual processing? Typically, cells in the primary visual cortex are classified as either simple or complex. They both show selectivity for edge-orientation but complex cells develop invariance to edge position within the receptive field (spatial phase). Using a data-driven model that extracts the spatial structures and nonlinearities associated with neuronal computation, we quantitatively describe the balance between selectivity and invariance in complex cells. Phase invariance is frequently partial, while invariance to orientation and spatial frequency are more extensive than expected. The invariance arises due to two independent factors: (1) the structure and number of filters and (2) the form of nonlinearities that act upon the filter outputs. Both vary more than previously considered, so primary visual cortex forms an elaborate set of generic feature sensitivities, providing the foundation for more sophisticated object processing.

Multiple-target tracking in human and machine vision

Humans are able to track multiple objects at any given time in their daily activities—for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human–computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.

Multiple-target tracking (MTT) is a challenging task vital for both a human’s daily life and for many artificial intelligent systems, such as those used for urban traffic control. Neuroscientists are interested in discovering the underlying neural mechanisms that successfully exploit cognitive resources, e.g., spatial attention or memory, during MTT. Computer-vision specialists aim to develop powerful MTT algorithms based on advanced models or data-driven computational methods. In this paper, we review MTT studies from both communities and discuss how findings from cognitive studies can inspire developers to construct higher performing MTT algorithms. Moreover, some directions have been proposed through which MTT algorithms could raise new questions in the cognitive science domain, and answering them can shed light on neural processes underlying MTT.

Cortical Balance Between ON and OFF Visual Responses Is Modulated by the Spatial Properties of the Visual Stimulus

The primary visual cortex of carnivores and primates is dominated by the OFF visual pathway and responds more strongly to dark than light stimuli. Here, we demonstrate that this cortical OFF dominance is modulated by the size and spatial frequency of the stimulus in awake primates and we uncover a main neuronal mechanism underlying this modulation. We show that large grating patterns with low spatial frequencies drive five times more OFF-dominated than ON-dominated neurons, but this pronounced cortical OFF dominance is strongly reduced when the grating size decreases and the spatial frequency increases, as when the stimulus moves away from the observer. We demonstrate that the reduction in cortical OFF dominance is not caused by a selective reduction of visual responses in OFF-dominated neurons but by a change in the ON/OFF response balance of neurons with diverse receptive field properties that can be ON or OFF dominated, simple, or complex. We conclude that cortical OFF dominance is continuously adjusted by a neuronal mechanism that modulates ON/OFF response balance in multiple cortical neurons when the spatial properties of the visual stimulus change with viewing distance and/or optical blur.

Diversity of Feature Selectivity in Macaque Visual Cortex Arising from a Limited Number of Broadly Tuned Input Channels

Spike (action potential) responses of most primary visual cortical cells in the macaque are sharply tuned for the orientation of a line or an edge, and neurons preferring similar orientations are clustered together in cortical columns. The preferred stimulus orientation of these columns span the full range of orientations, as observed in recordings of spikes and in classical optical imaging of intrinsic signals. However, when we imaged the putative thalamic input to striate cortical cells that can be seen in imaging of intrinsic signals when they are analyzed on a larger spatial scale, we found that the orientation domain map of the primary visual cortex did not show the same diversity of orientations. This map was dominated by just the one orientation that is most commonly preferred by neurons in the retina and the lateral geniculate nucleus. This supports cortical feature selectivity and columnar architecture being built upon feed-forward signals transmitted from the thalamus in a very limited number of broadly tuned input channels.

Diversity of Ocular Dominance Patterns in Visual Cortex Originates from Variations in Local Cortical Retinotopy

The primary visual cortex contains a detailed map of retinal stimulus position (retinotopic map) and eye input (ocular dominance map) that results from the precise arrangement of thalamic afferents during cortical development. For reasons that remain unclear, the patterns of ocular dominance are very diverse across species and can take the shape of highly organized stripes, convoluted beads, or no pattern at all. Here, we use a new image-processing algorithm to measure ocular dominance patterns more accurately than in the past. We use these measurements to demonstrate that ocular dominance maps follow a common organizing principle that makes the cortical axis with the slowest retinotopic gradient orthogonal to the ocular dominance stripes. We demonstrate this relation in multiple regions of the primary visual cortex from individual animals, and different species. Moreover, consistent with the increase in the retinotopic gradient with visual eccentricity, we demonstrate a strong correlation between eccentricity and ocular dominance stripe width. We also show that an eye/polarity grid emerges within the visual cortical map when the representation of light and dark stimuli segregates along an axis orthogonal to the ocular dominance stripes, as recently demonstrated in cats. Based on these results, we propose a developmental model of visual cortical topography that sorts thalamic afferents by eye input and stimulus polarity, and then maximizes the binocular retinotopic match needed for depth perception and the light-dark retinotopic mismatch needed to process stimulus orientation. In this model, the different ocular dominance patterns simply emerge from differences in local retinotopic cortical topography.SIGNIFICANCE STATEMENT Thalamocortical afferents segregate in primary visual cortex by eye input and light-dark polarity. This afferent segregation forms cortical patterns that vary greatly across species for reasons that remain unknown. Here we show that the formation of ocular dominance patterns follows a common organizing principle across species that aligns the cortical axis of ocular dominance segregation with the axis of slowest retinotopic gradient. Based on our results, we propose a model of visual cortical topography that sorts thalamic afferents by eye input and stimulus polarity along orthogonal axes with the slowest and fastest retinotopic gradients, respectively. This organization maximizes the binocular retinotopic match needed for depth perception and the light-dark retinotopic mismatch needed to process stimulus orientation in carnivores and primates.

Amblyopia Affects the ON Visual Pathway More than the OFF

Visual information reaches the cerebral cortex through parallel ON and OFF pathways that signal the presence of light and dark stimuli in visual scenes. We have previously demonstrated that optical blur reduces visual salience more for light than dark stimuli because it removes the high spatial frequencies from the stimulus, and low spatial frequencies drive weaker ON than OFF cortical responses. Therefore, we hypothesized that sustained optical blur during brain development should weaken ON cortical pathways more than OFF, increasing the dominance of darks in visual perception. Here we provide support for this hypothesis in humans with anisometropic amblyopia who suffered sustained optical blur early after birth in one of the eyes. In addition, we show that the dark dominance in visual perception also increases in strabismic amblyopes that have their vision to high spatial frequencies reduced by mechanisms not associated with optical blur. Together, we show that amblyopia increases visual dark dominance by 3-10 times and that the increase in dark dominance is strongly correlated with amblyopia severity. These results can be replicated with a computational model that uses greater luminance/response saturation in ON than OFF pathways and, as a consequence, reduces more ON than OFF cortical responses to stimuli with low spatial frequencies. We conclude that amblyopia affects the ON cortical pathway more than the OFF, a finding that could have implications for future amblyopia treatments.SIGNIFICANCE STATEMENT Amblyopia is a loss of vision that affects 2-5% of children across the world and originates from a deficit in visual cortical circuitry. Current models assume that amblyopia affects similarly ON and OFF visual pathways, which signal light and dark features in visual scenes. Against this current belief, here we demonstrate that amblyopia affects the ON visual pathway more than the OFF, a finding that could have implications for new amblyopia treatments targeted at strengthening a weak ON visual pathway.

Early Visual Processing of Feature Saliency Tasks: A Review of Psychophysical Experiments

The visual system is constantly bombarded with information originating from the outside world, but it is unable to process all the received information at any given time. In fact, the most salient parts of the visual scene are chosen to be processed involuntarily and immediately after the first glance along with endogenous signals in the brain. Vision scientists have shown that the early visual system, from retina to lateral geniculate nucleus (LGN) and then primary visual cortex, selectively processes the low-level features of the visual scene. Everything we perceive from the visual scene is based on these feature properties and their subsequent combination in higher visual areas. Different experiments have been designed to investigate the impact of these features on saliency and understand the relative visual mechanisms. In this paper, we review the psychophysical experiments which have been published in the last decades to indicate how the low-level salient features are processed in the early visual cortex and extract the most important and basic information of the visual scene. Important and open questions are discussed in this review as well and one might pursue these questions to investigate the impact of higher level features on saliency in complex scenes or natural images.

Thalamocortical Circuits and Functional Architecture

The thalamocortical pathway is the main route of communication between the eye and the cerebral cortex. During embryonic development, thalamocortical afferents travel to L4 and are sorted by receptive field position, eye of origin, and contrast polarity (i.e., preference for light or dark stimuli). In primates and carnivores, this sorting involves numerous afferents, most of which sample a limited region of the binocular field. Devoting abundant thalamocortical resources to process a limited visual field has a clear advantage: It allows many stimulus combinations to be sampled at each spatial location. Moreover, the sampling efficiency can be further enhanced by organizing the afferents in a cortical grid for eye input and contrast polarity. We argue that thalamocortical interactions within this eye-polarity grid can be used to represent multiple stimulus combinations found in nature and to build an accurate cortical map for multidimensional stimulus space.

Mouse color and wavelength-specific luminance contrast sensitivity are non-uniform across visual space

Mammalian visual behaviors, as well as responses in the neural systems underlying these behaviors, are driven by luminance and color contrast. With constantly improving tools for measuring activity in cell-type-specific populations in the mouse during visual behavior, it is important to define the extent of luminance and color information that is behaviorally accessible to the mouse. A non-uniform distribution of cone opsins in the mouse retina potentially complicates both luminance and color sensitivity; opposing gradients of short (UV-shifted) and middle (blue/green) cone opsins suggest that color discrimination and wavelength-specific luminance contrast sensitivity may differ with retinotopic location. Here we ask how well mice can discriminate color and wavelength-specific luminance changes across visuotopic space. We found that mice were able to discriminate color and were able to do so more broadly across visuotopic space than expected from the cone-opsin distribution. We also found wavelength-band-specific differences in luminance sensitivity.

Neuronal mechanisms underlying differences in spatial resolution between darks and lights in human vision

Artists and astronomers noticed centuries ago that humans perceive dark features in an image differently from light ones; however, the neuronal mechanisms underlying these dark/light asymmetries remained unknown. Based on computational modeling of neuronal responses, we have previously proposed that such perceptual dark/light asymmetries originate from a luminance/response saturation within the ON retinal pathway. Consistent with this prediction, here we show that stimulus conditions that increase ON luminance/response saturation (e.g., dark backgrounds) or its effect on light stimuli (e.g., optical blur) impair the perceptual discrimination and salience of light targets more than dark targets in human vision. We also show that, in cat visual cortex, the magnitude of the ON luminance/response saturation remains relatively constant under a wide range of luminance conditions that are common indoors, and only shifts away from the lowest luminance contrasts under low mesopic light. Finally, we show that the ON luminance/response saturation affects visual salience mostly when the high spatial frequencies of the image are reduced by poor illumination or optical blur. Because both low luminance and optical blur are risk factors in myopia, our results suggest a possible neuronal mechanism linking myopia progression with the function of the ON visual pathway.

Inhibition in Simple Cell Receptive Fields Is Broad and OFF-Subregion Biased

Inhibition in thalamorecipient layer 4 simple cells of primary visual cortex is believed to play important roles in establishing visual response properties and integrating visual inputs across their receptive fields (RFs). Simple cell RFs are characterized by nonoverlapping, spatially restricted subregions in which visual stimuli can either increase or decrease the firing rate of the cell, depending on contrast. Inhibition is believed to be triggered exclusively from visual stimulation of individual RF subregions. However, this view is at odds with the known anatomy of layer 4 interneurons in visual cortex and differs from recent findings in mouse visual cortex. Here we show with in vivo intracellular recordings in cats that while excitation is restricted to RF subregions, inhibition spans the width of simple cell RFs. Consequently, excitatory stimuli within a subregion concomitantly drive excitation and inhibition. Furthermore, we found that the distribution of inhibition across the RF is stronger toward OFF subregions. This inhibitory OFF-subregion bias has a functional consequence on spatial integration of inputs across the RF. A model based on the known anatomy of layer 4 demonstrates that the known proportion and connectivity of inhibitory neurons in layer 4 of primary visual cortex is sufficient to explain broad inhibition with an OFF-subregion bias while generating a variety of phase relations, including antiphase, between excitation and inhibition in response to drifting gratings.SIGNIFICANCE STATEMENT The wiring of excitatory and inhibitory neurons in cortical circuits is key to determining the response properties in sensory cortex. In the visual cortex, the first cells that receive visual input are simple cells in layer 4. The underlying circuitry responsible for the response properties of simple cells is not yet known. In this study, we challenge a long-held view concerning the pattern of inhibitory input and provide results that agree with current known anatomy. We show here that inhibition is evoked broadly across the receptive fields of simple cells, and we identify a surprising bias in inhibition within the receptive field. Our findings represent a step toward a unified view of inhibition across different species and sensory systems.

Interlayer Repulsion of Retinal Ganglion Cell Mosaics Regulates Spatial Organization of Functional Maps in the Visual Cortex

In higher mammals, orientation tuning of neurons is organized into a quasi-periodic pattern in the primary visual cortex. Our previous model studies suggested that the topography of cortical orientation maps may originate from moiré interference of ON and OFF retinal ganglion cell (RGC) mosaics, but did not account for how the consistent spatial period of maps could be achieved. Here we address this issue with two crucial findings on the development of RGC mosaics: first, homotypic local repulsion between RGCs can develop a long-range hexagonal periodicity. Second, heterotypic interaction restrains the alignment of ON and OFF mosaics, and generates a periodic interference pattern map with consistent spatial frequency. To validate our model, we quantitatively analyzed the RGC mosaics in cat data, and confirmed that the observed retinal mosaics showed evidence of heterotypic interactions, contrary to the previous view that ON and OFF mosaics are developed independently.SIGNIFICANCE STATEMENT Orientation map is one of the most studied functional maps in the brain, but it has remained unanswered how the consistent spatial periodicity of maps could be developed. In the current study, we address this issue with our developmental model for the retinal origin of orientation map. We showed that local repulsive interactions between retinal ganglion cells (RGCs) can develop a hexagonal periodicity in the RGC mosaics and restrict the alignment between ON and OFF mosaics, so that they generate a periodic pattern with consistent spatial frequency for both the RGC mosaics and the cortical orientation maps. Our results demonstrate that the organization of functional maps in visual cortex, including its structural consistency, may be constrained by a retinal blueprint.

A Global Orientation Map in the Primary Visual Cortex (V1): Could a Self Organizing Model Reveal Its Hidden Bias?

A remarkable accomplishment of self organizing models is their ability to simulate the development of feature maps in the cortex. Additionally, these models have been trained to tease out the differential causes of multiple feature maps, mapped on to the same output space. Recently, a Laterally Interconnected Synergetically Self Organizing Map (LISSOM) model has been used to simulate the mapping of eccentricity and meridional angle onto orthogonal axes in the primary visual cortex (V1). This model is further probed to simulate the development of the radial bias in V1, using a training set that consists of both radial (rectangular bars of random size and orientation) as well as non-radial stimuli. The radial bias describes the preference of the visual system toward orientations that match the angular position (meridional angle) of that orientation with respect to the point of fixation. Recent fMRI results have shown that there exists a coarse scale orientation map in V1, which resembles the meridional angle map, thereby providing a plausible neural basis for the radial bias. The LISSOM model, trained for the development of the retinotopic map, on probing for orientation preference, exhibits a coarse scale orientation map, consistent with these experimental results, quantified using the circular cross correlation (r_c ). The r_c between the orientation map developed on probing with a thin annular ring containing sinusoidal gratings with a spatial frequency of 0.5 cycles per degree (cpd) and the corresponding meridional map for the same annular ring, has a value of 0.8894. The results also suggest that the radial bias goes beyond the current understanding of a node to node correlation between the two maps.

Efficient Receptive Field Tiling in Primate V1

The primary visual cortex (V1) encodes a diverse set of visual features, including orientation, ocular dominance (OD), and spatial frequency (SF), whose joint organization must be precisely structured to optimize coverage within the retinotopic map. Prior experiments have only identified efficient coverage based on orthogonal maps. Here we used two-photon calcium imaging to reveal an alternative arrangement for OD and SF maps in macaque V1; their gradients run parallel but with unique spatial periods, whereby low-SF regions coincide with monocular regions. Next we mapped receptive fields and found surprisingly precise micro-retinotopy that yields a smaller point-image and requires more efficient inter-map geometry, thus underscoring the significance of map relationships. While smooth retinotopy is constraining, studies suggest that it improves both wiring economy and the V1 population code read downstream. Altogether, these data indicate that connectivity within V1 is finely tuned and precise at the level of individual neurons.

Push-Pull Receptive Field Organization and Synaptic Depression: Mechanisms for Reliably Encoding Naturalistic Stimuli in V1

Neurons in the primary visual cortex are known for responding vigorously but with high variability to classical stimuli such as drifting bars or gratings. By contrast, natural scenes are encoded more efficiently by sparse and temporal precise spiking responses. We used a conductance-based model of the visual system in higher mammals to investigate how two specific features of the thalamo-cortical pathway, namely push-pull receptive field organization and fast synaptic depression, can contribute to this contextual reshaping of V1 responses. By comparing cortical dynamics evoked respectively by natural vs. artificial stimuli in a comprehensive parametric space analysis, we demonstrate that the reliability and sparseness of the spiking responses during natural vision is not a mere consequence of the increased bandwidth in the sensory input spectrum. Rather, it results from the combined impacts of fast synaptic depression and push-pull inhibition, the later acting for natural scenes as a form of “effective” feed-forward inhibition as demonstrated in other sensory systems. Thus, the combination of feedforward-like inhibition with fast thalamo-cortical synaptic depression by simple cells receiving a direct structured input from thalamus composes a generic computational mechanism for generating a sparse and reliable encoding of natural sensory events.

Topology of ON and OFF inputs in visual cortex enables an invariant columnar architecture

Circuits in visual cortex integrate the information derived from separate ON and OFF pathways to construct orderly columnar representations of orientation and visual space^1–7. How this transformation is achieved to meet the specific topographic constraints of each representation remains unclear. Here we report several novel features of ON/OFF convergence visualized by mapping the receptive fields of layer 2/3 neurons in tree shrew visual cortex using two-photon imaging of GCaMP6 calcium signals. The spatially separate ON and OFF subfields of simple cells in layer 2/3 were found to exhibit topologically distinct relationships with the maps of visual space and orientation preference. The centers of OFF subfields for neurons in a given region of cortex were confined to a compact region of visual space and displayed a smooth visuotopic progression. In contrast, the centers of the ON subfields were distributed over a wider region of visual space, displayed significant visuotopic scatter, and an orientation-specific displacement consistent with orientation preference map structure. As a result, cortical columns exhibit an invariant aggregate receptive field structure: an OFF-dominated central region flanked by ON-dominated subfields. This distinct arrangement of ON- and OFF- inputs enables continuity in the mapping of both orientation and visual space and the generation of a columnar map of absolute spatial phase.

Principles underlying sensory map topography in primary visual cortex

The primary visual cortex contains a detailed map of the visual scene, which is represented according to multiple stimulus dimensions including spatial location, ocular dominance and orientation. The maps for spatial location and ocular dominance originate from the spatial arrangement of thalamic axons in cortex. However, the origin of the other maps remains unclear. Here we demonstrate that the cortical maps for orientation, direction and retinal disparity are all strongly related to the organization for spatial location of light (ON) and dark (OFF) stimuli, an organization that we show is OFF-dominated, OFF-centric and runs orthogonal to ocular dominance columns. Because this ON/OFF organization originates from the clustering of ON and OFF thalamic afferents in visual cortex, we conclude that all main features of cortical topography, including orientation, direction and retinal disparity, follow a common organizing principle that arranges thalamic axons with similar retinotopy and ON/OFF polarity in neighboring cortical regions.

A Division of Light and Dark in the Visual Cortex

The fate of ON-OFF receptive field segregation in the visual cortex has long eluded scrutiny. In this issue of Neuron, Smith et al. (2015) now reveal the intricate relationship between luminance polarity and orientation selectivity in the upper layers of ferret visual cortex.

Modular Representation of Luminance Polarity in the Superficial Layers of Primary Visual Cortex

The spatial arrangement of luminance increments (ON) and decrements (OFF) falling on the retina provides a wealth of information used by central visual pathways to construct coherent representations of visual scenes. But how the polarity of luminance change is represented in the activity of cortical circuits remains unclear. Using wide-field epifluorescence and two-photon imaging we demonstrate a robust modular representation of luminance polarity (ON or OFF) in the superficial layers of ferret primary visual cortex. Polarity-specific domains are found with both uniform changes in luminance and single light/dark edges, and include neurons selective for orientation and direction of motion. The integration of orientation and polarity preference is evident in the selectivity and discrimination capabilities of most layer 2/3 neurons. We conclude that polarity selectivity is an integral feature of layer 2/3 neurons, ensuring that the distinction between light and dark stimuli is available for further processing in downstream extrastriate areas.

Origins of feature selectivities and maps in the mammalian primary visual cortex

A common feature of the mammalian striate cortex is the arrangement of 'orientation domains' containing neurons preferring similar stimulus orientations. They are arranged as spokes of a pinwheel that converge at singularities known as 'pinwheel centers'. We propose that a cortical network of feedforward and intracortical lateral connections elaborates a full set of optimum orientations from geniculate inputs that show a bias to stimulus orientation and form a set of two or a small number of 'Cartesian' coordinates. Because each geniculate afferent carries signals only from one eye and its receptive field (RF) is either ON or OFF center, the network constructs also ocular dominance columns and a quasi-segregation of ON and OFF responses across the horizontal extent of the striate cortex.

Predicting cortical dark/bright asymmetries from natural image statistics and early visual transforms

The nervous system has evolved in an environment with structure and predictability. One of the ubiquitous principles of sensory systems is the creation of circuits that capitalize on this predictability. Previous work has identified predictable non-uniformities in the distributions of basic visual features in natural images that are relevant to the encoding tasks of the visual system. Here, we report that the well-established statistical distributions of visual features -- such as visual contrast, spatial scale, and depth -- differ between bright and dark image components. Following this analysis, we go on to trace how these differences in natural images translate into different patterns of cortical input that arise from the separate bright (ON) and dark (OFF) pathways originating in the retina. We use models of these early visual pathways to transform natural images into statistical patterns of cortical input. The models include the receptive fields and non-linear response properties of the magnocellular (M) and parvocellular (P) pathways, with their ON and OFF pathway divisions. The results indicate that there are regularities in visual cortical input beyond those that have previously been appreciated from the direct analysis of natural images. In particular, several dark/bright asymmetries provide a potential account for recently discovered asymmetries in how the brain processes visual features, such as violations of classic energy-type models. On the basis of our analysis, we expect that the dark/bright dichotomy in natural images plays a key role in the generation of both cortical and perceptual asymmetries.

Sensory systems must contend with a tremendous amount of diversity in the natural world. Gaining a detailed description of the natural world’s statistical regularities is a critical part of understanding how the nervous system is adapted to its environment. Here, we report that the well-established statistical distributions of basic visual features—such as visual contrast and spatial scale—diverge when separated into bright and dark components. Operations such as dark/bright segregation are key features of early visual pathways. By modeling these pathways, we demonstrate that the dark and bright visual patterns driving cortical networks are asymmetric across a number of visual features, producing previously unappreciated second-order regularities. The results provide a parsimonious account for recently discovered asymmetries in cortical activity.

Subcortical orientation biases explain orientation selectivity of visual cortical cells

The primary visual cortex of carnivores and primates shows an orderly progression of domains of neurons that are selective to a particular orientation of visual stimuli such as bars and gratings. We recorded from single-thalamic afferent fibers that terminate in these domains to address the issue whether the orientation sensitivity of these fibers could form the basis of the remarkable orientation selectivity exhibited by most cortical cells. We first performed optical imaging of intrinsic signals to obtain a map of orientation domains on the dorsal aspect of the anaesthetized cat's area 17. After confirming using electrophysiological recordings the orientation preferences of single neurons within one or two domains in each animal, we pharmacologically silenced the cortex to leave only the afferent terminals active. The inactivation of cortical neurons was achieved by the superfusion of either kainic acid or muscimol. Responses of single geniculate afferents were then recorded by the use of high impedance electrodes. We found that the orientation preferences of the afferents matched closely with those of the cells in the orientation domains that they terminated in (Pearson's r = 0.633, n = 22, P = 0.002). This suggests a possible subcortical origin for cortical orientation selectivity.