1
|
Bi Z, Li H, Tian L. Top-down generation of low-resolution representations improves visual perception and imagination. Neural Netw 2024; 171:440-456. [PMID: 38150870 DOI: 10.1016/j.neunet.2023.12.030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Revised: 11/30/2023] [Accepted: 12/18/2023] [Indexed: 12/29/2023]
Abstract
Perception or imagination requires top-down signals from high-level cortex to primary visual cortex (V1) to reconstruct or simulate the representations bottom-up stimulated by the seen images. Interestingly, top-down signals in V1 have lower spatial resolution than bottom-up representations. It is unclear why the brain uses low-resolution signals to reconstruct or simulate high-resolution representations. By modeling the top-down pathway of the visual system using the decoder of a variational auto-encoder (VAE), we reveal that low-resolution top-down signals can better reconstruct or simulate the information contained in the sparse activities of V1 simple cells, which facilitates perception and imagination. This advantage of low-resolution generation is related to facilitating high-level cortex to form geometry-respecting representations observed in experiments. Furthermore, we present two findings regarding this phenomenon in the context of AI-generated sketches, a style of drawings made of lines. First, we found that the quality of the generated sketches critically depends on the thickness of the lines in the sketches: thin-line sketches are harder to generate than thick-line sketches. Second, we propose a technique to generate high-quality thin-line sketches: instead of directly using original thin-line sketches, we use blurred sketches to train VAE or GAN (generative adversarial network), and then infer the thin-line sketches from the VAE- or GAN-generated blurred sketches. Collectively, our work suggests that low-resolution top-down generation is a strategy the brain uses to improve visual perception and imagination, which inspires new sketch-generation AI techniques.
Collapse
Affiliation(s)
- Zedong Bi
- Lingang Laboratory, Shanghai 200031, China.
| | - Haoran Li
- Department of Physics, Hong Kong Baptist University, Hong Kong, China
| | - Liang Tian
- Department of Physics, Hong Kong Baptist University, Hong Kong, China; Institute of Computational and Theoretical Studies, Hong Kong Baptist University, Hong Kong, China; Institute of Systems Medicine and Health Sciences, Hong Kong Baptist University, Hong Kong, China; State Key Laboratory of Environmental and Biological Analysis, Hong Kong Baptist University, Hong Kong, China.
| |
Collapse
|
2
|
Dai W, Wang T, Li Y, Yang Y, Zhang Y, Kang J, Wu Y, Yu H, Xing D. Dynamic Recruitment of the Feedforward and Recurrent Mechanism for Black-White Asymmetry in the Primary Visual Cortex. J Neurosci 2023; 43:5668-5684. [PMID: 37487737 PMCID: PMC10401654 DOI: 10.1523/jneurosci.0168-23.2023] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2023] [Revised: 07/11/2023] [Accepted: 07/14/2023] [Indexed: 07/26/2023] Open
Abstract
Black and white information is asymmetrically distributed in natural scenes, evokes asymmetric neuronal responses, and causes asymmetric perceptions. Recognizing the universality and essentiality of black-white asymmetry in visual information processing, the neural substrates for black-white asymmetry remain unclear. To disentangle the role of the feedforward and recurrent mechanisms in the generation of cortical black-white asymmetry, we recorded the V1 laminar responses and LGN responses of anesthetized cats of both sexes. In a cortical column, we found that black-white asymmetry starts at the input layer and becomes more pronounced in the output layer. We also found distinct dynamics of black-white asymmetry between the output layer and the input layer. Specifically, black responses dominate in all layers after stimulus onset. After stimulus offset, black and white responses are balanced in the input layer, but black responses still dominate in the output layer. Compared with that in the input layer, the rebound response in the output layer is significantly suppressed. The relative suppression strength evoked by white stimuli is notably stronger and depends on the location within the ON-OFF cortical map. A model with delayed and polarity-selective cortical suppression explains black-white asymmetry in the output layer, within which prominent recurrent connections are identified by Granger causality analysis. In addition to black-white asymmetry in response strength, the interlaminar differences in spatial receptive field varied dynamically. Our findings suggest that the feedforward and recurrent mechanisms are dynamically recruited for the generation of black-white asymmetry in V1.SIGNIFICANCE STATEMENT Black-white asymmetry is universal and essential in visual information processing, yet the neural substrates for cortical black-white asymmetry remain unknown. Leveraging V1 laminar recordings, we provided the first laminar pattern of black-white asymmetry in cat V1 and found distinct dynamics of black-white asymmetry between the output layer and the input layer. Comparing black-white asymmetry across three visual hierarchies, the LGN, V1 input layer, and V1 output layer, we demonstrated that the feedforward and recurrent mechanisms are dynamically recruited for the generation of cortical black-white asymmetry. Our findings not only enhance our understanding of laminar processing within a cortical column but also elucidate how feedforward connections and recurrent connections interact to shape neuronal response properties.
Collapse
Affiliation(s)
- Weifeng Dai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Tian Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
- College of Life Sciences, Beijing Normal University, Beijing, 100875, China
| | - Yang Li
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Yi Yang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Yange Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Jian Kang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Yujie Wu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Hongbo Yu
- School of Life Sciences, State Key Laboratory of Medical Neurobiology, Collaborative Innovation Center for Brain Science, Fudan University, Shanghai, 200438, China
| | - Dajun Xing
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| |
Collapse
|
3
|
St-Amand D, Baker CL. Model-Based Approach Shows ON Pathway Afferents Elicit a Transient Decrease of V1 Responses. J Neurosci 2023; 43:1920-1932. [PMID: 36759194 PMCID: PMC10027028 DOI: 10.1523/jneurosci.1220-22.2023] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Revised: 01/29/2023] [Accepted: 01/30/2023] [Indexed: 02/11/2023] Open
Abstract
Neurons in the primary visual cortex (V1) receive excitation and inhibition from distinct parallel pathways processing lightness (ON) and darkness (OFF). V1 neurons overall respond more strongly to dark than light stimuli, consistent with a preponderance of darker regions in natural images, as well as human psychophysics. However, it has been unclear whether this "dark-dominance" is because of more excitation from the OFF pathway or more inhibition from the ON pathway. To understand the mechanisms behind dark-dominance, we record electrophysiological responses of individual simple-type V1 neurons to natural image stimuli and then train biologically inspired convolutional neural networks to predict the neurons' responses. Analyzing a sample of 71 neurons (in anesthetized, paralyzed cats of either sex) has revealed their responses to be more driven by dark than light stimuli, consistent with previous investigations. We show that this asymmetry is predominantly because of slower inhibition to dark stimuli rather than to stronger excitation from the thalamocortical OFF pathway. Consistent with dark-dominant neurons having faster responses than light-dominant neurons, we find dark-dominance to solely occur in the early latencies of neurons' responses. Neurons that are strongly dark-dominated also tend to be less orientation-selective. This novel approach gives us new insight into the dark-dominance phenomenon and provides an avenue to address new questions about excitatory and inhibitory integration in cortical neurons.SIGNIFICANCE STATEMENT Neurons in the early visual cortex respond on average more strongly to dark than to light stimuli, but the mechanisms behind this bias have been unclear. Here we address this issue by combining single-unit electrophysiology with a novel machine learning model to analyze neurons' responses to natural image stimuli in primary visual cortex. Using these techniques, we find slower inhibition to light than to dark stimuli to be the leading mechanism behind stronger dark responses. This slower inhibition to light might help explain other empirical findings, such as why orientation selectivity is weaker at earlier response latencies. These results demonstrate how imbalances in excitation versus inhibition can give rise to response asymmetries in cortical neuron responses.
Collapse
Affiliation(s)
- David St-Amand
- McGill Vision Research Unit, Department of Ophthalmology & Visual Sciences, McGill University, Montreal, Quebec H3G 1A4, Canada
| | - Curtis L Baker
- McGill Vision Research Unit, Department of Ophthalmology & Visual Sciences, McGill University, Montreal, Quebec H3G 1A4, Canada
| |
Collapse
|
4
|
Luminance Contrast Shifts Dominance Balance between ON and OFF Pathways in Human Vision. J Neurosci 2023; 43:993-1007. [PMID: 36535768 PMCID: PMC9908321 DOI: 10.1523/jneurosci.1672-22.2022] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Revised: 11/14/2022] [Accepted: 12/10/2022] [Indexed: 12/23/2022] Open
Abstract
Human vision processes light and dark stimuli in visual scenes with separate ON and OFF neuronal pathways. In nature, stimuli lighter or darker than their local surround have different spatial properties and contrast distributions (Ratliff et al., 2010; Cooper and Norcia, 2015; Rahimi-Nasrabadi et al., 2021). Similarly, in human vision, we show that luminance contrast affects the perception of lights and darks differently. At high contrast, human subjects of both sexes locate dark stimuli faster and more accurately than light stimuli, which is consistent with a visual system dominated by the OFF pathway. However, at low contrast, they locate light stimuli faster and more accurately than dark stimuli, which is consistent with a visual system dominated by the ON pathway. Luminance contrast was strongly correlated with multiple ON/OFF dominance ratios estimated from light/dark ratios of performance errors, missed targets, or reaction times (RTs). All correlations could be demonstrated at multiple eccentricities of the central visual field with an ON-OFF perimetry test implemented in a head-mounted visual display. We conclude that high-contrast stimuli are processed faster and more accurately by OFF pathways than ON pathways. However, the OFF dominance shifts toward ON dominance when stimulus contrast decreases, as expected from the higher-contrast sensitivity of ON cortical pathways (Kremkow et al., 2014; Rahimi-Nasrabadi et al., 2021). The results highlight the importance of contrast polarity in visual field measurements and predict a loss of low-contrast vision in humans with ON pathway deficits, as demonstrated in animal models (Sarnaik et al., 2014).SIGNIFICANCE STATEMENT ON and OFF retino-thalamo-cortical pathways respond differently to luminance contrast. In both animal models and humans, low contrasts drive stronger responses from ON pathways, whereas high contrasts drive stronger responses from OFF pathways. We demonstrate that these ON-OFF pathway differences have a correlate in human vision. At low contrast, humans locate light targets faster and more accurately than dark targets but, as contrast increases, dark targets become more visible than light targets. We also demonstrate that contrast is strongly correlated with multiple light/dark ratios of visual performance in central vision. These results provide a link between neuronal physiology and human vision while emphasizing the importance of stimulus polarity in measurements of visual fields and contrast sensitivity.
Collapse
|
5
|
Abstract
The primary visual cortex signals the onset of light and dark stimuli with ON and OFF cortical pathways. Here, we demonstrate that both pathways generate similar response increments to large homogeneous surfaces and their response average increases with surface brightness. We show that, in cat visual cortex, response dominance from ON or OFF pathways is bimodally distributed when stimuli are smaller than one receptive field center but unimodally distributed when they are larger. Moreover, whereas small bright stimuli drive opposite responses from ON and OFF pathways (increased versus suppressed activity), large bright surfaces drive similar response increments. We show that this size-brightness relation emerges because strong illumination increases the size of light surfaces in nature and both ON and OFF cortical neurons receive input from ON thalamic pathways. We conclude that visual scenes are perceived as brighter when the average response increments from ON and OFF cortical pathways become stronger. Mazade et al. find that the visual cortex encodes brightness differently for small than large stimuli. Bright small stimuli drive cortical pathways signaling lights and suppress cortical pathways signaling darks. Conversely, large surfaces drive response increments from both pathways and appear brightest when the response average is strongest.
Collapse
|
6
|
Yang Y, Wang T, Li Y, Dai W, Yang G, Han C, Wu Y, Xing D. Coding strategy for surface luminance switches in the primary visual cortex of the awake monkey. Nat Commun 2022; 13:286. [PMID: 35022404 PMCID: PMC8755737 DOI: 10.1038/s41467-021-27892-3] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Accepted: 12/22/2021] [Indexed: 11/17/2022] Open
Abstract
Both surface luminance and edge contrast of an object are essential features for object identification. However, cortical processing of surface luminance remains unclear. In this study, we aim to understand how the primary visual cortex (V1) processes surface luminance information across its different layers. We report that edge-driven responses are stronger than surface-driven responses in V1 input layers, but luminance information is coded more accurately by surface responses. In V1 output layers, the advantage of edge over surface responses increased eight times and luminance information was coded more accurately at edges. Further analysis of neural dynamics shows that such substantial changes for neural responses and luminance coding are mainly due to non-local cortical inhibition in V1’s output layers. Our results suggest that non-local cortical inhibition modulates the responses elicited by the surfaces and edges of objects, and that switching the coding strategy in V1 promotes efficient coding for luminance. How brightness is encoded in the visual cortex remains incompletely understood. By recording from macaque V1, the authors revealed a switch from surface to edge encoding that is mediated by widespread inhibition in the output layers of the cortex.
Collapse
Affiliation(s)
- Yi Yang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Tian Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, College of Life Sciences, Beijing Normal University, Beijing, China
| | - Yang Li
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Weifeng Dai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Guanzhong Yang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Chuanliang Han
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Yujie Wu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Dajun Xing
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China.
| |
Collapse
|
7
|
Bereshpolova Y, Hei X, Alonso JM, Swadlow HA. Three rules govern thalamocortical connectivity of fast-spike inhibitory interneurons in the visual cortex. eLife 2020; 9:60102. [PMID: 33289630 PMCID: PMC7723404 DOI: 10.7554/elife.60102] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2020] [Accepted: 11/30/2020] [Indexed: 11/13/2022] Open
Abstract
Some cortical neurons receive highly selective thalamocortical (TC) input, but others do not. Here, we examine connectivity of single thalamic neurons (lateral geniculate nucleus, LGN) onto putative fast-spike inhibitory interneurons in layer 4 of rabbit visual cortex. We show that three 'rules' regulate this connectivity. These rules concern: (1) the precision of retinotopic alignment, (2) the amplitude of the postsynaptic local field potential elicited near the interneuron by spikes of the LGN neuron, and (3) the interneuron's response latency to strong, synchronous LGN input. We found that virtually all first-order fast-spike interneurons receive input from nearly all LGN axons that synapse nearby, regardless of their visual response properties. This was not the case for neighboring regular-spiking neurons. We conclude that profuse and highly promiscuous TC inputs to layer-4 fast-spike inhibitory interneurons generate response properties that are well-suited to mediate a fast, sensitive, and broadly tuned feed-forward inhibition of visual cortical excitatory neurons.
Collapse
Affiliation(s)
- Yulia Bereshpolova
- Department of Psychological Sciences, University of Connecticut, Storrs, United States
| | - Xiaojuan Hei
- Department of Psychological Sciences, University of Connecticut, Storrs, United States
| | - Jose-Manuel Alonso
- Department of Psychological Sciences, University of Connecticut, Storrs, United States.,Department of Biological and Vision Sciences, State University of New York College of Optometry, New York, United States
| | - Harvey A Swadlow
- Department of Psychological Sciences, University of Connecticut, Storrs, United States.,Department of Biological and Vision Sciences, State University of New York College of Optometry, New York, United States
| |
Collapse
|
8
|
Qiu S, Caldwell C, You J, Mendola J. Binocular rivalry from luminance and contrast. Vision Res 2020; 175:41-50. [DOI: 10.1016/j.visres.2020.06.006] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2020] [Revised: 06/21/2020] [Accepted: 06/24/2020] [Indexed: 11/16/2022]
|
9
|
Mazade R, Jin J, Pons C, Alonso JM. Functional Specialization of ON and OFF Cortical Pathways for Global-Slow and Local-Fast Vision. Cell Rep 2020; 27:2881-2894.e5. [PMID: 31167135 DOI: 10.1016/j.celrep.2019.05.007] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2018] [Revised: 03/07/2019] [Accepted: 04/30/2019] [Indexed: 12/20/2022] Open
Abstract
Visual information is processed in the cortex by ON and OFF pathways that respond to light and dark stimuli. Responses to darks are stronger, faster, and driven by a larger number of cortical neurons than responses to lights. Here, we demonstrate that these light-dark cortical asymmetries reflect a functional specialization of ON and OFF pathways for different stimulus properties. We show that large long-lasting stimuli drive stronger cortical responses when they are light, whereas small fast stimuli drive stronger cortical responses when they are dark. Moreover, we show that these light-dark asymmetries are preserved under a wide variety of luminance conditions that range from photopic to low mesopic light. Our results suggest that ON and OFF pathways extract different spatiotemporal information from visual scenes, making OFF local-fast signals better suited to maximize visual acuity and ON global-slow signals better suited to guide the eye movements needed for retinal image stabilization.
Collapse
Affiliation(s)
- Reece Mazade
- Department of Biological and Visual Sciences, SUNY College of Optometry, New York, NY 10036, USA
| | - Jianzhong Jin
- Department of Biological and Visual Sciences, SUNY College of Optometry, New York, NY 10036, USA
| | - Carmen Pons
- Department of Biological and Visual Sciences, SUNY College of Optometry, New York, NY 10036, USA
| | - Jose-Manuel Alonso
- Department of Biological and Visual Sciences, SUNY College of Optometry, New York, NY 10036, USA.
| |
Collapse
|
10
|
Roy A, Osik JJ, Meschede-Krasa B, Alford WT, Leman DP, Van Hooser SD. Synaptic and intrinsic mechanisms underlying development of cortical direction selectivity. eLife 2020; 9:e58509. [PMID: 32701059 PMCID: PMC7440916 DOI: 10.7554/elife.58509] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2020] [Accepted: 07/23/2020] [Indexed: 02/02/2023] Open
Abstract
Modifications of synaptic inputs and cell-intrinsic properties both contribute to neuronal plasticity and development. To better understand these mechanisms, we undertook an intracellular analysis of the development of direction selectivity in the ferret visual cortex, which occurs rapidly over a few days after eye opening. We found strong evidence of developmental changes in linear spatiotemporal receptive fields of simple cells, implying alterations in circuit inputs. Further, this receptive field plasticity was accompanied by increases in near-spike-threshold excitability and input-output gain that resulted in dramatically increased spiking responses in the experienced state. Increases in subthreshold membrane responses induced by the receptive field plasticity and the increased input-output spiking gain were both necessary to explain the elevated firing rates in experienced ferrets. These results demonstrate that cortical direction selectivity develops through a combination of plasticity in inputs and in cell-intrinsic properties.
Collapse
Affiliation(s)
- Arani Roy
- Department of Biology, Brandeis UniversityWalthamUnited States
- Volen Center for Complex Systems, Brandeis UniversityWalthamUnited States
| | - Jason J Osik
- Department of Biology, Brandeis UniversityWalthamUnited States
- Volen Center for Complex Systems, Brandeis UniversityWalthamUnited States
| | | | - Wesley T Alford
- Department of Biology, Brandeis UniversityWalthamUnited States
| | - Daniel P Leman
- Department of Biology, Brandeis UniversityWalthamUnited States
| | - Stephen D Van Hooser
- Department of Biology, Brandeis UniversityWalthamUnited States
- Volen Center for Complex Systems, Brandeis UniversityWalthamUnited States
- Sloan-Swartz Center for Theoretical Neurobiology Brandeis UniversityWalthamUnited States
| |
Collapse
|
11
|
Abstract
To model the responses of neurons in the early visual system, at least three basic components are required: a receptive field, a normalization term, and a specification of encoding noise. Here, we examine how the receptive field, the normalization factor, and the encoding noise affect the drive to model-neuron responses when stimulated with natural images. We show that when these components are modeled appropriately, the response drives elicited by natural stimuli are Gaussian-distributed and scale invariant, and very nearly maximize the sensitivity (d') for natural-image discrimination. We discuss the statistical models of natural stimuli that can account for these response statistics, and we show how some commonly used modeling practices may distort these results. Finally, we show that normalization can equalize important properties of neural response across different stimulus types. Specifically, narrowband (stimulus- and feature-specific) normalization causes model neurons to yield Gaussian response-drive statistics when stimulated with natural stimuli, 1/f noise stimuli, and white-noise stimuli. The current work makes recommendations for best practices and lays a foundation, grounded in the response statistics to natural stimuli, upon which to build principled models of more complex visual tasks.
Collapse
Affiliation(s)
- Arvind Iyer
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
| | - Johannes Burge
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA.,Neuroscience Graduate Group, University of Pennsylvania, Philadelphia, PA, USA.,Bioengineering Graduate Group, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
12
|
Jansen M, Jin J, Li X, Lashgari R, Kremkow J, Bereshpolova Y, Swadlow HA, Zaidi Q, Alonso JM. Cortical Balance Between ON and OFF Visual Responses Is Modulated by the Spatial Properties of the Visual Stimulus. Cereb Cortex 2020; 29:336-355. [PMID: 30321290 PMCID: PMC6294412 DOI: 10.1093/cercor/bhy221] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2018] [Indexed: 11/12/2022] Open
Abstract
The primary visual cortex of carnivores and primates is dominated by the OFF visual pathway and responds more strongly to dark than light stimuli. Here, we demonstrate that this cortical OFF dominance is modulated by the size and spatial frequency of the stimulus in awake primates and we uncover a main neuronal mechanism underlying this modulation. We show that large grating patterns with low spatial frequencies drive five times more OFF-dominated than ON-dominated neurons, but this pronounced cortical OFF dominance is strongly reduced when the grating size decreases and the spatial frequency increases, as when the stimulus moves away from the observer. We demonstrate that the reduction in cortical OFF dominance is not caused by a selective reduction of visual responses in OFF-dominated neurons but by a change in the ON/OFF response balance of neurons with diverse receptive field properties that can be ON or OFF dominated, simple, or complex. We conclude that cortical OFF dominance is continuously adjusted by a neuronal mechanism that modulates ON/OFF response balance in multiple cortical neurons when the spatial properties of the visual stimulus change with viewing distance and/or optical blur.
Collapse
Affiliation(s)
- Michael Jansen
- Department of Biological and Vision Sciences, Biol. Sci., SUNY College of Optometry, New York, NY, USA
| | - Jianzhong Jin
- Department of Biological and Vision Sciences, Biol. Sci., SUNY College of Optometry, New York, NY, USA
| | - Xiaobing Li
- Department of Biological and Vision Sciences, Biol. Sci., SUNY College of Optometry, New York, NY, USA
| | - Reza Lashgari
- Department of Biological and Vision Sciences, Biol. Sci., SUNY College of Optometry, New York, NY, USA.,Brain Engineering Research Center, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran
| | - Jens Kremkow
- Department of Biological and Vision Sciences, Biol. Sci., SUNY College of Optometry, New York, NY, USA.,Neuroscience Research Center, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | | | - Harvey A Swadlow
- Department of Biological and Vision Sciences, Biol. Sci., SUNY College of Optometry, New York, NY, USA.,Department of Psychology, University of Connecticut, Storrs, CT, USA
| | - Qasim Zaidi
- Department of Biological and Vision Sciences, Biol. Sci., SUNY College of Optometry, New York, NY, USA
| | - Jose-Manuel Alonso
- Department of Biological and Vision Sciences, Biol. Sci., SUNY College of Optometry, New York, NY, USA.,Department of Psychology, University of Connecticut, Storrs, CT, USA
| |
Collapse
|
13
|
Abstract
AbstractConvolutional neural networks (CNNs) lack robustness to test image corruptions that are not seen during training. In this paper, we propose a new layer for CNNs that increases their robustness to several types of corruptions of the input images. We call it a ‘push–pull’ layer and compute its response as the combination of two half-wave rectified convolutions, with kernels of different size and opposite polarity. Its implementation is based on a biologically motivated model of certain neurons in the visual system that exhibit response suppression, known as push–pull inhibition. We validate our method by replacing the first convolutional layer of the LeNet, ResNet and DenseNet architectures with our push–pull layer. We train the networks on original training images from the MNIST and CIFAR data sets and test them on images with several corruptions, of different types and severities, that are unseen by the training process. We experiment with various configurations of the ResNet and DenseNet models on a benchmark test set with typical image corruptions constructed on the CIFAR test images. We demonstrate that our push–pull layer contributes to a considerable improvement in robustness of classification of corrupted images, while maintaining state-of-the-art performance on the original image classification task. We released the code and trained models at the url http://github.com/nicstrisc/Push-Pull-CNN-layer.
Collapse
|
14
|
A model for the origin and development of visual orientation selectivity. PLoS Comput Biol 2019; 15:e1007254. [PMID: 31356590 PMCID: PMC6687209 DOI: 10.1371/journal.pcbi.1007254] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2018] [Revised: 08/08/2019] [Accepted: 07/09/2019] [Indexed: 12/14/2022] Open
Abstract
Orientation selectivity is a key property of primary visual cortex that contributes, downstream, to object recognition. The origin of orientation selectivity, however, has been debated for decades. It is known that on- and off-centre subcortical pathways converge onto single neurons in primary visual cortex, and that the spatial offset between these pathways gives rise to orientation selectivity. On- and off-centre pathways are intermingled, however, so it is unclear how their inputs to cortex come to be spatially segregated. We here describe a model in which the segregation occurs through Hebbian strengthening and weakening of geniculocortical synapses during the development of the visual system. Our findings include the following. 1. Neighbouring on- and off-inputs to cortex largely cancelled each other at the start of development. At each receptive field location, the Hebbian process increased the strength of one input sign at the expense of the other sign, producing a spatial segregation of on- and off-inputs. 2. The resulting orientation selectivity was precise in that the bandwidths of the orientation tuning functions fell within empirical estimates. 3. The model produced maps of preferred orientation–complete with iso-orientation domains and pinwheels–similar to those found in real cortex. 4. These maps did not originate in cortical processes, but from clustering of off-centre subcortical pathways and the relative location of neighbouring on-centre clusters. We conclude that a model with intermingled on- and off-pathways shaped by Hebbian synaptic plasticity can explain both the origin and development of orientation selectivity. Many neurons in mammalian primary visual cortex are highly selective for the orientation of visual contours and can therefore contribute to object recognition. Orientation selectivity depends on on- and off-centre retinal neurons that respond, respectively, to light and dark. We describe a signal-processing model that includes both subcortical pathways and cortical neurons. The model predicts the preferred orientation of a cortical neuron from the empirically determined spatial layout of retinal cells. Further, the subcortical-to-cortical connections change in strength during visual development, meaning that cortical neurons in the model have orientation selectivity just as precise as real neurons. Our model can therefore explain the origin of orientation selectivity and the way it develops during visual system maturation.
Collapse
|