1
|
Margalit E, Lee H, Finzi D, DiCarlo JJ, Grill-Spector K, Yamins DLK. A unifying framework for functional organization in early and higher ventral visual cortex. Neuron 2024; 112:2435-2451.e7. [PMID: 38733985 PMCID: PMC11257790 DOI: 10.1016/j.neuron.2024.04.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Revised: 12/08/2023] [Accepted: 04/15/2024] [Indexed: 05/13/2024]
Abstract
A key feature of cortical systems is functional organization: the arrangement of functionally distinct neurons in characteristic spatial patterns. However, the principles underlying the emergence of functional organization in the cortex are poorly understood. Here, we develop the topographic deep artificial neural network (TDANN), the first model to predict several aspects of the functional organization of multiple cortical areas in the primate visual system. We analyze the factors driving the TDANN's success and find that it balances two objectives: learning a task-general sensory representation and maximizing the spatial smoothness of responses according to a metric that scales with cortical surface area. In turn, the representations learned by the TDANN are more brain-like than in spatially unconstrained models. Finally, we provide evidence that the TDANN's functional organization balances performance with between-area connection length. Our results offer a unified principle for understanding the functional organization of the primate ventral visual system.
Collapse
Affiliation(s)
- Eshed Margalit
- Neurosciences Graduate Program, Stanford University, Stanford, CA 94305, USA.
| | - Hyodong Lee
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Dawn Finzi
- Department of Psychology, Stanford University, Stanford, CA 94305, USA; Department of Computer Science, Stanford University, Stanford, CA 94305, USA
| | - James J DiCarlo
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Center for Brains Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Kalanit Grill-Spector
- Department of Psychology, Stanford University, Stanford, CA 94305, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA 94305, USA
| | - Daniel L K Yamins
- Department of Psychology, Stanford University, Stanford, CA 94305, USA; Department of Computer Science, Stanford University, Stanford, CA 94305, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA 94305, USA
| |
Collapse
|
2
|
Zhao XN, Dong XS, Jiang DQ, Wu S, Tang SM, Yu C. Population coding for figure-ground texture segregation in macaque V1 and V4. Prog Neurobiol 2024; 240:102655. [PMID: 38969016 DOI: 10.1016/j.pneurobio.2024.102655] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Revised: 06/09/2024] [Accepted: 06/30/2024] [Indexed: 07/07/2024]
Abstract
Object recognition often involves the brain segregating objects from their surroundings. Neurophysiological studies of figure-ground texture segregation have yielded inconsistent results, particularly on whether V1 neurons can perform figure-ground texture segregation or just detect texture borders. To address this issue from a population perspective, we utilized two-photon calcium imaging to simultaneously record the responses of large samples of V1 and V4 neurons to figure-ground texture stimuli in awake, fixating macaques. The average response changes indicate that V1 neurons mainly detect texture borders, while V4 neurons are involved in figure-ground segregation. However, population analysis (SVM decoding of PCA-transformed neuronal responses) reveal that V1 neurons not only detect figure-ground borders, but also contribute to figure-ground texture segregation, although requiring substantially more principal components than V4 neurons to reach a 75 % decoding accuracy. Individually, V1/V4 neurons showing larger (negative/positive) figure-ground response differences contribute more to figure-ground segregation. But for V1 neurons, the contribution becomes significant only when many principal components are considered. We conclude that V1 neurons participate in figure-ground segregation primarily by defining the figure borders, and the poorly structured figure-ground information V1 neurons carry could be further utilized by V4 neurons to accomplish figure-ground segregation.
Collapse
Affiliation(s)
- Xing-Nan Zhao
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China; PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, China
| | - Xing-Si Dong
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China; PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, China
| | - Dan-Qing Jiang
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China; PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, China
| | - Si Wu
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China; PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, China; IDG-McGovern Institute for Brain Research, Peking University, Beijing, China
| | - Shi-Ming Tang
- PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, China; IDG-McGovern Institute for Brain Research, Peking University, Beijing, China; School of Life Sciences, Peking University, Beijing, China.
| | - Cong Yu
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China; IDG-McGovern Institute for Brain Research, Peking University, Beijing, China.
| |
Collapse
|
3
|
Zhang SH, Zhao XN, Jiang DQ, Tang SM, Yu C. Ocular dominance-dependent binocular combination of monocular neuronal responses in macaque V1. eLife 2024; 13:RP92839. [PMID: 38568729 PMCID: PMC10990486 DOI: 10.7554/elife.92839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/05/2024] Open
Abstract
Primates rely on two eyes to perceive depth, while maintaining stable vision when either one eye or both eyes are open. Although psychophysical and modeling studies have investigated how monocular signals are combined to form binocular vision, the underlying neuronal mechanisms, particularly in V1 where most neurons exhibit binocularity with varying eye preferences, remain poorly understood. Here, we used two-photon calcium imaging to compare the monocular and binocular responses of thousands of simultaneously recorded V1 superficial-layer neurons in three awake macaques. During monocular stimulation, neurons preferring the stimulated eye exhibited significantly stronger responses compared to those preferring both eyes. However, during binocular stimulation, the responses of neurons preferring either eye were suppressed on the average, while those preferring both eyes were enhanced, resulting in similar neuronal responses irrespective of their eye preferences, and an overall response level similar to that with monocular viewing. A neuronally realistic model of binocular combination, which incorporates ocular dominance-dependent divisive interocular inhibition and binocular summation, is proposed to account for these findings.
Collapse
Affiliation(s)
- Sheng-Hui Zhang
- School of Psychological and Cognitive Sciences, Peking UniversityBeijingChina
- PKU-Tsinghua Center for Life Sciences, Peking UniversityBeijingChina
| | - Xing-Nan Zhao
- School of Psychological and Cognitive Sciences, Peking UniversityBeijingChina
- PKU-Tsinghua Center for Life Sciences, Peking UniversityBeijingChina
| | - Dan-Qing Jiang
- School of Psychological and Cognitive Sciences, Peking UniversityBeijingChina
- PKU-Tsinghua Center for Life Sciences, Peking UniversityBeijingChina
| | - Shi-Ming Tang
- PKU-Tsinghua Center for Life Sciences, Peking UniversityBeijingChina
- School of Life Sciences, Peking UniversityBeijingChina
- IDG-McGovern Institute for Brain Research, Peking UniversityBeijingChina
| | - Cong Yu
- School of Psychological and Cognitive Sciences, Peking UniversityBeijingChina
- IDG-McGovern Institute for Brain Research, Peking UniversityBeijingChina
| |
Collapse
|
4
|
Margalit E, Lee H, Finzi D, DiCarlo JJ, Grill-Spector K, Yamins DLK. A Unifying Principle for the Functional Organization of Visual Cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.18.541361. [PMID: 37292946 PMCID: PMC10245753 DOI: 10.1101/2023.05.18.541361] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
A key feature of many cortical systems is functional organization: the arrangement of neurons with specific functional properties in characteristic spatial patterns across the cortical surface. However, the principles underlying the emergence and utility of functional organization are poorly understood. Here we develop the Topographic Deep Artificial Neural Network (TDANN), the first unified model to accurately predict the functional organization of multiple cortical areas in the primate visual system. We analyze the key factors responsible for the TDANN's success and find that it strikes a balance between two specific objectives: achieving a task-general sensory representation that is self-supervised, and maximizing the smoothness of responses across the cortical sheet according to a metric that scales relative to cortical surface area. In turn, the representations learned by the TDANN are lower dimensional and more brain-like than those in models that lack a spatial smoothness constraint. Finally, we provide evidence that the TDANN's functional organization balances performance with inter-area connection length, and use the resulting models for a proof-of-principle optimization of cortical prosthetic design. Our results thus offer a unified principle for understanding functional organization and a novel view of the functional role of the visual system in particular.
Collapse
Affiliation(s)
- Eshed Margalit
- Neurosciences Graduate Program, Stanford University, Stanford, CA 94305
| | - Hyodong Lee
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139
| | - Dawn Finzi
- Department of Psychology, Stanford University, Stanford, CA 94305
- Department of Computer Science, Stanford University, Stanford, CA 94305
| | - James J DiCarlo
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139
- Center for Brains Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA 02139
| | - Kalanit Grill-Spector
- Department of Psychology, Stanford University, Stanford, CA 94305
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA 94305
| | - Daniel L K Yamins
- Department of Psychology, Stanford University, Stanford, CA 94305
- Department of Computer Science, Stanford University, Stanford, CA 94305
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA 94305
| |
Collapse
|
5
|
Manenti GL, Dizaji AS, Schwiedrzik CM. Variability in training unlocks generalization in visual perceptual learning through invariant representations. Curr Biol 2023; 33:817-826.e3. [PMID: 36724782 DOI: 10.1016/j.cub.2023.01.011] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Revised: 12/24/2022] [Accepted: 01/06/2023] [Indexed: 02/03/2023]
Abstract
Stimulus and location specificity are long considered hallmarks of visual perceptual learning. This renders visual perceptual learning distinct from other forms of learning, where generalization can be more easily attained, and therefore unsuitable for practical applications, where generalization is key. Based on the hypotheses derived from the structure of the visual system, we test here whether stimulus variability can unlock generalization in perceptual learning. We train subjects in orientation discrimination, while we vary the amount of variability in a task-irrelevant feature, spatial frequency. We find that, independently of task difficulty, this manipulation enables generalization of learning to new stimuli and locations, while not negatively affecting the overall amount of learning on the task. We then use deep neural networks to investigate how variability unlocks generalization. We find that networks develop invariance to the task-irrelevant feature when trained with variable inputs. The degree of learned invariance strongly predicts generalization. A reliance on invariant representations can explain variability-induced generalization in visual perceptual learning. This suggests new targets for understanding the neural basis of perceptual learning in the higher-order visual cortex and presents an easy-to-implement modification of common training paradigms that may benefit practical applications.
Collapse
Affiliation(s)
- Giorgio L Manenti
- Neural Circuits and Cognition Lab, European Neuroscience Institute Göttingen, A Joint Initiative of the University Medical Center Göttingen and the Max Planck Society, Grisebachstraße 5, 37077 Göttingen, Germany; Perception and Plasticity Group, German Primate Center, Leibniz Institute for Primate Research, Kellnerweg 4, 37077 Göttingen, Germany; Systems Neuroscience Program, Graduate School for Neurosciences, Biophysics and Molecular Biosciences (GGNB), 37077 Göttingen, Germany
| | - Aslan S Dizaji
- Neural Circuits and Cognition Lab, European Neuroscience Institute Göttingen, A Joint Initiative of the University Medical Center Göttingen and the Max Planck Society, Grisebachstraße 5, 37077 Göttingen, Germany
| | - Caspar M Schwiedrzik
- Neural Circuits and Cognition Lab, European Neuroscience Institute Göttingen, A Joint Initiative of the University Medical Center Göttingen and the Max Planck Society, Grisebachstraße 5, 37077 Göttingen, Germany; Perception and Plasticity Group, German Primate Center, Leibniz Institute for Primate Research, Kellnerweg 4, 37077 Göttingen, Germany.
| |
Collapse
|
6
|
Ju NS, Guan SC, Tang SM, Yu C. Macaque V1 responses to 2nd-order contrast-modulated stimuli and the possible subcortical and cortical contributions. Prog Neurobiol 2022; 217:102315. [PMID: 35809761 DOI: 10.1016/j.pneurobio.2022.102315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Revised: 07/01/2022] [Accepted: 07/05/2022] [Indexed: 12/01/2022]
Abstract
Natural images comprise contours and boundaries defined by 1st-order luminance-modulated (LM) cues that are readily encoded by V1 neurons, and 2nd-order contrast-modulated (CM) cues that carry local, but not over-the-space, luminance changes. The neurophysiological foundations for CM processing remain unsolved. Here we used two-photon calcium imaging to demonstrate that V1 superficial-layer neurons respond to both LM and CM gratings in awake, fixating, macaques, with overall LM responses stronger than CM responses. Furthermore, adaptation experiments revealed that LM responses were similarly suppressed by LM and CM adaptation, with moderately larger effects by iso-orientation adaptation than by orthogonal adaptation, suggesting that LM and CM orientation responses likely share a strong orientation-non-selective subcortical origin. In contrast, CM responses were substantially more suppressed by iso-orientation than by orthogonal LM and CM adaptation, likely suggesting stronger orientation-specific intracortical influences for CM responses than for LM responses, besides shared orientation-non-selective subcortical influences. These results thus may indicate a subcortical-to-V1 filter-rectify-filter mechanism for CM processing: Local luminance changes in CM stimuli are initially encoded by orientation-non-selective subcortical neurons, and the outputs are half-wave rectified, and then summed by V1 neurons to signal CM orientation, which may be further substantially refined by intracortical influences.
Collapse
Affiliation(s)
- Nian-Sheng Ju
- School of Life Sciences, Peking University, Beijing, China
| | - Shu-Chen Guan
- PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, China
| | - Shi-Ming Tang
- School of Life Sciences, Peking University, Beijing, China; PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, China; IDG-McGovern Institute for Brain Research, Peking University, Beijing, China.
| | - Cong Yu
- PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, China; IDG-McGovern Institute for Brain Research, Peking University, Beijing, China; School of Psychological and Cognitive Sciences, Peking University, Beijing, China.
| |
Collapse
|