1
|
Srinath R, Ni AM, Marucci C, Cohen MR, Brainard DH. Orthogonal neural representations support perceptual judgements of natural stimuli. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.14.580134. [PMID: 38464018 PMCID: PMC10925131 DOI: 10.1101/2024.02.14.580134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
In natural behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on simple backgrounds. Natural viewing, however, carries a set of challenges that are inaccessible using artificial stimuli, including neural responses to background objects that are task-irrelevant. An emerging body of evidence suggests that the visual abilities of humans and animals can be modeled through the linear decoding of task-relevant information from visual cortex. This idea suggests the hypothesis that irrelevant features of a natural scene should impair performance on a visual task only if their neural representations intrude on the linear readout of the task relevant feature, as would occur if the representations of task-relevant and irrelevant features are not orthogonal in the underlying neural population. We tested this hypothesis using human psychophysics and monkey neurophysiology, in response to parametrically variable naturalistic stimuli. We demonstrate that 1) the neural representation of one feature (the position of a central object) in visual area V4 is orthogonal to those of several background features, 2) the ability of human observers to precisely judge object position was largely unaffected by task-irrelevant variation in those background features, and 3) many features of the object and the background are orthogonally represented by V4 neural responses. Our observations are consistent with the hypothesis that orthogonal neural representations can support stable perception of objects and features despite the tremendous richness of natural visual scenes.
Collapse
Affiliation(s)
- Ramanujan Srinath
- equal contribution
- Department of Neurobiology and Neuroscience Institute, The University of Chicago, Chicago, IL 60637, USA
| | - Amy M. Ni
- equal contribution
- Department of Neurobiology and Neuroscience Institute, The University of Chicago, Chicago, IL 60637, USA
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Claire Marucci
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Marlene R. Cohen
- Department of Neurobiology and Neuroscience Institute, The University of Chicago, Chicago, IL 60637, USA
- equal contribution
| | - David H. Brainard
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA
- equal contribution
| |
Collapse
|
2
|
Cadena SA, Willeke KF, Restivo K, Denfield G, Sinz FH, Bethge M, Tolias AS, Ecker AS. Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks. PLoS Comput Biol 2024; 20:e1012056. [PMID: 38781156 PMCID: PMC11115319 DOI: 10.1371/journal.pcbi.1012056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Accepted: 04/08/2024] [Indexed: 05/25/2024] Open
Abstract
Responses to natural stimuli in area V4-a mid-level area of the visual ventral stream-are well predicted by features from convolutional neural networks (CNNs) trained on image classification. This result has been taken as evidence for the functional role of V4 in object classification. However, we currently do not know if and to what extent V4 plays a role in solving other computational objectives. Here, we investigated normative accounts of V4 (and V1 for comparison) by predicting macaque single-neuron responses to natural images from the representations extracted by 23 CNNs trained on different computer vision tasks including semantic, geometric, 2D, and 3D types of tasks. We found that V4 was best predicted by semantic classification features and exhibited high task selectivity, while the choice of task was less consequential to V1 performance. Consistent with traditional characterizations of V4 function that show its high-dimensional tuning to various 2D and 3D stimulus directions, we found that diverse non-semantic tasks explained aspects of V4 function that are not captured by individual semantic tasks. Nevertheless, jointly considering the features of a pair of semantic classification tasks was sufficient to yield one of our top V4 models, solidifying V4's main functional role in semantic processing and suggesting that V4's selectivity to 2D or 3D stimulus properties found by electrophysiologists can result from semantic functional goals.
Collapse
Affiliation(s)
- Santiago A. Cadena
- Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Göttingen, Germany
- Institute for Theoretical Physics and Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany
- Bernstein Center for Computational Neuroscience, Tübingen, Germany
- International Max Planck Research School for Intelligent Systems, Tübingen, Germany
| | - Konstantin F. Willeke
- Bernstein Center for Computational Neuroscience, Tübingen, Germany
- International Max Planck Research School for Intelligent Systems, Tübingen, Germany
- Institute for Bioinformatics and Medical Informatics, University Tübingen, Tübingen, Germany
| | - Kelli Restivo
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, United States of America
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
| | - George Denfield
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, United States of America
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
| | - Fabian H. Sinz
- Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Göttingen, Germany
- Bernstein Center for Computational Neuroscience, Tübingen, Germany
- International Max Planck Research School for Intelligent Systems, Tübingen, Germany
- Institute for Bioinformatics and Medical Informatics, University Tübingen, Tübingen, Germany
| | - Matthias Bethge
- Institute for Theoretical Physics and Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany
- Bernstein Center for Computational Neuroscience, Tübingen, Germany
| | - Andreas S. Tolias
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, United States of America
- Department of Neuroscience, Baylor College of Medicine, Houston, Texas, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, Texas, United States of America
| | - Alexander S. Ecker
- Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Göttingen, Germany
- Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany
| |
Collapse
|
3
|
Jang H, Tong F. Improved modeling of human vision by incorporating robustness to blur in convolutional neural networks. Nat Commun 2024; 15:1989. [PMID: 38443349 PMCID: PMC10915141 DOI: 10.1038/s41467-024-45679-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2023] [Accepted: 01/30/2024] [Indexed: 03/07/2024] Open
Abstract
Whenever a visual scene is cast onto the retina, much of it will appear degraded due to poor resolution in the periphery; moreover, optical defocus can cause blur in central vision. However, the pervasiveness of blurry or degraded input is typically overlooked in the training of convolutional neural networks (CNNs). We hypothesized that the absence of blurry training inputs may cause CNNs to rely excessively on high spatial frequency information for object recognition, thereby causing systematic deviations from biological vision. We evaluated this hypothesis by comparing standard CNNs with CNNs trained on a combination of clear and blurry images. We show that blur-trained CNNs outperform standard CNNs at predicting neural responses to objects across a variety of viewing conditions. Moreover, blur-trained CNNs acquire increased sensitivity to shape information and greater robustness to multiple forms of visual noise, leading to improved correspondence with human perception. Our results provide multi-faceted neurocomputational evidence that blurry visual experiences may be critical for conferring robustness to biological visual systems.
Collapse
Affiliation(s)
- Hojin Jang
- Department of Psychology, Vanderbilt Vision Research Center, Vanderbilt University, Nashville, TN, USA.
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA.
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea.
| | - Frank Tong
- Department of Psychology, Vanderbilt Vision Research Center, Vanderbilt University, Nashville, TN, USA.
| |
Collapse
|
4
|
Liu G, Chen Y, Chen Y, Shi Y, Zhang M, Shen G, Qi P, Li J, Ma D, Yu F, Huang X. Indirect Electrocatalysis S─N/S─S Bond Construction by Robust Polyoxometalate Based Foams. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2023; 35:e2304716. [PMID: 37392073 DOI: 10.1002/adma.202304716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Revised: 06/23/2023] [Accepted: 06/29/2023] [Indexed: 07/02/2023]
Abstract
Indirect electrocatalytic conversion of cheap organic raw materials via the activation of S─H and N─H bonds into the value-added S─N/S─S bonds chemicals for industrial rubber production is a promising strategy to realize the atomic economic reaction, during which the kinetic inhibition that is associated with the electron transfer at the electrode/electrolyte interface in traditional direct electrocatalysis can be eliminated to achieve higher performance. In this work, a series of di-copper-substituted phosphotungstatebased foams (PW10 Cu2 @CMC) are fabricated with tunable loadings (17 to 44 wt%), which can be successfully applied in indirect electrocatalytic syntheses of sulfenamides and disulfides. Specifically, the optimal PW10 Cu2 @CMC (44 wt%) exhibits excellent electrocatalytic performance for the construction of S─N/S─S bonds (yields up to 99%) coupling with the efficient production of H2 (≈50 µmol g-1 h-1 ). Remarkably, it enables the scale-up production (≈14.4 g in a batch experiment) and the obtained products can serve as rubber vulcanization accelerators with superior properties to traditional industrial rubber additives in real industrial processes. This powerful catalysis system that can simultaneously produce rubber vulcanization accelerator and H2 may inaugurate a new electrocatalytic avenue to explore polyoxometalate-based foam catalysts in electrocatalysis field.
Collapse
Affiliation(s)
- Gang Liu
- Shandong Provincial Key Laboratory of Chemical Energy Storage and Novel Cell Technology, School of Chemistry & Chemical Engineering, Liaocheng University, Liaocheng, Shandong, 252059, P. R. China
| | - Yifa Chen
- National and Local Joint Engineering Research Center of MPTES in High Energy and Safety LIBs, Engineering Research Center of MTEES (Ministry of Education), Key Lab. of ETESPG(GHEI), School of Chemistry, South China Normal University, Guangzhou, 510006, P. R. China
| | - Yulu Chen
- National and Local Joint Engineering Research Center of MPTES in High Energy and Safety LIBs, Engineering Research Center of MTEES (Ministry of Education), Key Lab. of ETESPG(GHEI), School of Chemistry, South China Normal University, Guangzhou, 510006, P. R. China
| | - Yanqi Shi
- Shandong Provincial Key Laboratory of Chemical Energy Storage and Novel Cell Technology, School of Chemistry & Chemical Engineering, Liaocheng University, Liaocheng, Shandong, 252059, P. R. China
| | - Meiyu Zhang
- Shandong Provincial Key Laboratory of Chemical Energy Storage and Novel Cell Technology, School of Chemistry & Chemical Engineering, Liaocheng University, Liaocheng, Shandong, 252059, P. R. China
| | - Guodong Shen
- Shandong Provincial Key Laboratory of Chemical Energy Storage and Novel Cell Technology, School of Chemistry & Chemical Engineering, Liaocheng University, Liaocheng, Shandong, 252059, P. R. China
| | - Pengfei Qi
- Shandong Provincial Key Laboratory of Chemical Energy Storage and Novel Cell Technology, School of Chemistry & Chemical Engineering, Liaocheng University, Liaocheng, Shandong, 252059, P. R. China
| | - Jikun Li
- College of Chemistry and Chemical Engineering, Taishan University, Tai'an, Shandong, 271021, P. R. China
| | - Delong Ma
- National Rubber Additive Engineering Technology Center, Liaocheng, Shandong, 252059, P. R. China
| | - Fei Yu
- Jiangsu Collaborative Innovation Centre of Biomedical Functional Materials, Jiangsu Key Laboratory of New Power Batteries, School of Chemistry and Materials Science, Nanjing Normal University, Nanjing, 210023, P. R. China
| | - Xianqiang Huang
- Shandong Provincial Key Laboratory of Chemical Energy Storage and Novel Cell Technology, School of Chemistry & Chemical Engineering, Liaocheng University, Liaocheng, Shandong, 252059, P. R. China
| |
Collapse
|
5
|
Jang H, Tong F. Improved modeling of human vision by incorporating robustness to blur in convolutional neural networks. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.07.29.551089. [PMID: 37577646 PMCID: PMC10418076 DOI: 10.1101/2023.07.29.551089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/15/2023]
Abstract
Whenever a visual scene is cast onto the retina, much of it will appear degraded due to poor resolution in the periphery; moreover, optical defocus can cause blur in central vision. However, the pervasiveness of blurry or degraded input is typically overlooked in the training of convolutional neural networks (CNNs). We hypothesized that the absence of blurry training inputs may cause CNNs to rely excessively on high spatial frequency information for object recognition, thereby causing systematic deviations from biological vision. We evaluated this hypothesis by comparing standard CNNs with CNNs trained on a combination of clear and blurry images. We show that blur-trained CNNs outperform standard CNNs at predicting neural responses to objects across a variety of viewing conditions. Moreover, blur-trained CNNs acquire increased sensitivity to shape information and greater robustness to multiple forms of visual noise, leading to improved correspondence with human perception. Our results provide novel neurocomputational evidence that blurry visual experiences are very important for conferring robustness to biological visual systems.
Collapse
Affiliation(s)
- Hojin Jang
- Department of Psychology and Vanderbilt Vision Research Center Vanderbilt University
| | - Frank Tong
- Department of Psychology and Vanderbilt Vision Research Center Vanderbilt University
| |
Collapse
|
6
|
Vannuscorps G, Galaburda A, Caramazza A. From intermediate shape-centered representations to the perception of oriented shapes: response to commentaries. Cogn Neuropsychol 2023; 40:71-94. [PMID: 37642330 DOI: 10.1080/02643294.2023.2250511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 07/14/2023] [Accepted: 07/31/2023] [Indexed: 08/31/2023]
Abstract
In this response paper, we start by addressing the main points made by the commentators on the target article's main theoretical conclusions: the existence and characteristics of the intermediate shape-centered representations (ISCRs) in the visual system, their emergence from edge detection mechanisms operating on different types of visual properties, and how they are eventually reunited in higher order frames of reference underlying conscious visual perception. We also address the much-commented issue of the possible neural mechanisms of the ISCRs. In the final section, we address more specific and general comments, questions, and suggestions which, albeit very interesting, were less directly focused on the main conclusions of the target paper.
Collapse
Affiliation(s)
- Gilles Vannuscorps
- Department of Psychology, Harvard University, Cambridge, MA, USA
- Institute of Psychological Sciences, Université catholique de Louvain, Louvain-la-Neuve, Belgium
- Institute of Neuroscience, Université catholique de Louvain, Louvain-la-Neuve, Belgium
- Louvain Bionics, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Albert Galaburda
- Department of Neurology, Harvard Medical School and Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Alfonso Caramazza
- Department of Psychology, Harvard University, Cambridge, MA, USA
- Center for Mind/Brain Sciences (CIMeC), Università degli Studi di Trento, Rovereto, Italy
| |
Collapse
|
7
|
Shishikura M, Tamura H, Sakai K. Correlation between neural responses and human perception in figure-ground segregation. Front Syst Neurosci 2023; 16:999575. [PMID: 36713684 PMCID: PMC9877615 DOI: 10.3389/fnsys.2022.999575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Accepted: 12/28/2022] [Indexed: 01/13/2023] Open
Abstract
Segmentation of a natural scene into objects (figures) and background (ground) is one of crucial functions for object recognition and scene understanding. Recent studies have investigated neural mechanisms underlying figure-ground (FG) segregation and reported neural modulation to FG in the intermediate-level visual area, V4, of macaque monkeys (FG neurons). However, whether FG neurons contribute to the perception of FG segregation has not been clarified. To examine the contribution of FG neurons, we examined the correlations between perceptual consistency (PC), which quantified perceptual ambiguity in FG determination, and the reliability of neural signals in response to FG. First, we evaluated PCs for the images that were used in the previous neural recording in V4; specifically, we measured how consistently FG can be determined across trials and participants for each stimulus. The PCs were widely distributed, so that we identified the ambiguity in FG segregation for each stimulus. Next, we analyzed the correlation between the PCs and the reliability of neural modulation to FG. We found that the stimuli with higher PCs evoked more consistent and greater modulation in the responses of single neurons than those with lower PCs. Since perception is expected to show a greater correlation with responses of neural population compared to those of single neurons, we examined the correlation between the PCs and the consistency of the population responses in FG determination. Stimuli with higher PCs evoked higher population consistency than those with lower PCs. Finally, we analyzed the correlation between the PCs and neural latencies in FG modulation. We found that the stimuli with higher PCs showed shorter reaction times in FG perception and evoked shorter modulation latencies in FG neurons. These results indicate that the responses of FG neurons recorded from macaque monkeys show significant correlations with human FG perception, suggesting that V4 neurons with FG-dependent responses contribute to the perception of FG segregation.
Collapse
Affiliation(s)
| | - Hiroshi Tamura
- Graduate School of Frontier Biosciences, Osaka University, Suita, Japan,Center for Information and Neural Networks, Suita, Osaka, Japan
| | - Ko Sakai
- Department of Computer Science, University of Tsukuba, Tsukuba, Japan,*Correspondence: Ko Sakai,
| |
Collapse
|
8
|
Burk DC, Sheinberg DL. Neurons in inferior temporal cortex are sensitive to motion trajectory during degraded object recognition. Cereb Cortex Commun 2022; 3:tgac034. [PMID: 36168516 PMCID: PMC9499820 DOI: 10.1093/texcom/tgac034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 08/03/2022] [Accepted: 08/05/2022] [Indexed: 11/30/2022] Open
Abstract
Our brains continuously acquire sensory information and make judgments even when visual information is limited. In some circumstances, an ambiguous object can be recognized from how it moves, such as an animal hopping or a plane flying overhead. Yet it remains unclear how movement is processed by brain areas involved in visual object recognition. Here we investigate whether inferior temporal (IT) cortex, an area known for its relevance in visual form processing, has access to motion information during recognition. We developed a matching task that required monkeys to recognize moving shapes with variable levels of shape degradation. Neural recordings in area IT showed that, surprisingly, some IT neurons responded stronger to degraded shapes than clear ones. Furthermore, neurons exhibited motion sensitivity at different times during the presentation of the blurry target. Population decoding analyses showed that motion patterns could be decoded from IT neuron pseudo-populations. Contrary to previous findings, these results suggest that neurons in IT can integrate visual motion and shape information, particularly when shape information is degraded, in a way that has been previously overlooked. Our results highlight the importance of using challenging multifeature recognition tasks to understand the role of area IT in naturalistic visual object recognition.
Collapse
Affiliation(s)
- Diana C Burk
- Department of Neuroscience, Brown University , Providence, RI 02912 , United States
| | - David L Sheinberg
- Department of Neuroscience, Brown University , Providence, RI 02912 , United States
- Carney Institute for Brain Science, Brown University , Providence, RI 02912 , United States
| |
Collapse
|
9
|
Popovkina DV, Pasupathy A. Davida's deficits: weak encoding of impoverished stimuli or faulty egocentric representation? Cogn Neuropsychol 2022; 39:99-102. [PMID: 35676872 DOI: 10.1080/02643294.2022.2083947] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
Vannuscorps and colleagues present the fascinating case of Davida, a young person who makes systematic errors in judgments related to orientations of sharp or high-contrast visual stimuli. In this commentary, we discuss the findings in the context of observations from mid-level ventral visual stream physiology. We propose two additional interpretations for the specificity of the behavioural deficits: the observed impairments in orientation judgments may be consistent with a system that is not able to unambiguously represent certain impoverished stimuli, or with a system that is not able to translate visual input into head- or body-centered coordinates. Davida's case offers a unique glimpse into the complex cascade of transformations that enable accurate orientation judgments, and sparks curiosity about which mechanistic disruptions can produce such specific unstable percepts.
Collapse
Affiliation(s)
- Dina V Popovkina
- Department of Psychology, University of Washington, Seattle, WA, USA
| | - Anitha Pasupathy
- Department of Biological Structure, University of Washington, Seattle, WA, USA.,Washington National Primate Research Center, University of Washington, Seattle, WA, USA
| |
Collapse
|
10
|
Limballe A, Kulpa R, Bennett S. Using Blur for Perceptual Investigation and Training in Sport? A Clear Picture of the Evidence and Implications for Future Research. Front Psychol 2022; 12:752582. [PMID: 35308077 PMCID: PMC8926072 DOI: 10.3389/fpsyg.2021.752582] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Accepted: 12/24/2021] [Indexed: 11/13/2022] Open
Abstract
Dynamic, interactive sports require athletes to identify, pick-up and process relevant information in a very limited time, in order to then make an appropriate response. Perceptual-cognitive skills are, therefore, a key determinant of elite sporting performance. Recently, sport scientists have investigated ways to assess and train perceptual-cognitive skills, with one such method involving the use of blurred stimuli. Here, we describe the two main methods used to generate blur (i.e., dioptric and Gaussian) and then review the current findings in a sports context. Overall, it has been shown the use of blur can enhance performance and learning of sporting tasks in novice participants, especially when the blur is applied to peripheral stimuli. However, while intermediate and expert level participants are relatively impervious to the presence of blur, it remains to be determined if they are positive effects on learning. In a final section, we describe some of the methodological issues that limit the application of blur and then discuss the potential use of virtual reality to extend the current research base in sporting contexts.
Collapse
Affiliation(s)
| | | | - Simon Bennett
- School of Sport and Exercise Sciences, Liverpool John Moores University, Liverpool, United Kingdom
| |
Collapse
|
11
|
Vannuscorps G, Galaburda A, Caramazza A. Shape-centered representations of bounded regions of space mediate the perception of objects. Cogn Neuropsychol 2021; 39:1-50. [PMID: 34427539 DOI: 10.1080/02643294.2021.1960495] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
We report the study of a woman who perceives 2D bounded regions of space ("shapes") defined by sharp edges of medium to high contrast as if they were rotated by 90, 180 degrees around their centre, mirrored across their own axes, or both. In contrast, her perception of 3D, strongly blurred or very low contrast shapes, and of stimuli emerging from a collection of shapes, is intact. This suggests that a stage in the process of constructing the conscious visual representation of a scene consists of representing mutually exclusive bounded regions extracted from the initial retinotopic space in "shape-centered" frames of reference. The selectivity of the disorder to shapes originally biased toward the parvocellular subcortical pathway, and the absence of any other type of error, additionally invite new hypotheses about the operations involved in computing these "intermediate shape-centered representations" and in mapping them onto higher frames for perception and action.
Collapse
Affiliation(s)
- Gilles Vannuscorps
- Department of Psychology, Harvard University, Cambridge, MA, USA.,Institute of Psychological Sciences, Université catholique de Louvain, Louvain-la-Neuve, Belgium.,Institute of Neuroscience, Université catholique de Louvain, Louvain-la-Neuve, Belgium.,Louvain Bionics, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Albert Galaburda
- Department of Neurology, Harvard Medical School and Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Alfonso Caramazza
- Department of Psychology, Harvard University, Cambridge, MA, USA.,Center for Mind/Brain Sciences (CIMeC), Università degli Studi di Trento, Rovereto, Italy
| |
Collapse
|
12
|
Kawakami S, Ito T, Makino Y, Hashimoto M, Yano M. A cell model in the ventral visual pathway for the detection of circles of curvature constituting figures. Heliyon 2020; 6:e05397. [PMID: 33305029 PMCID: PMC7711303 DOI: 10.1016/j.heliyon.2020.e05397] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2020] [Revised: 07/04/2020] [Accepted: 10/28/2020] [Indexed: 11/01/2022] Open
Abstract
The contour of an arbitrary figure can be represented as a group of circles of curvature in contact with it, with each curvature circle represented by its center OC and radius r. We propose a series of cell models for detecting this circle, which is composed of a lateral geniculate nucleus (LGN) cell, nondirectionally selective (NDS) simple cell, and curvature-circle detection cell (CDC). The LGN and NDS simple cells were previously modeled. The CDC has been modeled as follows. Each tangent in contact with this circle is detected by an NDS simple cell that performs the Hough transformation of LGN cell responses, and then this tangent is transformed to a three-dimensional (3D) normal line in a CDC column. This transformation has been named a 3D normal-line transform. Performing this transformation for all tangents causes a CDC at the intersection of these normal lines to fire most intensively, and thus the OC and r of the circle is detected as the coordinates of this intersection. Therefore, the CDC has been modeled as this 3D normal-line transform. Based on this CDC, we model two types of constancy CDC: a position-invariant CDC and a curvature-invariant CDC. These three types of CDC reflect the response to various stimuli in actual area V4 cells. In order to validate these CDC types neurophysiologically, we propose an experimental method using microelectrodes. Cell models previously reported correspond to this hierarchy: the S1, S2, and C2 cells correspond to the NDS simple cell, CDC, and position-invariant CDC, respectively.
Collapse
Affiliation(s)
- Susumu Kawakami
- Tohoku University, Research Institute of Electrical Communication, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan
| | - Takehiro Ito
- Tohoku University, Research Institute of Electrical Communication, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan
| | - Yoshinari Makino
- Tohoku University, Research Institute of Electrical Communication, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan.,Tohoku Gakuin University, Department of Information Science, Faculty of Liberal Arts, 2-1-1 Tenjinzawa, Izumi-ku, Sendai 981-3193, Japan
| | - Makoto Hashimoto
- Tohoku University, Research Institute of Electrical Communication, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan
| | - Masafumi Yano
- Tohoku University, Research Institute of Electrical Communication, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan.,Tohoku Institute of Technology, 35-1 Yagiyamakasumichou, Taihaku-ku, Sendai 982-8577, Japan
| |
Collapse
|
13
|
Abstract
Area V4-the focus of this review-is a mid-level processing stage along the ventral visual pathway of the macaque monkey. V4 is extensively interconnected with other visual cortical areas along the ventral and dorsal visual streams, with frontal cortical areas, and with several subcortical structures. Thus, it is well poised to play a broad and integrative role in visual perception and recognition-the functional domain of the ventral pathway. Neurophysiological studies in monkeys engaged in passive fixation and behavioral tasks suggest that V4 responses are dictated by tuning in a high-dimensional stimulus space defined by form, texture, color, depth, and other attributes of visual stimuli. This high-dimensional tuning may underlie the development of object-based representations in the visual cortex that are critical for tracking, recognizing, and interacting with objects. Neurophysiological and lesion studies also suggest that V4 responses are important for guiding perceptual decisions and higher-order behavior.
Collapse
Affiliation(s)
- Anitha Pasupathy
- Department of Biological Structure, University of Washington, Seattle, Washington 98195, USA; ,
- Washington National Primate Research Center, University of Washington, Seattle, Washington 98121, USA
| | - Dina V Popovkina
- Department of Psychology, University of Washington, Seattle, Washington 98105, USA;
| | - Taekjun Kim
- Department of Biological Structure, University of Washington, Seattle, Washington 98195, USA; ,
- Washington National Primate Research Center, University of Washington, Seattle, Washington 98121, USA
| |
Collapse
|
14
|
Zamarashkina P, Popovkina DV, Pasupathy A. Timing of response onset and offset in macaque V4: stimulus and task dependence. J Neurophysiol 2020; 123:2311-2325. [PMID: 32401171 PMCID: PMC7311726 DOI: 10.1152/jn.00586.2019] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 05/06/2020] [Accepted: 05/07/2020] [Indexed: 11/22/2022] Open
Abstract
In the primate visual cortex, both the magnitude of the neuronal response and its timing can carry important information about the visual world, but studies typically focus only on response magnitude. Here, we examine the onset and offset latency of the responses of neurons in area V4 of awake, behaving macaques across several experiments in the context of a variety of stimuli and task paradigms. Our results highlight distinct contributions of stimuli and tasks to V4 response latency. We found that response onset latencies are shorter than typically cited (median = 75.5 ms), supporting a role for V4 neurons in rapid object and scene recognition functions. Moreover, onset latencies are longer for smaller stimuli and stimulus outlines, consistent with the hypothesis that longer latencies are associated with higher spatial frequency content. Strikingly, we found that onset latencies showed no significant dependence on stimulus occlusion, unlike in inferotemporal cortex, nor on task demands. Across the V4 population, onset latencies had a broad distribution, reflecting the diversity of feedforward, recurrent, and feedback connections that inform the responses of individual neurons. Response offset latencies, on the other hand, displayed the opposite tendency in their relationship to stimulus and task attributes: they are less influenced by stimulus appearance but are shorter in guided saccade tasks compared with fixation tasks. The observation that response latency is influenced by stimulus- and task-associated factors emphasizes a need to examine response timing alongside firing rate in determining the functional role of area V4.NEW & NOTEWORTHY Onset and offset timing of neuronal responses can provide information about visual environment and neuron's role in visual processing and its anatomical connectivity. In the first comprehensive examination of onset and offset latencies in the intermediate visual cortical area V4, we find neurons respond faster than previously reported, making them ideally suited to contribute to rapid object and scene recognition. While response onset reflects stimulus characteristics, timing of response offset is influenced more by behavioral task.
Collapse
Affiliation(s)
- Polina Zamarashkina
- Department of Biological Structure, University of Washington, Washington National Primate Research Center, Seattle, Washington
| | - Dina V Popovkina
- Department of Psychology, University of Washington, Seattle, Washington
| | - Anitha Pasupathy
- Department of Biological Structure, University of Washington, Washington National Primate Research Center, Seattle, Washington
| |
Collapse
|
15
|
Goddard E, Mullen KT. fMRI representational similarity analysis reveals graded preferences for chromatic and achromatic stimulus contrast across human visual cortex. Neuroimage 2020; 215:116780. [PMID: 32276074 DOI: 10.1016/j.neuroimage.2020.116780] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2019] [Revised: 03/18/2020] [Accepted: 03/24/2020] [Indexed: 01/23/2023] Open
Abstract
Human visual cortex is partitioned into different functional areas that, from lower to higher, become increasingly selective and responsive to complex feature dimensions. Here we use a Representational Similarity Analysis (RSA) of fMRI-BOLD signals to make quantitative comparisons across LGN and multiple visual areas of the low-level stimulus information encoded in the patterns of voxel responses. Our stimulus set was picked to target the four functionally distinct subcortical channels that input visual cortex from the LGN: two achromatic sinewave stimuli that favor the responses of the high-temporal magnocellular and high-spatial parvocellular pathways, respectively, and two chromatic stimuli isolating the L/M-cone opponent and S-cone opponent pathways, respectively. Each stimulus type had three spatial extents to sample both foveal and para-central visual field. With the RSA, we compare quantitatively the response specializations for individual stimuli and combinations of stimuli in each area and how these change across visual cortex. First, our results replicate the known response preferences for motion/flicker in the dorsal visual areas. In addition, we identify two distinct gradients along the ventral visual stream. In the early visual areas (V1-V3), the strongest differential representation is for the achromatic high spatial frequency stimuli, suitable for form vision, and a very weak differentiation of chromatic versus achromatic contrast. Emerging in ventral occipital areas (V4, VO1 and VO2), however, is an increasingly strong separation of the responses to chromatic versus achromatic contrast and a decline in the high spatial frequency representation. These gradients provide new insight into how visual information is transformed across the visual cortex.
Collapse
Affiliation(s)
- Erin Goddard
- McGill Vision Research, Department of Ophthalmology & Visual Sciences, McGill University, Montreal, QC, H3G1A4, Canada
| | - Kathy T Mullen
- McGill Vision Research, Department of Ophthalmology & Visual Sciences, McGill University, Montreal, QC, H3G1A4, Canada.
| |
Collapse
|
16
|
Zheng W, Gou C, Wang FY. A novel approach inspired by optic nerve characteristics for few-shot occluded face recognition. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2019.09.045] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
17
|
Object shape and surface properties are jointly encoded in mid-level ventral visual cortex. Curr Opin Neurobiol 2019; 58:199-208. [PMID: 31586749 DOI: 10.1016/j.conb.2019.09.009] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2019] [Revised: 08/30/2019] [Accepted: 09/11/2019] [Indexed: 11/22/2022]
Abstract
Recognizing a myriad visual objects rapidly is a hallmark of the primate visual system. Traditional theories of object recognition have focused on how crucial form features, for example, the orientation of edges, may be extracted in early visual cortex and utilized to recognize objects. An alternative view argues that much of early and mid-level visual processing focuses on encoding surface characteristics, for example, texture. Neurophysiological evidence from primate area V4 supports a third alternative - the joint, but independent, encoding of form and texture - that would be advantageous for segmenting objects from the background in natural scenes and for object recognition that is independent of surface texture. Future studies that leverage deep convolutional network models, especially focusing on network failures to match biology and behavior, can advance our insights into how such a joint representation of form and surface properties might emerge in visual cortex.
Collapse
|
18
|
Vision: Dialogues between Deep Networks and the Brain. Curr Biol 2019; 29:R634-R637. [DOI: 10.1016/j.cub.2019.05.072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
19
|
MacCormick IJC, Williams BM, Zheng Y, Li K, Al-Bander B, Czanner S, Cheeseman R, Willoughby CE, Brown EN, Spaeth GL, Czanner G. Accurate, fast, data efficient and interpretable glaucoma diagnosis with automated spatial analysis of the whole cup to disc profile. PLoS One 2019; 14:e0209409. [PMID: 30629635 PMCID: PMC6328156 DOI: 10.1371/journal.pone.0209409] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2018] [Accepted: 12/05/2018] [Indexed: 11/25/2022] Open
Abstract
Background Glaucoma is the leading cause of irreversible blindness worldwide. It is a heterogeneous group of conditions with a common optic neuropathy and associated loss of peripheral vision. Both over and under-diagnosis carry high costs in terms of healthcare spending and preventable blindness. The characteristic clinical feature of glaucoma is asymmetrical optic nerve rim narrowing, which is difficult for humans to quantify reliably. Strategies to improve and automate optic disc assessment are therefore needed to prevent sight loss. Methods We developed a novel glaucoma detection algorithm that segments and analyses colour photographs to quantify optic nerve rim consistency around the whole disc at 15-degree intervals. This provides a profile of the cup/disc ratio, in contrast to the vertical cup/disc ratio in common use. We introduce a spatial probabilistic model, to account for the optic nerve shape, we then use this model to derive a disc deformation index and a decision rule for glaucoma. We tested our algorithm on two separate image datasets (ORIGA and RIM-ONE). Results The spatial algorithm accurately distinguished glaucomatous and healthy discs on internal and external validation (AUROC 99.6% and 91.0% respectively). It achieves this using a dataset 100-times smaller than that required for deep learning algorithms, is flexible to the type of cup and disc segmentation (automated or semi-automated), utilises images with missing data, and is correlated with the disc size (p = 0.02) and the rim-to-disc at the narrowest rim (p<0.001, in external validation). Discussion The spatial probabilistic algorithm is highly accurate, highly data efficient and it extends to any imaging hardware in which the boundaries of cup and disc can be segmented, thus making the algorithm particularly applicable to research into disease mechanisms, and also glaucoma screening in low resource settings.
Collapse
Affiliation(s)
- Ian J. C. MacCormick
- Department of Eye & Vision Science, Institute of Ageing and Chronic Disease, University of Liverpool, Liverpool, United Kingdom
- Centre for Clinical Brain Sciences, University of Edinburgh, Chancellor's Building, Edinburgh, United Kingdom
| | - Bryan M. Williams
- Department of Eye & Vision Science, Institute of Ageing and Chronic Disease, University of Liverpool, Liverpool, United Kingdom
| | - Yalin Zheng
- Department of Eye & Vision Science, Institute of Ageing and Chronic Disease, University of Liverpool, Liverpool, United Kingdom
- St Paul’s Eye Unit, Royal Liverpool University Hospitals NHS Trust, Liverpool, United Kingdom
| | - Kun Li
- Medical Information Engineering Department, Taishan Medical School, TaiAn City, ShanDong Province, China
| | - Baidaa Al-Bander
- Department of Electrical Engineering and Electronics, University of Liverpool, Brownlow Hill, Liverpool, United Kingdom
| | - Silvester Czanner
- School of Computing, Mathematics and Digital Technology, Faculty of Science and Engineering, Manchester Metropolitan University, Manchester, Manchester, United Kingdom
| | - Rob Cheeseman
- St Paul’s Eye Unit, Royal Liverpool University Hospitals NHS Trust, Liverpool, United Kingdom
| | - Colin E. Willoughby
- Biomedical Sciences Research Institute, Faculty of Life & Health Sciences, Ulster University, Coleraine, Northern Ireland
- Department of Ophthalmology, Royal Victoria Hospital, Belfast, Northern Ireland
| | - Emery N. Brown
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
- Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| | - George L. Spaeth
- Glaucoma Research Center, Wills Eye Hospital, Philadelphia, Pennsylvania, United States of America
| | - Gabriela Czanner
- Department of Eye & Vision Science, Institute of Ageing and Chronic Disease, University of Liverpool, Liverpool, United Kingdom
- St Paul’s Eye Unit, Royal Liverpool University Hospitals NHS Trust, Liverpool, United Kingdom
- Department of Applied Mathematics, Faculty of Engineering and Technology, Liverpool John Moores University, Liverpool, United Kingdom
- * E-mail:
| |
Collapse
|