1
|
Konkle T, Alvarez GA. A self-supervised domain-general learning framework for human ventral stream representation. Nat Commun 2022; 13:491. [PMID: 35078981 PMCID: PMC8789817 DOI: 10.1038/s41467-022-28091-4] [Citation(s) in RCA: 28] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Accepted: 12/13/2021] [Indexed: 12/25/2022] Open
Abstract
Anterior regions of the ventral visual stream encode substantial information about object categories. Are top-down category-level forces critical for arriving at this representation, or can this representation be formed purely through domain-general learning of natural image structure? Here we present a fully self-supervised model which learns to represent individual images, rather than categories, such that views of the same image are embedded nearby in a low-dimensional feature space, distinctly from other recently encountered views. We find that category information implicitly emerges in the local similarity structure of this feature space. Further, these models learn hierarchical features which capture the structure of brain responses across the human ventral visual stream, on par with category-supervised models. These results provide computational support for a domain-general framework guiding the formation of visual representation, where the proximate goal is not explicitly about category information, but is instead to learn unique, compressed descriptions of the visual world.
Collapse
Affiliation(s)
- Talia Konkle
- Department of Psychology & Center for Brain Science, Harvard University, Cambridge, MA, USA.
| | - George A Alvarez
- Department of Psychology & Center for Brain Science, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
2
|
Kosakowski HL, Cohen MA, Takahashi A, Keil B, Kanwisher N, Saxe R. Selective responses to faces, scenes, and bodies in the ventral visual pathway of infants. Curr Biol 2022; 32:265-274.e5. [PMID: 34784506 PMCID: PMC8792213 DOI: 10.1016/j.cub.2021.10.064] [Citation(s) in RCA: 30] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2021] [Revised: 09/27/2021] [Accepted: 10/28/2021] [Indexed: 01/26/2023]
Abstract
Three of the most robust functional landmarks in the human brain are the selective responses to faces in the fusiform face area (FFA), scenes in the parahippocampal place area (PPA), and bodies in the extrastriate body area (EBA). Are the selective responses of these regions present early in development or do they require many years to develop? Prior evidence leaves this question unresolved. We designed a new 32-channel infant magnetic resonance imaging (MRI) coil and collected high-quality functional MRI (fMRI) data from infants (2-9 months of age) while they viewed stimuli from four conditions-faces, bodies, objects, and scenes. We find that infants have face-, scene-, and body-selective responses in the location of the adult FFA, PPA, and EBA, respectively, powerfully constraining accounts of cortical development.
Collapse
Affiliation(s)
- Heather L Kosakowski
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, USA.
| | - Michael A Cohen
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, USA; Department of Psychology and Program in Neuroscience, Amherst College, 220 South Pleasant Street, Amherst, MA, USA
| | - Atsushi Takahashi
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, USA
| | - Boris Keil
- Institute of Medical Physics and Radiation Protection, Department of Life Science Engineering, Mittelhessen University of Applied Science, Giessen, Germany
| | - Nancy Kanwisher
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, USA
| | - Rebecca Saxe
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, USA
| |
Collapse
|
3
|
OUP accepted manuscript. Cereb Cortex 2022; 32:4913-4933. [DOI: 10.1093/cercor/bhab524] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Revised: 12/16/2021] [Accepted: 12/17/2021] [Indexed: 11/12/2022] Open
|
4
|
de Haas B, Sereno MI, Schwarzkopf DS. Inferior Occipital Gyrus Is Organized along Common Gradients of Spatial and Face-Part Selectivity. J Neurosci 2021; 41:5511-5521. [PMID: 34016715 PMCID: PMC8221599 DOI: 10.1523/jneurosci.2415-20.2021] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2020] [Revised: 04/02/2021] [Accepted: 04/05/2021] [Indexed: 11/21/2022] Open
Abstract
The ventral visual stream of the human brain is subdivided into patches with categorical stimulus preferences, like faces or scenes. However, the functional organization within these areas is less clear. Here, we used functional magnetic resonance imaging and vertex-wise tuning models to independently probe spatial and face-part preferences in the inferior occipital gyrus (IOG) of healthy adult males and females. The majority of responses were well explained by Gaussian population tuning curves for both retinotopic location and the preferred relative position within a face. Parameter maps revealed a common gradient of spatial and face-part selectivity, with the width of tuning curves drastically increasing from posterior to anterior IOG. Tuning peaks clustered more idiosyncratically but were also correlated across maps of visual and face space. Preferences for the upper visual field went along with significantly increased coverage of the upper half of the face, matching recently discovered biases in human perception. Our findings reveal a broad range of neural face-part selectivity in IOG, ranging from narrow to "holistic." IOG is functionally organized along this gradient, which in turn is correlated with retinotopy.SIGNIFICANCE STATEMENT Brain imaging has revealed a lot about the large-scale organization of the human brain and visual system. For example, occipital cortex contains map-like representations of the visual field, while neurons in ventral areas cluster into patches with categorical preferences, like faces or scenes. Much less is known about the functional organization within these areas. Here, we focused on a well established face-preferring area-the inferior occipital gyrus (IOG). A novel neuroimaging paradigm allowed us to map the retinotopic and face-part tuning of many recording sites in IOG independently. We found a steep posterior-anterior gradient of decreasing face-part selectivity, which correlated with retinotopy. This suggests the functional role of ventral areas is not uniform and may follow retinotopic "protomaps."
Collapse
Affiliation(s)
- Benjamin de Haas
- Department of Psychology, Justus Liebig Universität, 35394 Giessen, Germany
- Experimental Psychology, University College London, London WC1E 6BT, United Kingdom
| | - Martin I Sereno
- Experimental Psychology, University College London, London WC1E 6BT, United Kingdom
- SDSU Imaging Center, San Diego State University, San Diego, California 92182
| | - D Samuel Schwarzkopf
- Experimental Psychology, University College London, London WC1E 6BT, United Kingdom
- School of Optometry and Vision Science, University of Auckland, Auckland 1142, New Zealand
| |
Collapse
|
5
|
DiNicola LM, Buckner RL. Precision Estimates of Parallel Distributed Association Networks: Evidence for Domain Specialization and Implications for Evolution and Development. Curr Opin Behav Sci 2021; 40:120-129. [PMID: 34263017 DOI: 10.1016/j.cobeha.2021.03.029] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
Humans can reason about other minds, comprehend language and imagine. These abilities depend on association regions that exhibit evolutionary expansion and prolonged postnatal development. Precision maps within individuals reveal these expanded zones are populated by multiple specialized networks that each possess a spatially distributed motif but remain anatomically separated throughout the cortex for language, social and mnemonic / spatial functions. Rather than converge on multi-domain regions or hubs, these networks include distinct regions within rostral prefrontal and temporal association zones. To account for these observations, we propose the expansion-fractionation-specialization (EFS) hypothesis: evolutionary expansion of human association cortex may have allowed for an archetype distributed network to fractionate into multiple specialized networks. Human development may recapitulate fractionation and specialization when these abilities emerge.
Collapse
Affiliation(s)
- Lauren M DiNicola
- Department of Psychology, Center for Brain Science, Harvard University, Cambridge, MA 02138 USA
| | - Randy L Buckner
- Department of Psychology, Center for Brain Science, Harvard University, Cambridge, MA 02138 USA.,Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129 USA.,Department of Psychiatry, Massachusetts General Hospital, Charlestown, MA 02129 USA
| |
Collapse
|
6
|
Maffei A, Sessa P. Event-related network changes unfold the dynamics of cortical integration during face processing. Psychophysiology 2021; 58:e13786. [PMID: 33550632 DOI: 10.1111/psyp.13786] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Revised: 01/18/2021] [Accepted: 01/22/2021] [Indexed: 11/28/2022]
Abstract
Face perception arises from a collective activation of brain regions in the occipital, parietal and temporal cortices. Despite the wide acknowledgment that these regions act in an intertwined network, the network behavior itself is poorly understood. Here we present a study in which time-varying connectivity estimated from EEG activity elicited by facial expressions presentation was characterized using graph-theoretical measures of node centrality and global network topology. Results revealed that face perception results from a dynamic reshaping of the network architecture, characterized by the emergence of hubs located in the occipital and temporal regions of the scalp. The importance of these nodes can be observed from the early stages of visual processing and reaches a climax in the same time-window in which the face-sensitive N170 is observed. Furthermore, using Granger causality, we found that the time-evolving centrality of these nodes is associated with ERP amplitude, providing a direct link between the network state and local neural response. Additionally, investigating global network topology by means of small-worldness and modularity, we found that face processing requires a functional network with a strong small-world organization that maximizes integration, at the cost of segregated subdivisions. Interestingly, we found that this architecture is not static, but instead, it is implemented by the network from stimulus onset to ~200 ms. Altogether, this study reveals the event-related changes underlying face processing at the network level, suggesting that a distributed processing mechanism operates through dynamically weighting the contribution of the cortical regions involved.
Collapse
Affiliation(s)
- Antonio Maffei
- Padova Neuroscience Center (PNC), University of Padova, Padova, Italy
| | - Paola Sessa
- Padova Neuroscience Center (PNC), University of Padova, Padova, Italy.,Department of Developmental and Social Psychology, University of Padova, Padova, Italy
| |
Collapse
|
7
|
Raz G, Saxe R. Learning in Infancy Is Active, Endogenously Motivated, and Depends on the Prefrontal Cortices. ACTA ACUST UNITED AC 2020. [DOI: 10.1146/annurev-devpsych-121318-084841] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
A common view of learning in infancy emphasizes the role of incidental sensory experiences from which increasingly abstract statistical regularities are extracted. In this view, infant brains initially support basic sensory and motor functions, followed by maturation of higher-level association cortex. Here, we critique this view and posit that, by contrast and more like adults, infants are active, endogenously motivated learners who structure their own learning through flexible selection of attentional targets and active interventions on their environment. We further argue that the infant brain, and particularly the prefrontal cortex (PFC), is well equipped to support these learning behaviors. We review recent progress in characterizing the function of the infant PFC, which suggests that, as in adults, the PFC is functionally specialized and highly connected. Together, we present an integrative account of infant minds and brains, in which the infant PFC represents multiple intrinsic motivations, which are leveraged for active learning.
Collapse
Affiliation(s)
- Gal Raz
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
| | - Rebecca Saxe
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
| |
Collapse
|
8
|
Behrmann M, Plaut DC. Hemispheric Organization for Visual Object Recognition: A Theoretical Account and Empirical Evidence. Perception 2020; 49:373-404. [PMID: 31980013 PMCID: PMC9944149 DOI: 10.1177/0301006619899049] [Citation(s) in RCA: 40] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Despite the similarity in structure, the hemispheres of the human brain have somewhat different functions. A traditional view of hemispheric organization asserts that there are independent and largely lateralized domain-specific regions in ventral occipitotemporal (VOTC), specialized for the recognition of distinct classes of objects. Here, we offer an alternative account of the organization of the hemispheres, with a specific focus on face and word recognition. This alternative account relies on three computational principles: distributed representations and knowledge, cooperation and competition between representations, and topography and proximity. The crux is that visual recognition results from a network of regions with graded functional specialization that is distributed across both hemispheres. Specifically, the claim is that face recognition, which is acquired relatively early in life, is processed by VOTC regions in both hemispheres. Once literacy is acquired, word recognition, which is co-lateralized with language areas, primarily engages the left VOTC and, consequently, face recognition is primarily, albeit not exclusively, mediated by the right VOTC. We review psychological and neural evidence from a range of studies conducted with normal and brain-damaged adults and children and consider findings which challenge this account. Last, we offer suggestions for future investigations whose findings may further refine this account.
Collapse
Affiliation(s)
- Marlene Behrmann
- Department of Psychology and Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - David C. Plaut
- Department of Psychology and Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| |
Collapse
|
9
|
Connectivity at the origins of domain specificity in the cortical face and place networks. Proc Natl Acad Sci U S A 2020; 117:6163-6169. [PMID: 32123077 DOI: 10.1073/pnas.1911359117] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022] Open
Abstract
It is well established that the adult brain contains a mosaic of domain-specific networks. But how do these domain-specific networks develop? Here we tested the hypothesis that the brain comes prewired with connections that precede the development of domain-specific function. Using resting-state fMRI in the youngest sample of newborn humans tested to date, we indeed found that cortical networks that will later develop strong face selectivity (including the "proto" occipital face area and fusiform face area) and scene selectivity (including the "proto" parahippocampal place area and retrosplenial complex) by adulthood, already show domain-specific patterns of functional connectivity as early as 27 d of age (beginning as early as 6 d of age). Furthermore, we asked how these networks are functionally connected to early visual cortex and found that the proto face network shows biased functional connectivity with foveal V1, while the proto scene network shows biased functional connectivity with peripheral V1. Given that faces are almost always experienced at the fovea, while scenes always extend across the entire periphery, these differential inputs may serve to facilitate domain-specific processing in each network after that function develops, or even guide the development of domain-specific function in each network in the first place. Taken together, these findings reveal domain-specific and eccentricity-biased connectivity in the earliest days of life, placing new constraints on our understanding of the origins of domain-specific cortical networks.
Collapse
|
10
|
Op de Beeck HP, Pillet I, Ritchie JB. Factors Determining Where Category-Selective Areas Emerge in Visual Cortex. Trends Cogn Sci 2019; 23:784-797. [PMID: 31327671 DOI: 10.1016/j.tics.2019.06.006] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2019] [Revised: 06/21/2019] [Accepted: 06/21/2019] [Indexed: 11/26/2022]
Abstract
A hallmark of functional localization in the human brain is the presence of areas in visual cortex specialized for representing particular categories such as faces and words. Why do these areas appear where they do during development? Recent findings highlight several general factors to consider when answering this question. Experience-driven category selectivity arises in regions that have: (i) pre-existing selectivity for properties of the stimulus, (ii) are appropriately placed in the computational hierarchy of the visual system, and (iii) exhibit domain-specific patterns of connectivity to nonvisual regions. In other words, cortical location of category selectivity is constrained by what category will be represented, how it will be represented, and why the representation will be used.
Collapse
Affiliation(s)
- Hans P Op de Beeck
- Department of Brain and Cognition and Leuven Brain Institute, KU Leuven, Belgium. @kuleuven.be
| | - Ineke Pillet
- Department of Brain and Cognition and Leuven Brain Institute, KU Leuven, Belgium
| | - J Brendan Ritchie
- Department of Brain and Cognition and Leuven Brain Institute, KU Leuven, Belgium
| |
Collapse
|
11
|
Silson EH, Steel A, Kidder A, Gilmore AW, Baker CI. Distinct subdivisions of human medial parietal cortex support recollection of people and places. eLife 2019; 8:47391. [PMID: 31305238 PMCID: PMC6667275 DOI: 10.7554/elife.47391] [Citation(s) in RCA: 56] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2019] [Accepted: 07/13/2019] [Indexed: 12/11/2022] Open
Abstract
Human medial parietal cortex (MPC) is implicated in multiple cognitive processes including memory recall, visual scene processing and navigation, and is a core component of the default mode network. Here, we demonstrate distinct subdivisions of MPC that are selectively recruited during memory recall of either specific people or places. First, distinct regions of MPC exhibited differential functional connectivity with medial and lateral regions of ventral temporal cortex (VTC). Second, these same medial regions showed selective, but negative, responses to the visual presentation of different stimulus categories, with clear preferences for scenes and faces. Finally, and most critically, these regions were differentially recruited during memory recall of either people or places with a strong familiarity advantage. Taken together, these data suggest that the organizing principle defining the medial-lateral axis of VTC is reflected in MPC, but in the context of memory recall.
Collapse
Affiliation(s)
- Edward H Silson
- Laboratory of Brain & Cognition, National Institute of Mental Health, Bethesda, United States
| | - Adam Steel
- Laboratory of Brain & Cognition, National Institute of Mental Health, Bethesda, United States.,Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Alexis Kidder
- Laboratory of Brain & Cognition, National Institute of Mental Health, Bethesda, United States
| | - Adrian W Gilmore
- Laboratory of Brain & Cognition, National Institute of Mental Health, Bethesda, United States
| | - Chris I Baker
- Laboratory of Brain & Cognition, National Institute of Mental Health, Bethesda, United States
| |
Collapse
|