1
|
Dipani A, McNeal N, Ratan Murty NA. Linking faces to social cognition: The temporal pole as a potential social switch. Proc Natl Acad Sci U S A 2024; 121:e2411735121. [PMID: 39024106 DOI: 10.1073/pnas.2411735121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/20/2024] Open
Affiliation(s)
- Alish Dipani
- Cognition and Brain Science, School of Psychology, Georgia Institute of Technology, Atlanta, GA 30332
- Center of Excellence in Computational Cognition, Georgia Institute of Technology, Atlanta, GA 30332
| | - Nikolas McNeal
- School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332
| | - N Apurva Ratan Murty
- Cognition and Brain Science, School of Psychology, Georgia Institute of Technology, Atlanta, GA 30332
- Center of Excellence in Computational Cognition, Georgia Institute of Technology, Atlanta, GA 30332
| |
Collapse
|
2
|
Sun Y, Men W, Kennerknecht I, Fang W, Zheng HF, Zhang W, Rao Y. Human genetics of face recognition: discovery of MCTP2 mutations in humans with face blindness (congenital prosopagnosia). Genetics 2024; 227:iyae047. [PMID: 38547502 DOI: 10.1093/genetics/iyae047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Accepted: 03/19/2024] [Indexed: 06/06/2024] Open
Abstract
Face recognition is important for both visual and social cognition. While prosopagnosia or face blindness has been known for seven decades and face-specific neurons for half a century, the molecular genetic mechanism is not clear. Here we report results after 17 years of research with classic genetics and modern genomics. From a large family with 18 congenital prosopagnosia (CP) members with obvious difficulties in face recognition in daily life, we uncovered a fully cosegregating private mutation in the MCTP2 gene which encodes a calcium binding transmembrane protein expressed in the brain. After screening through cohorts of 6589, we found more CPs and their families, allowing detection of more CP associated mutations in MCTP2. Face recognition differences were detected between 14 carriers with the frameshift mutation S80fs in MCTP2 and 19 noncarrying volunteers. Six families including one with 10 members showed the S80fs-CP correlation. Functional magnetic resonance imaging found association of impaired recognition of individual faces by MCTP2 mutant CPs with reduced repetition suppression to repeated facial identities in the right fusiform face area. Our results have revealed genetic predisposition of MCTP2 mutations in CP, 76 years after the initial report of prosopagnosia and 47 years after the report of the first CP. This is the first time a gene required for a higher form of visual social cognition was found in humans.
Collapse
Affiliation(s)
- Yun Sun
- Chinese Institutes for Medical Research, Capital Medical University, Beijing 100069, China
- Chinese Institute for Brain Research, Peking-Tsinghua Center for Life Sciences, PKU-IDG/McGovern Institute for Brain Research, School of Life Sciences, Peking University, Beijing 100871, China
- Institute of Molecular Physiology, Shenzhen Bay Laboratory, Shenzhen 518107, China
| | - Weiwei Men
- Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Beijing Key Lab for Medical Physics and Engineering, Institute of Heavy Ion Physics, School of Physics, Peking University, Beijing 100871, China
| | - Ingo Kennerknecht
- Institute of Human Genetics, Westfälische Wilhelms-Universität, Münster 48149, Germany
| | - Wan Fang
- Chinese Institute for Brain Research, Peking-Tsinghua Center for Life Sciences, PKU-IDG/McGovern Institute for Brain Research, School of Life Sciences, Peking University, Beijing 100871, China
| | - Hou-Feng Zheng
- School of Life Sciences, Westlake University, Hangzhou, Zhejiang 310024, China
| | - Wenxia Zhang
- Chinese Institutes for Medical Research, Capital Medical University, Beijing 100069, China
- Chinese Institute for Brain Research, Peking-Tsinghua Center for Life Sciences, PKU-IDG/McGovern Institute for Brain Research, School of Life Sciences, Peking University, Beijing 100871, China
| | - Yi Rao
- Chinese Institutes for Medical Research, Capital Medical University, Beijing 100069, China
- Chinese Institute for Brain Research, Peking-Tsinghua Center for Life Sciences, PKU-IDG/McGovern Institute for Brain Research, School of Life Sciences, Peking University, Beijing 100871, China
- Institute of Molecular Physiology, Shenzhen Bay Laboratory, Shenzhen 518107, China
| |
Collapse
|
3
|
Karlsson EM, Carey DP. Hemispheric asymmetry of hand and tool perception in left- and right-handers with known language dominance. Neuropsychologia 2024; 196:108837. [PMID: 38428518 DOI: 10.1016/j.neuropsychologia.2024.108837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 12/20/2023] [Accepted: 02/21/2024] [Indexed: 03/03/2024]
Abstract
Regions in the brain that are selective for images of hands and tools have been suggested to be lateralised to the left hemisphere of right-handed individuals. In left-handers, many functions related to tool use or tool pantomime may also depend more on the left hemisphere. This result seems surprising, given that the dominant hand of these individuals is controlled by the right hemisphere. One explanation is that the left hemisphere is dominant for speech and language in the majority of left-handers, suggesting a supraordinate control system for complex motor sequencing that is required for skilled tool use, as well as for speech. In the present study, we examine if this left-hemispheric specialisation extends to perception of hands and tools in left- and right-handed individuals. We, crucially, also include a group of left-handers with right-hemispheric language dominance to examine their asymmetry biases. The results suggest that tools lateralise to the left hemisphere in most right-handed individuals with left-hemispheric language dominance. Tools also lateralise to the language dominant hemisphere in right-hemispheric language dominant left-handers, but the result for left-hemispheric language dominant left-handers are more varied, and no clear bias towards one hemisphere is found. Hands did not show a group-level asymmetry pattern in any of the groups. These results suggest a more complex picture regarding hemispheric overlap of hand and tool representations, and that visual appearance of tools may be driven in part by both language dominance and the hemisphere which controls the motor-dominant hand.
Collapse
Affiliation(s)
- Emma M Karlsson
- Institute of Cognitive Neuroscience, School of Psychology and Sport Science, Bangor University, Bangor, UK; Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium.
| | - David P Carey
- Institute of Cognitive Neuroscience, School of Psychology and Sport Science, Bangor University, Bangor, UK
| |
Collapse
|
4
|
Zafirova Y, Bognár A, Vogels R. Configuration-sensitive face-body interactions in primate visual cortex. Prog Neurobiol 2024; 232:102545. [PMID: 38042248 PMCID: PMC10788614 DOI: 10.1016/j.pneurobio.2023.102545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 09/28/2023] [Accepted: 11/20/2023] [Indexed: 12/04/2023]
Abstract
Traditionally, the neural processing of faces and bodies is studied separately, although they are encountered together, as parts of an agent. Despite its social importance, it is poorly understood how faces and bodies interact, particularly at the single-neuron level. Here, we examined the interaction between faces and bodies in the macaque inferior temporal (IT) cortex, targeting an fMRI-defined patch. We recorded responses of neurons to monkey images in which the face was in its natural location (natural face-body configuration), or in which the face was mislocated with respect to the upper body (unnatural face-body configuration). On average, the neurons did not respond stronger to the natural face-body configurations compared to the summed responses to their faces and bodies, presented in isolation. However, the neurons responded stronger to the natural compared to the unnatural face-body configurations. This configuration effect was present for face- and monkey-centered images, did not depend on local feature differences between configurations, and was present when the face was replaced by a small object. The face-body interaction rules differed between natural and unnatural configurations. In sum, we show for the first time that single IT neurons process faces and bodies in a configuration-specific manner, preferring natural face-body configurations.
Collapse
Affiliation(s)
- Yordanka Zafirova
- Laboratorium voor Neuro, en Psychofysiologie, Department of Neurosciences, KU Leuven, Belgium; Leuven Brain Institute, KU Leuven, Belgium
| | - Anna Bognár
- Laboratorium voor Neuro, en Psychofysiologie, Department of Neurosciences, KU Leuven, Belgium; Leuven Brain Institute, KU Leuven, Belgium
| | - Rufin Vogels
- Laboratorium voor Neuro, en Psychofysiologie, Department of Neurosciences, KU Leuven, Belgium; Leuven Brain Institute, KU Leuven, Belgium.
| |
Collapse
|
5
|
Demirlek C, Karakılıç M, Sarıkaya E, Bayrakçı A, Verim B, Gülyüksel F, Yalınçetin B, Oral E, Gelal F, Zorlu N, Bora E. Neural correlates of mental state decoding and mental state reasoning in schizophrenia. Psychiatry Res Neuroimaging 2023; 336:111744. [PMID: 37979348 DOI: 10.1016/j.pscychresns.2023.111744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 10/17/2023] [Accepted: 10/24/2023] [Indexed: 11/20/2023]
Abstract
Theory of mind skills are disrupted in schizophrenia. However, various theory of mind tasks measure different neurocognitive domains. This multimodal neuroimaging study aimed to investigate the neuroanatomical correlates of mental state decoding and reasoning components of theory of mind in schizophrenia and healthy controls (HCs) using T1-weighted and diffusion-weighted (DTI) magnetic resonance imaging (MRI). Sixty-two patients with schizophrenia and 34 HCs were included. The Reading the Mind in the Eyes (RMET) and Hinting tests were used to evaluate mental state decoding and reasoning, respectively. Correlations between social cognition and cortical parameters (thickness, volume, surface area), or DTI scalars (fractional anisotropy, axial diffusivity, radial diffusivity) were cluster-based corrected for multiple comparisons. In schizophrenia, RMET scores showed positive correlations in 3 clusters, including left insula thickness, right superior-temporal thickness, left superior-temporal-sulcus volume, and DTI analysis revealed that fractional anisotropy showed positive correlations in 3 clusters, including right inferior-fronto-occipital fasciculus, left forceps-major, left inferior-fronto-occipital fasciculus. In schizophrenia, Hinting test scores showed positive correlations in 3 clusters in T1-weighted MRI, including left superior-temporal-sulcus volume, left superior-temporal-sulcus surface area, left pars-orbitalis volume. In conclusion, this study provided evidence for the involvement of particular cortical regions and white matter tracts in mental state decoding and reasoning.
Collapse
Affiliation(s)
- Cemal Demirlek
- Department of Neurosciences, Institute of Health Sciences, Dokuz Eylul University, Izmir, Turkey; Department of Psychiatry, McLean Hospital, Harvard Medical School, Belmont, MA, USA.
| | - Merve Karakılıç
- Department of Psychiatry, Katip Celebi University, Ataturk Education and Research Hospital, Izmir, Turkey
| | - Ecenaz Sarıkaya
- Department of Psychiatry, Katip Celebi University, Ataturk Education and Research Hospital, Izmir, Turkey
| | - Adem Bayrakçı
- Department of Psychiatry, Katip Celebi University, Ataturk Education and Research Hospital, Izmir, Turkey
| | - Burcu Verim
- Department of Neurosciences, Institute of Health Sciences, Dokuz Eylul University, Izmir, Turkey
| | - Funda Gülyüksel
- Department of Psychiatry, Katip Celebi University, Ataturk Education and Research Hospital, Izmir, Turkey
| | - Berna Yalınçetin
- Department of Neurosciences, Institute of Health Sciences, Dokuz Eylul University, Izmir, Turkey
| | - Elif Oral
- Department of Psychiatry, Katip Celebi University, Ataturk Education and Research Hospital, Izmir, Turkey
| | - Fazıl Gelal
- Department of Radiodiagnostics, Katip Celebi University, Ataturk Education and Research Hospital, Izmir, Turkey
| | - Nabi Zorlu
- Department of Psychiatry, Katip Celebi University, Ataturk Education and Research Hospital, Izmir, Turkey
| | - Emre Bora
- Department of Neurosciences, Institute of Health Sciences, Dokuz Eylul University, Izmir, Turkey; Department of Psychiatry, Dokuz Eylul University Medical School, Izmir, Turkey; Department of Psychiatry, Melbourne Neuropsychiatry Centre, University of Melbourne and, Melbourne Health, Carlton South, Victoria 3053, Australia
| |
Collapse
|
6
|
Plaza PL, Renier L, Rosemann S, De Volder AG, Rauschecker JP. Sound-encoded faces activate the left fusiform face area in the early blind. PLoS One 2023; 18:e0286512. [PMID: 37992062 PMCID: PMC10664868 DOI: 10.1371/journal.pone.0286512] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Accepted: 05/17/2023] [Indexed: 11/24/2023] Open
Abstract
Face perception in humans and nonhuman primates is accomplished by a patchwork of specialized cortical regions. How these regions develop has remained controversial. In sighted individuals, facial information is primarily conveyed via the visual modality. Early blind individuals, on the other hand, can recognize shapes using auditory and tactile cues. Here we demonstrate that such individuals can learn to distinguish faces from houses and other shapes by using a sensory substitution device (SSD) presenting schematic faces as sound-encoded stimuli in the auditory modality. Using functional MRI, we then asked whether a face-selective brain region like the fusiform face area (FFA) shows selectivity for faces in the same subjects, and indeed, we found evidence for preferential activation of the left FFA by sound-encoded faces. These results imply that FFA development does not depend on experience with visual faces per se but may instead depend on exposure to the geometry of facial configurations.
Collapse
Affiliation(s)
- Paula L. Plaza
- Laboratory of Integrative Neuroscience and Cognition, Department of Neuroscience, Georgetown University Medical Center, Washington, DC, United States of America
| | - Laurent Renier
- Laboratory of Integrative Neuroscience and Cognition, Department of Neuroscience, Georgetown University Medical Center, Washington, DC, United States of America
- Neural Rehabilitation Laboratory, Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium
| | - Stephanie Rosemann
- Laboratory of Integrative Neuroscience and Cognition, Department of Neuroscience, Georgetown University Medical Center, Washington, DC, United States of America
| | - Anne G. De Volder
- Neural Rehabilitation Laboratory, Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium
| | - Josef P. Rauschecker
- Laboratory of Integrative Neuroscience and Cognition, Department of Neuroscience, Georgetown University Medical Center, Washington, DC, United States of America
| |
Collapse
|
7
|
Narganes-Pineda C, Paz-Alonso PM, Marotta A, Lupiáñez J, Chica AB. Neural basis of social attention: common and distinct mechanisms for social and nonsocial orienting stimuli. Cereb Cortex 2023; 33:11010-11024. [PMID: 37782936 DOI: 10.1093/cercor/bhad339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 08/22/2023] [Accepted: 08/24/2023] [Indexed: 10/04/2023] Open
Abstract
Social and nonsocial directional stimuli (such as gaze and arrows, respectively) share their ability to trigger attentional processes, although the issue of whether social stimuli generate other additional (and unique) attentional effects is still under debate. In this study, we used the spatial interference paradigm to explore, using functional magnetic resonance imaging, shared and dissociable brain activations produced by gaze and arrows. Results showed a common set of regions (right parieto-temporo-occipital) similarly involved in conflict resolution for gaze and arrows stimuli, which showed stronger co-activation for incongruent than congruent trials. The frontal eye field showed stronger functional connectivity with occipital regions for congruent as compared with incongruent trials, and this effect was enhanced for gaze as compared with arrow stimuli in the right hemisphere. Moreover, spatial interference produced by incongruent (as compared with congruent) arrows was associated with increased functional coupling between the right frontal eye field and a set of regions in the left hemisphere. This result was not observed for incongruent (as compared with congruent) gaze stimuli. The right frontal eye field also showed greater coupling with left temporo-occipital regions for those conditions in which larger conflict was observed (arrow incongruent vs. gaze incongruent trials, and gaze congruent vs. arrow congruent trials). These findings support the view that social and nonsocial stimuli share some attentional mechanisms, while at the same time highlighting other differential effects. Highlights Attentional orienting triggered by social (gaze) and nonsocial (arrow) cues is comparable. When social and nonsocial stimuli are used as targets, qualitatively different behavioral effects are observed. This study explores the neural bases of shared and dissociable neural mechanisms for social and nonsocial stimuli. Shared mechanisms were found in the functional coupling between right parieto-temporo-occipital regions. Dissociable mechanisms were found in the functional coupling between right frontal eye field and ipsilateral and contralateral occipito-temporal regions.
Collapse
Affiliation(s)
- Cristina Narganes-Pineda
- Department of Experimental Psychology and Mind, Brain, and Behavior Research Center (CIMCYC), University of Granada, Campus de Cartuja S/N, 18071, Granada, Spain
| | - Pedro M Paz-Alonso
- BCBL, Basque Center on Cognition, Brain, and Language, Mikeletegi Pasealekua 69, 20009 Donostia, Gipuzkoa, Spain
- IKERBASQUE, Basque Foundation for Science, 48013 Bilbo, Bizkaia, Spain
| | - Andrea Marotta
- Department of Experimental Psychology and Mind, Brain, and Behavior Research Center (CIMCYC), University of Granada, Campus de Cartuja S/N, 18071, Granada, Spain
| | - Juan Lupiáñez
- Department of Experimental Psychology and Mind, Brain, and Behavior Research Center (CIMCYC), University of Granada, Campus de Cartuja S/N, 18071, Granada, Spain
| | - Ana B Chica
- Department of Experimental Psychology and Mind, Brain, and Behavior Research Center (CIMCYC), University of Granada, Campus de Cartuja S/N, 18071, Granada, Spain
| |
Collapse
|
8
|
van Dyck LE, Gruber WR. Modeling Biological Face Recognition with Deep Convolutional Neural Networks. J Cogn Neurosci 2023; 35:1521-1537. [PMID: 37584587 DOI: 10.1162/jocn_a_02040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/17/2023]
Abstract
Deep convolutional neural networks (DCNNs) have become the state-of-the-art computational models of biological object recognition. Their remarkable success has helped vision science break new ground, and recent efforts have started to transfer this achievement to research on biological face recognition. In this regard, face detection can be investigated by comparing face-selective biological neurons and brain areas to artificial neurons and model layers. Similarly, face identification can be examined by comparing in vivo and in silico multidimensional "face spaces." In this review, we summarize the first studies that use DCNNs to model biological face recognition. On the basis of a broad spectrum of behavioral and computational evidence, we conclude that DCNNs are useful models that closely resemble the general hierarchical organization of face recognition in the ventral visual pathway and the core face network. In two exemplary spotlights, we emphasize the unique scientific contributions of these models. First, studies on face detection in DCNNs indicate that elementary face selectivity emerges automatically through feedforward processing even in the absence of visual experience. Second, studies on face identification in DCNNs suggest that identity-specific experience and generative mechanisms facilitate this particular challenge. Taken together, as this novel modeling approach enables close control of predisposition (i.e., architecture) and experience (i.e., training data), it may be suited to inform long-standing debates on the substrates of biological face recognition.
Collapse
|
9
|
Almeida J, Fracasso A, Kristensen S, Valério D, Bergström F, Chakravarthi R, Tal Z, Walbrin J. Neural and behavioral signatures of the multidimensionality of manipulable object processing. Commun Biol 2023; 6:940. [PMID: 37709924 PMCID: PMC10502059 DOI: 10.1038/s42003-023-05323-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 09/04/2023] [Indexed: 09/16/2023] Open
Abstract
Understanding how we recognize objects requires unravelling the variables that govern the way we think about objects and the neural organization of object representations. A tenable hypothesis is that the organization of object knowledge follows key object-related dimensions. Here, we explored, behaviorally and neurally, the multidimensionality of object processing. We focused on within-domain object information as a proxy for the decisions we typically engage in our daily lives - e.g., identifying a hammer in the context of other tools. We extracted object-related dimensions from subjective human judgments on a set of manipulable objects. We show that the extracted dimensions are cognitively interpretable and relevant - i.e., participants are able to consistently label them, and these dimensions can guide object categorization; and are important for the neural organization of knowledge - i.e., they predict neural signals elicited by manipulable objects. This shows that multidimensionality is a hallmark of the organization of manipulable object knowledge.
Collapse
Affiliation(s)
- Jorge Almeida
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.
| | - Alessio Fracasso
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | - Stephanie Kristensen
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Daniela Valério
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Fredrik Bergström
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- Department of Psychology, University of Gothenburg, Gothenburg, Sweden
| | | | - Zohar Tal
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Jonathan Walbrin
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
10
|
Lim C, Inagaki M, Shinozaki T, Fujita I. Analysis of convolutional neural networks reveals the computational properties essential for subcortical processing of facial expression. Sci Rep 2023; 13:10908. [PMID: 37407668 DOI: 10.1038/s41598-023-37995-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Accepted: 06/30/2023] [Indexed: 07/07/2023] Open
Abstract
Perception of facial expression is crucial for primate social interactions. This visual information is processed through the ventral cortical pathway and the subcortical pathway. However, the subcortical pathway exhibits inaccurate processing, and the responsible architectural and physiological properties remain unclear. To investigate this, we constructed and examined convolutional neural networks with three key properties of the subcortical pathway: a shallow layer architecture, concentric receptive fields at the initial processing stage, and a greater degree of spatial pooling. These neural networks achieved modest accuracy in classifying facial expressions. By replacing these properties, individually or in combination, with corresponding cortical features, performance gradually improved. Similar to amygdala neurons, some units in the final processing layer exhibited sensitivity to retina-based spatial frequencies (SFs), while others were sensitive to object-based SFs. Replacement of any of these properties affected the coordinates of the SF encoding. Therefore, all three properties limit the accuracy of facial expression information and are essential for determining the SF representation coordinate. These findings characterize the role of the subcortical computational processes in facial expression recognition.
Collapse
Affiliation(s)
- Chanseok Lim
- Laboratory for Cognitive Neuroscience, Graduate School of Frontier Biosciences, Osaka University, 1-4 Yamadaoka, Suita, Osaka, 565-0871, Japan
- Perceptual and Cognitive Neuroscience Laboratory, Graduate School of Frontier Biosciences, Osaka University, 1-4 Yamadaoka, Suita, Osaka, 565-0871, Japan
| | - Mikio Inagaki
- Laboratory for Cognitive Neuroscience, Graduate School of Frontier Biosciences, Osaka University, 1-4 Yamadaoka, Suita, Osaka, 565-0871, Japan
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, 1-4 Yamadaoka, Suita, Osaka, 565-0871, Japan
| | - Takashi Shinozaki
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, 1-4 Yamadaoka, Suita, Osaka, 565-0871, Japan
- Computational Neuroscience Laboratory, Faculty of Informatics, Kindai University, 3-4-1 Kowakae, Higashiosaka, Osaka, 577-8502, Japan
| | - Ichiro Fujita
- Laboratory for Cognitive Neuroscience, Graduate School of Frontier Biosciences, Osaka University, 1-4 Yamadaoka, Suita, Osaka, 565-0871, Japan.
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, 1-4 Yamadaoka, Suita, Osaka, 565-0871, Japan.
- Research Organization of Science and Technology, Ritsumeikan University, 1-1-1 Noji-Higashi, Kusatsu, Shiga, 525-8577, Japan.
| |
Collapse
|
11
|
Li D, Chang L. Representational geometry of incomplete faces in macaque face patches. Cell Rep 2023; 42:112673. [PMID: 37342911 DOI: 10.1016/j.celrep.2023.112673] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 04/23/2023] [Accepted: 06/06/2023] [Indexed: 06/23/2023] Open
Abstract
The neural code of faces has been intensively studied in the macaque face patch system. Although the majority of previous studies used complete faces as stimuli, faces are often seen partially in daily life. Here, we investigated how face-selective cells represent two types of incomplete faces: face fragments and occluded faces, with the location of the fragment/occluder and the facial features systematically varied. Contrary to popular belief, we found that the preferred face regions identified with two stimulus types are dissociated in many face cells. This dissociation can be explained by the nonlinear integration of information from different face parts and is closely related to a curved representation of face completeness in the state space, which allows a clear discrimination between different stimulus types. Furthermore, identity-related facial features are represented in a subspace orthogonal to the nonlinear dimension of face completeness, supporting a condition-general code of facial identity.
Collapse
Affiliation(s)
- Dongyuan Li
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai 200031, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Le Chang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai 200031, China; University of Chinese Academy of Sciences, Beijing 100049, China.
| |
Collapse
|
12
|
Monteiro LCP, Ripardo RC, Torro-Alves N, Souza GS. Facial morphometric differences across face databases: influence of ethnicities and sex. Front Neurosci 2023; 17:1130867. [PMID: 37404467 PMCID: PMC10315459 DOI: 10.3389/fnins.2023.1130867] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 06/05/2023] [Indexed: 07/06/2023] Open
Abstract
The scientific need for standardized, high-quality facial stimuli has driven the creation of several face image databases in recent years. These stimuli are particularly important in facial asymmetry research. However, previous studies have reported facial anthropometric differences across a variety of ethnicities. This highlights the need to investigate whether these differences can also impact the use of face image databases, particularly in facial asymmetry research. In this study, we investigated facial asymmetry-based morphometric differences between the multi-ethnic Chicago Face Database (CFD) and the LACOP Face Database, which is composed of Brazilian subjects. We found reliable differences in facial asymmetry between the two databases, which were related to ethnic groups. Specifically, differences in eye and mouth asymmetry seem to drive these differences. The asymmetry-based morphometric differences among databases and ethnicities found in this study reinforce the necessity of creating multi-ethnic face databases.
Collapse
Affiliation(s)
- Luis Carlos Pereira Monteiro
- Neuroscience and Cell Biology Graduate Program, Institute of Biological Sciences, Federal University of Pará, Belém, Brazil
- Center for Tropical Medicine, Federal University of Pará, Belém, Brazil
| | - Rachel Coelho Ripardo
- Neuroscience and Behavior Graduate Program, Center for Behavioral Theory and Research, Federal University of Pará, Belém, Brazil
| | - Nelson Torro-Alves
- Department of Psychology, Federal University of Paraíba, João Pessoa, Brazil
| | - Givago Silva Souza
- Neuroscience and Cell Biology Graduate Program, Institute of Biological Sciences, Federal University of Pará, Belém, Brazil
- Center for Tropical Medicine, Federal University of Pará, Belém, Brazil
| |
Collapse
|
13
|
Kunimatsu J, Amita H, Hikosaka O. Neuronal mechanism of the encoding of socially familiar faces in the striatum tail. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.10.540108. [PMID: 37425892 PMCID: PMC10327190 DOI: 10.1101/2023.05.10.540108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/11/2023]
Abstract
Although we can quickly locate a familiar person even in a crowd, the underlying neuronal mechanism remains unclear. Recently, we found that the striatum tail (STRt), which is part of the basal ganglia, is sensitive to long-term reward history. Here, we show that long-term value-coding neurons are involved in the detection of socially familiar faces. Many STRt neurons respond to facial images, especially to those of socially familiar persons. Additionally, we found that these face-responsive neurons also encode the stable values of many objects based on long-term reward experiences. Interestingly, the strength of neuronal modulation of social familiarity bias (familiar or unfamiliar) and object value bias (high-valued or low-valued) were positively correlated. These results suggest that both social familiarity and stable object-value information are mediated by a common neuronal mechanism. This mechanism may contribute to the rapid detection of familiar faces in real-world contexts.
Collapse
Affiliation(s)
- Jun Kunimatsu
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| | - Hidetoshi Amita
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
- Systems Neuroscience Section, Center for the Evolutionary Origins of Human Behavior, Kyoto University, Inuyama, Aichi 484-8506, Japan
| | - Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
| |
Collapse
|
14
|
Pang W, Zhou W, Ruan Y, Zhang L, Shu H, Zhang Y, Zhang Y. Visual Deprivation Alters Functional Connectivity of Neural Networks for Voice Recognition: A Resting-State fMRI Study. Brain Sci 2023; 13:brainsci13040636. [PMID: 37190601 DOI: 10.3390/brainsci13040636] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2023] [Revised: 03/29/2023] [Accepted: 04/04/2023] [Indexed: 05/17/2023] Open
Abstract
Humans recognize one another by identifying their voices and faces. For sighted people, the integration of voice and face signals in corresponding brain networks plays an important role in facilitating the process. However, individuals with vision loss primarily resort to voice cues to recognize a person's identity. It remains unclear how the neural systems for voice recognition reorganize in the blind. In the present study, we collected behavioral and resting-state fMRI data from 20 early blind (5 females; mean age = 22.6 years) and 22 sighted control (7 females; mean age = 23.7 years) individuals. We aimed to investigate the alterations in the resting-state functional connectivity (FC) among the voice- and face-sensitive areas in blind subjects in comparison with controls. We found that the intranetwork connections among voice-sensitive areas, including amygdala-posterior "temporal voice areas" (TVAp), amygdala-anterior "temporal voice areas" (TVAa), and amygdala-inferior frontal gyrus (IFG) were enhanced in the early blind. The blind group also showed increased FCs of "fusiform face area" (FFA)-IFG and "occipital face area" (OFA)-IFG but decreased FCs between the face-sensitive areas (i.e., FFA and OFA) and TVAa. Moreover, the voice-recognition accuracy was positively related to the strength of TVAp-FFA in the sighted, and the strength of amygdala-FFA in the blind. These findings indicate that visual deprivation shapes functional connectivity by increasing the intranetwork connections among voice-sensitive areas while decreasing the internetwork connections between the voice- and face-sensitive areas. Moreover, the face-sensitive areas are still involved in the voice-recognition process in blind individuals through pathways such as the subcortical-occipital or occipitofrontal connections, which may benefit the visually impaired greatly during voice processing.
Collapse
Affiliation(s)
- Wenbin Pang
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
- China National Clinical Research Center for Neurological Diseases, Beijing 100070, China
| | - Wei Zhou
- Beijing Key Lab of Learning and Cognition, School of Psychology, Capital Normal University, Beijing 100048, China
| | - Yufang Ruan
- School of Communication Sciences and Disorders, Faculty of Medicine and Health Sciences, McGill University, Montréal, QC H3A 1G1, Canada
- Centre for Research on Brain, Language and Music, Montréal, QC H3A 1G1, Canada
| | - Linjun Zhang
- School of Chinese as a Second Language, Peking University, Beijing 100871, China
| | - Hua Shu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China
| | - Yang Zhang
- Department of Speech-Language-Hearing Sciences and Center for Neurobehavioral Development, The University of Minnesota, Minneapolis, MN 55455, USA
| | - Yumei Zhang
- China National Clinical Research Center for Neurological Diseases, Beijing 100070, China
- Department of Rehabilitation, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| |
Collapse
|
15
|
Koyano KW, Esch EM, Hong JJ, Waidmann EN, Wu H, Leopold DA. Progressive neuronal plasticity in primate visual cortex during stimulus familiarization. SCIENCE ADVANCES 2023; 9:eade4648. [PMID: 36961903 PMCID: PMC10038346 DOI: 10.1126/sciadv.ade4648] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Accepted: 02/21/2023] [Indexed: 06/18/2023]
Abstract
The primate brain is equipped to learn and remember newly encountered visual stimuli such as faces and objects. In the macaque inferior temporal (IT) cortex, neurons mark the familiarity of a visual stimulus through response modification, often involving a decrease in spiking rate. Here, we investigate the emergence of this neural plasticity by longitudinally tracking IT neurons during several weeks of familiarization with face images. We found that most neurons in the anterior medial (AM) face patch exhibited a gradual decline in their late-phase visual responses to multiple stimuli. Individual neurons varied from days to weeks in their rates of plasticity, with time constants determined by the number of days of exposure rather than the cumulative number of presentations. We postulate that the sequential recruitment of neurons with experience-modified responses may provide an internal and graded measure of familiarity strength, which is a key mnemonic component of visual recognition.
Collapse
Affiliation(s)
- Kenji W. Koyano
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - Elena M. Esch
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - Julie J. Hong
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - Elena N. Waidmann
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - Haitao Wu
- Chemistry and Synthesis Center, National Heart, Lung and Blood Institute, National Institutes of Health, Bethesda, MD 20892, USA
| | - David A. Leopold
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20892, USA
- Neurophysiology Imaging Facility, National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, National Eye Institute, Bethesda MD 20892, USA
| |
Collapse
|
16
|
Yang Z, Freiwald WA. Encoding of dynamic facial information in the middle dorsal face area. Proc Natl Acad Sci U S A 2023; 120:e2212735120. [PMID: 36787369 PMCID: PMC9974491 DOI: 10.1073/pnas.2212735120] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Accepted: 01/04/2023] [Indexed: 02/15/2023] Open
Abstract
Faces in motion reveal a plethora of information through visual dynamics. Faces can move in complex patterns while transforming facial shape, e.g., during the generation of different emotional expressions. While motion and shape processing have been studied extensively in separate research enterprises, much less is known about their conjunction during biological motion. Here, we took advantage of the discovery in brain-imaging studies of an area in the dorsal portion of the macaque monkey superior temporal sulcus (STS), the middle dorsal face area (MD), with selectivity for naturalistic face motion. To gain mechanistic insights into the coding of facial motion, we recorded single-unit activity from MD, testing whether and how MD cells encode face motion. The MD population was highly sensitive to naturalistic facial motion and facial shape. Some MD cells responded only to the conjunction of facial shape and motion, others were selective for facial shape even without movement, and yet others were suppressed by facial motion. We found that this heterogeneous MD population transforms face motion into a higher dimensional activity space, a representation that would allow for high sensitivity to relevant small-scale movements. Indeed, we show that many MD cells carry such sensitivity for eye movements. We further found that MD cells encode motion of head, mouth, and eyes in a separable manner, requiring the use of multiple reference frames. Thus, MD is a bona fide face-motion area that uses highly heterogeneous cell populations to create codes capturing even complex facial motion trajectories.
Collapse
Affiliation(s)
- Zetian Yang
- Laboratory of Neural Systems, The Rockefeller University, New York, NY10065
| | - Winrich A. Freiwald
- Laboratory of Neural Systems, The Rockefeller University, New York, NY10065
- The Center for Brains, Minds and Machines, Cambridge, MA02139
| |
Collapse
|
17
|
Zhou Q, Du J, Gao R, Hu S, Yu T, Wang Y, Pan NC. Discriminative neural pathways for perception-cognition activity of color and face in the human brain. Cereb Cortex 2023; 33:1972-1984. [PMID: 35580851 DOI: 10.1093/cercor/bhac186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 04/22/2022] [Accepted: 04/24/2022] [Indexed: 11/13/2022] Open
Abstract
Human performance can be examined using a visual lens. The identification of psychophysical colors and emotional faces with perceptual visual pathways may remain invalid for simple detection tasks. In particular, how the visual dorsal and ventral processing streams handle discriminative visual perceptions and subsequent cognition activities are obscure. We explored these issues using stereoelectroencephalography recordings, which were obtained from patients with pharmacologically resistant epilepsy. Delayed match-to-sample paradigms were used for analyzing the processing of simple colors and complex emotional faces in the human brain. We showed that the angular-cuneus gyrus acts as a pioneer in discriminating the 2 features, and dorsal regions, including the middle frontal gyrus (MFG) and postcentral gyrus, as well as ventral regions, such as the middle temporal gyrus (MTG) and posterior superior temporal sulcus (pSTS), were involved in processing incongruent colors and faces. Critically, the beta and gamma band activities between the cuneus and MTG and between the cuneus and pSTS would tune a separate pathway of incongruency processing. In addition, posterior insular gyrus, fusiform, and MFG were found for attentional modulation of the 2 features via alpha band activities. These findings suggest the neural basis of the discriminative pathways of perception-cognition activities in the human brain.
Collapse
Affiliation(s)
- Qilin Zhou
- Department of Neurology, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China.,Beijing Key Laboratory of Neuromodulation, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China
| | - Jialin Du
- Department of Pharmacy Phase I Clinical Trial Center, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China
| | - Runshi Gao
- Beijing Institute of Functional Neurosurgery, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China
| | - Shimin Hu
- Department of Neurology, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China.,Beijing Key Laboratory of Neuromodulation, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China
| | - Tao Yu
- Beijing Institute of Functional Neurosurgery, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China
| | - Yuping Wang
- Department of Neurology, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China.,Beijing Key Laboratory of Neuromodulation, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China.,Institute of sleep and consciousness disorders, Center of Epilepsy, Beijing Institute for Brain Disorders, Capital Medical University, No. 10, Xi Tou Tiao, Youanmen wai, Fengtai District, Beijing, 100069, China
| | - Na Clara Pan
- Department of Neurology, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China.,Beijing Key Laboratory of Neuromodulation, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China
| |
Collapse
|
18
|
Zafirova Y, Cui D, Raman R, Vogels R. Keep the head in the right place: Face-body interactions in inferior temporal cortex. Neuroimage 2022; 264:119676. [PMID: 36216293 DOI: 10.1016/j.neuroimage.2022.119676] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Revised: 09/23/2022] [Accepted: 10/06/2022] [Indexed: 11/05/2022] Open
Abstract
In primates, faces and bodies activate distinct regions in the inferior temporal (IT) cortex and are typically studied separately. Yet, primates interact with whole agents and not with random concatenations of faces and bodies. Despite its social importance, it is still poorly understood how faces and bodies interact in IT. Here, we addressed this gap by measuring fMRI activations to whole agents and to unnatural face-body configurations in which the head was mislocated with respect to the body, and examined how these relate to the sum of the activations to their corresponding faces and bodies. First, we mapped patches in the IT of awake macaques that were activated more by images of whole monkeys compared to objects and found that these mostly overlapped with body and face patches. In a second fMRI experiment, we obtained no evidence for superadditive responses in these "monkey patches", with the activation to the monkeys being less or equal to the summed face-body activations. However, monkey patches in the anterior IT were activated more by natural compared to unnatural configurations. The stronger activations to natural configurations could not be explained by the summed face-body activations. These univariate results were supported by regression analyses in which we modeled the activations to both configurations as a weighted linear combination of the activations to the faces and bodies, showing higher regression coefficients for the natural compared to the unnatural configurations. Deeper layers of trained convolutional neural networks also contained units that responded more to natural compared to unnatural monkey configurations. Unlike the monkey fMRI patches, these units showed substantial superadditive responses to the natural configurations. Our monkey fMRI data suggest configuration-sensitive face-body interactions in anterior IT, adding to the evidence for an integrated face-body processing in the primate ventral visual stream, and open the way for mechanistic studies using single unit recordings in these patches.
Collapse
Affiliation(s)
- Yordanka Zafirova
- Laboratorium voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Belgium; Leuven Brain Institute, KU Leuven, Belgium
| | - Ding Cui
- Laboratorium voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Belgium; Leuven Brain Institute, KU Leuven, Belgium
| | - Rajani Raman
- Laboratorium voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Belgium; Leuven Brain Institute, KU Leuven, Belgium
| | - Rufin Vogels
- Laboratorium voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Belgium; Leuven Brain Institute, KU Leuven, Belgium.
| |
Collapse
|
19
|
Vila‐Vidal M, Khawaja M, Carreño M, Roldán P, Rumià J, Donaire A, Deco G, Tauste Campo A. Assessing the coupling between local neural activity and global connectivity fluctuations: Application to human intracranial electroencephalography during a cognitive task. Hum Brain Mapp 2022; 44:1173-1192. [PMID: 36437716 PMCID: PMC9875936 DOI: 10.1002/hbm.26150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Revised: 10/17/2022] [Accepted: 10/20/2022] [Indexed: 11/29/2022] Open
Abstract
Cognitive-relevant information is processed by different brain areas that cooperate to eventually produce a response. The relationship between local activity and global brain states during such processes, however, remains for the most part unexplored. To address this question, we designed a simple face-recognition task performed in patients with drug-resistant epilepsy and monitored with intracranial electroencephalography (EEG). Based on our observations, we developed a novel analytical framework (named "local-global" framework) to statistically correlate the brain activity in every recorded gray-matter region with the widespread connectivity fluctuations as proxy to identify concurrent local activations and global brain phenomena that may plausibly reflect a common functional network during cognition. The application of the local-global framework to the data from three subjects showed that similar connectivity fluctuations found across patients were mainly coupled to the local activity of brain areas involved in face information processing. In particular, our findings provide preliminary evidence that the reported global measures might be a novel signature of functional brain activity reorganization when a stimulus is processed in a task context regardless of the specific recorded areas.
Collapse
Affiliation(s)
- Manel Vila‐Vidal
- Center for Brain and Cognition, Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain,Computational Biology and Complex Systems Group, Department of PhysicsUniversitat Politècnica de CatalunyaBarcelonaSpain
| | | | - Mar Carreño
- Epilepsy ProgramHospital ClínicBarcelonaSpain,Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS)BarcelonaSpain
| | - Pedro Roldán
- Epilepsy Program, NeurosurgeryHospital ClínicBarcelonaSpain
| | - Jordi Rumià
- Epilepsy Program, NeurosurgeryHospital ClínicBarcelonaSpain
| | - Antonio Donaire
- Epilepsy ProgramHospital ClínicBarcelonaSpain,Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS)BarcelonaSpain,CIBERBBN, Networking Centre on Bioengineering, Biomaterials and NanomedicineBarcelonaSpain
| | - Gustavo Deco
- Computational Biology and Complex Systems Group, Department of PhysicsUniversitat Politècnica de CatalunyaBarcelonaSpain,Institució Catalana de Recerca i Estudis AvançatsBarcelonaSpain
| | - Adrià Tauste Campo
- Center for Brain and Cognition, Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain,Computational Biology and Complex Systems Group, Department of PhysicsUniversitat Politècnica de CatalunyaBarcelonaSpain
| |
Collapse
|
20
|
Taubert J, Japee S, Patterson A, Wild H, Goyal S, Yu D, Ungerleider LG. A broadly tuned network for affective body language in the macaque brain. SCIENCE ADVANCES 2022; 8:eadd6865. [PMID: 36427322 PMCID: PMC9699662 DOI: 10.1126/sciadv.add6865] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Accepted: 10/27/2022] [Indexed: 06/16/2023]
Abstract
Body language is a powerful tool that we use to communicate how we feel, but it is unclear whether other primates also communicate in this way. Here, we use functional magnetic resonance imaging to show that the body-selective patches in macaques are activated by affective body language. Unexpectedly, we found these regions to be tolerant of naturalistic variation in posture as well as species; the bodies of macaques, humans, and domestic cats all evoked a stronger response when they conveyed fear than when they conveyed no affect. Multivariate analyses confirmed that the neural representation of fear-related body expressions was species-invariant. Collectively, these findings demonstrate that, like humans, macaques have body-selective brain regions in the ventral visual pathway for processing affective body language. These data also indicate that representations of body stimuli in these regions are built on the basis of emergent properties, such as socio-affective meaning, and not just putative image properties.
Collapse
Affiliation(s)
- Jessica Taubert
- Section on Neurocircuitry, Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, MD 20892, USA
- School of Psychology, The University of Queensland, Brisbane, QLD 4072, Australia
| | - Shruti Japee
- Section on Neurocircuitry, Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - Amanda Patterson
- Section on Neurocircuitry, Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - Hannah Wild
- Section on Neurocircuitry, Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - Shivani Goyal
- Section on Neurocircuitry, Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - David Yu
- Section on Neurocircuitry, Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, MD 20892, USA
| | - Leslie G. Ungerleider
- Section on Neurocircuitry, Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, MD 20892, USA
| |
Collapse
|
21
|
COVID-19 masks increase the influence of face recognition algorithm decisions on human decisions in unfamiliar face matching. PLoS One 2022; 17:e0277625. [PMID: 36409731 PMCID: PMC9678274 DOI: 10.1371/journal.pone.0277625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Accepted: 10/31/2022] [Indexed: 11/22/2022] Open
Abstract
Face masks, recently adopted to reduce the spread of COVID-19, have had the unintended consequence of increasing the difficulty of face recognition. In security applications, face recognition algorithms are used to identify individuals and present results for human review. This combination of human and algorithm capabilities, known as human-algorithm teaming, is intended to improve total system performance. However, prior work has shown that human judgments of face pair similarity-confidence can be biased by an algorithm's decision even in the case of an error by that algorithm. This can reduce team effectiveness, particularly for difficult face pairs. We conducted two studies to examine whether face masks, now routinely present in security applications, impact the degree to which this cognitive bias is experienced by humans. We first compared the influence of algorithm's decisions on human similarity-confidence ratings in the presence and absence of face masks and found that face masks more than doubled the influence of algorithm decisions on human similarity-confidence ratings. We then investigated if this increase in cognitive bias was dependent on perceived algorithm accuracy by also presenting algorithm accuracy rates in the presence of face masks. We found that making humans aware of the potential for algorithm errors mitigated the increase in cognitive bias due to face masks. Our findings suggest that humans reviewing face recognition algorithm decisions should be made aware of the potential for algorithm errors to improve human-algorithm team performance.
Collapse
|
22
|
Liu N, Behrmann M, Turchi JN, Avidan G, Hadj-Bouziane F, Ungerleider LG. Bidirectional and parallel relationships in macaque face circuit revealed by fMRI and causal pharmacological inactivation. Nat Commun 2022; 13:6787. [PMID: 36351907 PMCID: PMC9646786 DOI: 10.1038/s41467-022-34451-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 10/25/2022] [Indexed: 11/11/2022] Open
Abstract
Although the presence of face patches in primate inferotemporal (IT) cortex is well established, the functional and causal relationships among these patches remain elusive. In two monkeys, muscimol was infused sequentially into each patch or pair of patches to assess their respective influence on the remaining IT face network and the amygdala, as determined using fMRI. The results revealed that anterior face patches required input from middle face patches for their responses to both faces and objects, while the face selectivity in middle face patches arose, in part, from top-down input from anterior face patches. Moreover, we uncovered a parallel fundal-lateral functional organization in the IT face network, supporting dual routes (dorsal-ventral) in face processing within IT cortex as well as between IT cortex and the amygdala. Our findings of the causal relationship among the face patches demonstrate that the IT face circuit is organized into multiple functional compartments.
Collapse
Affiliation(s)
- Ning Liu
- Section on Neurocircuitry, Laboratory of Brain and Cognition, NIMH, NIH, Bethesda, MD, 20892, USA.
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, 100101, Beijing, China.
| | - Marlene Behrmann
- Department of Ophthalmology, University of Pittsburgh, Pittsburgh, PA, 15213, USA
- Department of Psychology and Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| | - Janita N Turchi
- Laboratory of Neuropsychology, NIMH, NIH, Bethesda, MD, 20892, USA
| | - Galia Avidan
- Department of Psychology, Ben-Gurion University of the Negev, Beer-Sheva, 8410501, Israel
| | - Fadila Hadj-Bouziane
- Section on Neurocircuitry, Laboratory of Brain and Cognition, NIMH, NIH, Bethesda, MD, 20892, USA
- INSERM, U1028, CNRS UMR5292, Lyon Neuroscience Research Center, ImpAct Team, F-69000, Lyon, France
- University UCBL Lyon 1, F-69000, Lyon, France
| | - Leslie G Ungerleider
- Section on Neurocircuitry, Laboratory of Brain and Cognition, NIMH, NIH, Bethesda, MD, 20892, USA
| |
Collapse
|
23
|
Thome I, García Alanis JC, Volk J, Vogelbacher C, Steinsträter O, Jansen A. Let's face it: The lateralization of the face perception network as measured with fMRI is not clearly right dominant. Neuroimage 2022; 263:119587. [PMID: 36031183 DOI: 10.1016/j.neuroimage.2022.119587] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Revised: 08/17/2022] [Accepted: 08/22/2022] [Indexed: 11/18/2022] Open
Abstract
The neural face perception network is distributed across both hemispheres. However, the dominant role in humans is virtually unanimously attributed to the right hemisphere. Interestingly, there are, to our knowledge, no imaging studies that systematically describe the distribution of hemispheric lateralization in the core system of face perception across subjects in large cohorts so far. To address this, we determined the hemispheric lateralization of all core system regions (i.e., occipital face area (OFA), fusiform face area (FFA), posterior superior temporal sulcus (pSTS)) in 108 healthy subjects using functional magnetic resonance imaging (fMRI). We were particularly interested in the variability of hemispheric lateralization across subjects and explored how many subjects can be classified as right-dominant based on the fMRI activation pattern. We further assessed lateralization differences between different regions of the core system and analyzed the influence of handedness and sex on the lateralization with a generalized mixed effects regression model. As expected, brain activity was on average stronger in right-hemispheric brain regions than in their left-hemispheric homologues. This asymmetry was, however, only weakly pronounced in comparison to other lateralized brain functions (such as language and spatial attention) and strongly varied between individuals. Only half of the subjects in the present study could be classified as right-hemispheric dominant. Additionally, we did not detect significant lateralization differences between core system regions. Our data did also not support a general leftward shift of hemispheric lateralization in left-handers. Only the interaction of handedness and sex in the FFA revealed that specifically left-handed men were significantly more left-lateralized compared to right-handed males. In essence, our fMRI data did not support a clear right-hemispheric dominance of the face perception network. Our findings thus ultimately question the dogma that the face perception network - as measured with fMRI - can be characterized as "typically right lateralized".
Collapse
Affiliation(s)
- Ina Thome
- Laboratory for Multimodal Neuroimaging, Department of Psychiatry, University of Marburg, Marburg, Germany; Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Marburg, Germany.
| | - José C García Alanis
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Marburg, Germany; Clinical Child and Adolescent Psychology, Department of Psychology, University of Marburg, Marburg, Germany; Analysis and Modeling of Complex Data Lab, Institute of Psychology, University of Mainz, Mainz, Germany
| | - Jannika Volk
- Laboratory for Multimodal Neuroimaging, Department of Psychiatry, University of Marburg, Marburg, Germany; Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Marburg, Germany
| | - Christoph Vogelbacher
- Laboratory for Multimodal Neuroimaging, Department of Psychiatry, University of Marburg, Marburg, Germany; Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Marburg, Germany
| | - Olaf Steinsträter
- Core-Facility BrainImaging, Faculty of Medicine, University of Marburg, Marburg, Germany
| | - Andreas Jansen
- Laboratory for Multimodal Neuroimaging, Department of Psychiatry, University of Marburg, Marburg, Germany; Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Marburg, Germany; Core-Facility BrainImaging, Faculty of Medicine, University of Marburg, Marburg, Germany.
| |
Collapse
|
24
|
A neuronal social trait space for first impressions in the human amygdala and hippocampus. Mol Psychiatry 2022; 27:3501-3509. [PMID: 35672377 DOI: 10.1038/s41380-022-01583-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/28/2021] [Accepted: 04/12/2022] [Indexed: 11/09/2022]
Abstract
People instantaneously evaluate faces with significant agreement on evaluations of social traits. However, the neural basis for such rapid spontaneous face evaluation remains largely unknown. Here, we recorded from 490 neurons in the human amygdala and hippocampus and found that the neuronal activity was associated with the geometry of a social trait space. We further investigated the temporal evolution and modulation on the social trait representation, and we employed encoding and decoding models to reveal the critical social traits for the trait space. We also recorded from another 938 neurons and replicated our findings using different social traits. Together, our results suggest that there exists a neuronal population code for a comprehensive social trait space in the human amygdala and hippocampus that underlies spontaneous first impressions. Changes in such neuronal social trait space may have implications for the abnormal processing of social information observed in some neurological and psychiatric disorders.
Collapse
|
25
|
Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models. Proc Natl Acad Sci U S A 2022; 119:e2115047119. [PMID: 35767642 PMCID: PMC9271164 DOI: 10.1073/pnas.2115047119] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
Discerning the subtle differences between individuals’ faces is crucial for social functioning. It requires us not only to solve general challenges of object recognition (e.g., invariant recognition over changes in view or lighting) but also to be attuned to the specific ways in which face structure varies. Three-dimensional morphable models based on principal component analyses of real faces provide descriptions of statistical differences between faces, as well as tools to generate novel faces. We rendered large sets of realistic face pairs from such a model and collected similarity and same/different identity judgments. The statistical model predicted human perception as well as state-of-the-art image-computable neural networks. Results underscore the statistical tuning of face encoding. Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component–based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a “face space” based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.
Collapse
|
26
|
Rossion B. Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: Neural basis. Neuropsychologia 2022; 173:108279. [PMID: 35667496 DOI: 10.1016/j.neuropsychologia.2022.108279] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Revised: 04/30/2022] [Accepted: 05/25/2022] [Indexed: 10/18/2022]
Abstract
Patient PS sustained her dramatic brain injury in 1992, the same year as the first report of a neuroimaging study of human face recognition. The present paper complements the review on the functional nature of PS's prosopagnosia (part I), illustrating how her case study directly, i.e., through neuroimaging investigations of her brain structure and activity, but also indirectly, through neural studies performed on other clinical cases and neurotypical individuals, inspired and constrained neural models of human face recognition. In the dominant right hemisphere for face recognition in humans, PS's main lesion concerns (inputs to) the inferior occipital gyrus (IOG), in a region where face-selective activity is typically found in normal individuals ('Occipital Face Area', OFA). Her case study initially supported the criticality of this region for face identity recognition (FIR) and provided the impetus for transcranial magnetic stimulation (TMS), intracerebral electrical stimulation, and cortical surgery studies that have generally supported this view. Despite PS's right IOG lesion, typical face-selectivity is found anteriorly in the middle portion of the fusiform gyrus, a hominoid structure (termed the right 'Fusiform Face Area', FFA) that is widely considered to be the most important region for human face recognition. This finding led to the original proposal of direct anatomico-functional connections from early visual cortices to the FFA, bypassing the IOG/OFA (lulu), a hypothesis supported by further neuroimaging studies of PS, other neurological cases and neuro-typical individuals with original visual stimulation paradigms, data recordings and analyses. The proposal of a lack of sensitivity to face identity in PS's right FFA due to defective reentrant inputs from the IOG/FFA has also been supported by other cases, functional connectivity and cortical surgery studies. Overall, neural studies of, and based on, the case of prosopagnosia PS strongly question the hierarchical organization of the human neural face recognition system, supporting a more flexible and dynamic view of this key social brain function.
Collapse
Affiliation(s)
- Bruno Rossion
- Université de Lorraine, CNRS, CRAN, F-54000, Nancy, France; CHRU-Nancy, Service de Neurologie, F-5400, France; Psychological Sciences Research Institute, Institute of Neuroscience, University of Louvain, Belgium.
| |
Collapse
|
27
|
Frederick DA, Reynolds TA, Barrera CA, Murray SB. Demographic and sociocultural predictors of face image satisfaction: The U.S. Body Project I. Body Image 2022; 41:1-16. [PMID: 35228101 DOI: 10.1016/j.bodyim.2022.01.016] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/16/2020] [Revised: 12/22/2021] [Accepted: 01/26/2022] [Indexed: 11/28/2022]
Abstract
Despite substantial literature surrounding how people process and perceive faces, there is very little research investigating how people evaluate their own faces. We examined how gender, body mass, race, age, and sexual orientation were linked to people's satisfaction with the appearance of their eyes, nose, facial shape, and face overall among 11,620 adults recruited via Mechanical Turk. Most people mostly or definitely agreed they were happy with their facial appearance. There were notable racial differences, with Asian participants tending to report greater dissatisfaction. For example, only 66% of Asian women and 60% of Asian men mostly or definitely agreed that they were happy with the appearance of their eyes, which was lower than other racial groups. BMI and age were not strongly associated with face satisfaction. Sexual minority men were less satisfied than heterosexual men. About one in four gay and bisexual men, compared to only one in seven heterosexual men, reported dissatisfaction with their overall facial appearance. Men and women with poorer face image engaged in more appearance surveillance, more strongly internalized the thin-ideal, and perceived stronger sociocultural pressures from peers, parents, and media. The current study highlights important sociocultural and demographic factors tied to poorer face image.
Collapse
Affiliation(s)
- David A Frederick
- Crean College of Health and Behavioral Sciences, Chapman University, Orange, CA, USA.
| | - Tania A Reynolds
- Psychology Department, University of New Mexico, Albuquerque, NM, USA; The Kinsey Institute, Indiana University, Bloomington, IN, USA
| | - Carlos A Barrera
- Department of Psychology, University of California, San Diego, La Jolla, CA, USA
| | - Stuart B Murray
- Department of Psychiatry & Behavioral Sciences, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA
| |
Collapse
|
28
|
Taubert J, Wardle SG, Tardiff CT, Koele EA, Kumar S, Messinger A, Ungerleider LG. The cortical and subcortical correlates of face pareidolia in the macaque brain. Soc Cogn Affect Neurosci 2022; 17:965-976. [PMID: 35445247 PMCID: PMC9629476 DOI: 10.1093/scan/nsac031] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Revised: 03/27/2022] [Accepted: 04/19/2022] [Indexed: 01/12/2023] Open
Abstract
Face detection is a foundational social skill for primates. This vital function is thought to be supported by specialized neural mechanisms; however, although several face-selective regions have been identified in both humans and nonhuman primates, there is no consensus about which region(s) are involved in face detection. Here, we used naturally occurring errors of face detection (i.e. objects with illusory facial features referred to as examples of 'face pareidolia') to identify regions of the macaque brain implicated in face detection. Using whole-brain functional magnetic resonance imaging to test awake rhesus macaques, we discovered that a subset of face-selective patches in the inferior temporal cortex, on the lower lateral edge of the superior temporal sulcus, and the amygdala respond more to objects with illusory facial features than matched non-face objects. Multivariate analyses of the data revealed differences in the representation of illusory faces across the functionally defined regions of interest. These differences suggest that the cortical and subcortical face-selective regions contribute uniquely to the detection of facial features. We conclude that face detection is supported by a multiplexed system in the primate brain.
Collapse
Affiliation(s)
- Jessica Taubert
- Correspondence should be addressed to Jessica Taubert, School of Psychology, The University of Queensland, Building 24A, St Lucia, QLD 4067, Australia. E-mail:
| | - Susan G Wardle
- Laboratory of Brain and Cognition, The National Institute of Mental Health, NIH, Bethesda, MD 20892, USA
| | - Clarissa T Tardiff
- Laboratory of Brain and Cognition, The National Institute of Mental Health, NIH, Bethesda, MD 20892, USA
| | - Elissa A Koele
- Laboratory of Brain and Cognition, The National Institute of Mental Health, NIH, Bethesda, MD 20892, USA
| | - Susheel Kumar
- Laboratory of Brain and Cognition, The National Institute of Mental Health, NIH, Bethesda, MD 20892, USA
| | - Adam Messinger
- Laboratory of Brain and Cognition, The National Institute of Mental Health, NIH, Bethesda, MD 20892, USA
| | | |
Collapse
|
29
|
Jovanovic V, Fishbein AR, de la Mothe L, Lee KF, Miller CT. Behavioral context affects social signal representations within single primate prefrontal cortex neurons. Neuron 2022; 110:1318-1326.e4. [PMID: 35108498 PMCID: PMC10064486 DOI: 10.1016/j.neuron.2022.01.020] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Revised: 12/19/2021] [Accepted: 01/14/2022] [Indexed: 11/15/2022]
Abstract
We tested whether social signal processing in more traditional, head-restrained contexts is representative of the putative natural analog-social communication-by comparing responses to vocalizations within individual neurons in marmoset prefrontal cortex (PFC) across a series of behavioral contexts ranging from traditional to naturalistic. Although vocalization-responsive neurons were evident in all contexts, cross-context consistency was notably limited. A response to these social signals when subjects were head-restrained was not predictive of a comparable neural response to the identical vocalizations during natural communication. This pattern was evident both within individual neurons and at a population level, as PFC activity could be reliably decoded for the behavioral context in which vocalizations were heard. These results suggest that neural representations of social signals in primate PFC are not static but highly flexible and likely reflect how nuances of the dynamic behavioral contexts affect the perception of these signals and what they communicate.
Collapse
Affiliation(s)
- Vladimir Jovanovic
- Cortical Systems and Behavior Laboratory, University of California, San Diego, La Jolla, CA 92093, USA; Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA 92093, USA
| | - Adam Ryan Fishbein
- Cortical Systems and Behavior Laboratory, University of California, San Diego, La Jolla, CA 92093, USA
| | - Lisa de la Mothe
- Department of Psychology, Tennessee State University, Nashville, TN 37209, USA
| | - Kuo-Fen Lee
- Laboratory for Peptide Biology, Salk Institute, La Jolla, CA 92093, USA
| | - Cory Thomas Miller
- Cortical Systems and Behavior Laboratory, University of California, San Diego, La Jolla, CA 92093, USA; Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA 92093, USA.
| |
Collapse
|
30
|
Dobs K, Martinez J, Kell AJE, Kanwisher N. Brain-like functional specialization emerges spontaneously in deep neural networks. SCIENCE ADVANCES 2022; 8:eabl8913. [PMID: 35294241 PMCID: PMC8926347 DOI: 10.1126/sciadv.abl8913] [Citation(s) in RCA: 39] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Accepted: 01/21/2022] [Indexed: 05/10/2023]
Abstract
The human brain contains multiple regions with distinct, often highly specialized functions, from recognizing faces to understanding language to thinking about what others are thinking. However, it remains unclear why the cortex exhibits this high degree of functional specialization in the first place. Here, we consider the case of face perception using artificial neural networks to test the hypothesis that functional segregation of face recognition in the brain reflects a computational optimization for the broader problem of visual recognition of faces and other visual categories. We find that networks trained on object recognition perform poorly on face recognition and vice versa and that networks optimized for both tasks spontaneously segregate themselves into separate systems for faces and objects. We then show functional segregation to varying degrees for other visual categories, revealing a widespread tendency for optimization (without built-in task-specific inductive biases) to lead to functional specialization in machines and, we conjecture, also brains.
Collapse
Affiliation(s)
- Katharina Dobs
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Psychology, Justus Liebig University Giessen, Giessen, Germany
| | - Julio Martinez
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Psychology, Stanford University, Stanford, CA, USA
| | | | - Nancy Kanwisher
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
31
|
The cost of beauty: Perception of makeup and male mate choice. CURRENT PSYCHOLOGY 2022. [DOI: 10.1007/s12144-021-02677-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
32
|
Waller BM, Kavanagh E, Micheletta J, Clark PR, Whitehouse J. The face is central to primate multicomponent signals. INT J PRIMATOL 2022. [DOI: 10.1007/s10764-021-00260-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
AbstractA wealth of experimental and observational evidence suggests that faces have become increasingly important in the communication system of primates over evolutionary time and that both the static and moveable aspects of faces convey considerable information. Therefore, whenever there is a visual component to any multicomponent signal the face is potentially relevant. However, the role of the face is not always considered in primate multicomponent communication research. We review the literature and make a case for greater focus on the face going forward. We propose that the face can be overlooked for two main reasons: first, due to methodological difficulty. Examination of multicomponent signals in primates is difficult, so scientists tend to examine a limited number of signals in combination. Detailed examination of the subtle and dynamic components of facial signals is particularly hard to achieve in studies of primates. Second, due to a common assumption that the face contains “emotional” content. A priori categorisation of facial behavior as “emotional” ignores the potentially communicative and predictive information present in the face that might contribute to signals. In short, we argue that the face is central to multicomponent signals (and also many multimodal signals) and suggest future directions for investigating this phenomenon.
Collapse
|
33
|
Meisner OC, Nair A, Chang SWC. Amygdala connectivity and implications for social cognition and disorders. HANDBOOK OF CLINICAL NEUROLOGY 2022; 187:381-403. [PMID: 35964984 PMCID: PMC9436700 DOI: 10.1016/b978-0-12-823493-8.00017-1] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
The amygdala is a hub of subcortical region that is crucial in a wide array of affective and motivation-related behaviors. While early research contributed significantly to our understanding of this region's extensive connections to other subcortical and cortical regions, recent methodological advances have enabled researchers to better understand the details of these circuits and their behavioral contributions. Much of this work has focused specifically on investigating the role of amygdala circuits in social cognition. In this chapter, we review both long-standing knowledge and novel research on the amygdala's structure, function, and involvement in social cognition. We focus specifically on the amygdala's circuits with the medial prefrontal cortex, the orbitofrontal cortex, and the hippocampus, as these regions share extensive anatomic and functional connections with the amygdala. Furthermore, we discuss how dysfunction in the amygdala may contribute to social deficits in clinical disorders including autism spectrum disorder, social anxiety disorder, and Williams syndrome. We conclude that social functions mediated by the amygdala are orchestrated through multiple intricate interactions between the amygdala and its interconnected brain regions, endorsing the importance of understanding the amygdala from network perspectives.
Collapse
Affiliation(s)
- Olivia C Meisner
- Department of Psychology, Yale University, New Haven, CT, United States; Interdepartmental Neuroscience Program, Yale University, New Haven, CT, United States
| | - Amrita Nair
- Department of Psychology, Yale University, New Haven, CT, United States
| | - Steve W C Chang
- Department of Psychology, Yale University, New Haven, CT, United States; Interdepartmental Neuroscience Program, Yale University, New Haven, CT, United States.
| |
Collapse
|
34
|
MIYASHITA Y. Operating principles of the cerebral cortex as a six-layered network in primates: beyond the classic canonical circuit model. PROCEEDINGS OF THE JAPAN ACADEMY. SERIES B, PHYSICAL AND BIOLOGICAL SCIENCES 2022; 98:93-111. [PMID: 35283409 PMCID: PMC8948418 DOI: 10.2183/pjab.98.007] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Accepted: 10/28/2021] [Indexed: 06/14/2023]
Abstract
The cerebral cortex performs its computations with many six-layered fundamental units, collectively spreading along the cortical sheet. What is the local network structure and the operating dynamics of such a fundamental unit? Previous investigations of primary sensory areas revealed a classic "canonical" circuit model, leading to an expectation of similar circuit organization and dynamics throughout the cortex. This review clarifies the different circuit dynamics at play in the higher association cortex of primates that implements computation for high-level cognition such as memory and attention. Instead of feedforward processing of response selectivity through Layers 4 to 2/3 that the classic canonical circuit stipulates, memory recall in primates occurs in Layer 5/6 with local backward projection to Layer 2/3, after which the retrieved information is sent back from Layer 6 to lower-level cortical areas for further retrieval of nested associations of target attributes. In this review, a novel "dynamic multimode module (D3M)" in the primate association cortex is proposed, as a new "canonical" circuit model performing this operation.
Collapse
Affiliation(s)
- Yasushi MIYASHITA
- Department of Physiology, The University of Tokyo School of Medicine, Tokyo, Japan
- Juntendo University, Graduate School of Medicine, Tokyo, Japan
| |
Collapse
|
35
|
Human face and gaze perception is highly context specific and involves bottom-up and top-down neural processing. Neurosci Biobehav Rev 2021; 132:304-323. [PMID: 34861296 DOI: 10.1016/j.neubiorev.2021.11.042] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2021] [Revised: 11/24/2021] [Accepted: 11/24/2021] [Indexed: 11/21/2022]
Abstract
This review summarizes human perception and processing of face and gaze signals. Face and gaze signals are important means of non-verbal social communication. The review highlights that: (1) some evidence is available suggesting that the perception and processing of facial information starts in the prenatal period; (2) the perception and processing of face identity, expression and gaze direction is highly context specific, the effect of race and culture being a case in point. Culture affects by means of experiential shaping and social categorization the way in which information on face and gaze is collected and perceived; (3) face and gaze processing occurs in the so-called 'social brain'. Accumulating evidence suggests that the processing of facial identity, facial emotional expression and gaze involves two parallel and interacting pathways: a fast and crude subcortical route and a slower cortical pathway. The flow of information is bi-directional and includes bottom-up and top-down processing. The cortical networks particularly include the fusiform gyrus, superior temporal sulcus (STS), intraparietal sulcus, temporoparietal junction and medial prefrontal cortex.
Collapse
|
36
|
One object, two networks? Assessing the relationship between the face and body-selective regions in the primate visual system. Brain Struct Funct 2021; 227:1423-1438. [PMID: 34792643 DOI: 10.1007/s00429-021-02420-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Accepted: 10/22/2021] [Indexed: 10/19/2022]
Abstract
Faces and bodies are often treated as distinct categories that are processed separately by face- and body-selective brain regions in the primate visual system. These regions occupy distinct regions of visual cortex and are often thought to constitute independent functional networks. Yet faces and bodies are part of the same object and their presence inevitably covary in naturalistic settings. Here, we re-evaluate both the evidence supporting the independent processing of faces and bodies and the organizational principles that have been invoked to explain this distinction. We outline four hypotheses ranging from completely separate networks to a single network supporting the perception of whole people or animals. The current evidence, especially in humans, is compatible with all of these hypotheses, making it presently unclear how the representation of faces and bodies is organized in the cortex.
Collapse
|
37
|
Hauk O, Rice GE, Volfart A, Magnabosco F, Ralph MAL, Rossion B. Face-selective responses in combined EEG/MEG recordings with fast periodic visual stimulation (FPVS). Neuroimage 2021; 242:118460. [PMID: 34363957 PMCID: PMC8463833 DOI: 10.1016/j.neuroimage.2021.118460] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Revised: 07/14/2021] [Accepted: 08/04/2021] [Indexed: 11/29/2022] Open
Abstract
Fast periodic visual stimulation (FPVS) allows the recording of objective brain responses of human face categorization (i.e., generalizable face-selective responses) with high signal-to-noise ratio. This approach has been successfully employed in a number of scalp electroencephalography (EEG) studies but has not been used with magnetoencephalography (MEG) yet, let alone with combined MEG/EEG recordings and distributed source estimation. Here, we presented various natural images of faces periodically (1.2 Hz) among natural images of objects (base frequency 6 Hz) whilst recording simultaneous EEG and MEG in 15 participants. Both measurement modalities showed face-selective responses at 1.2 Hz and harmonics across participants, with high and comparable signal-to-noise ratio (SNR) in about 3 min of stimulation. The correlation of face categorization responses between EEG and two MEG sensor types was lower than between the two MEG sensor types, indicating that the two sensor modalities provide independent information about the sources of face-selective responses. Face-selective EEG responses were right-lateralized as reported previously, and were numerically but non-significantly right-lateralized in MEG data. Distributed source estimation based on combined EEG/MEG signals confirmed a more bilateral face-selective response in visual brain regions located anteriorly to the common response to all stimuli at 6 Hz and harmonics. Conventional sensor and source space analyses of evoked responses in the time domain further corroborated this result. Our results demonstrate that FPVS in combination with simultaneously recorded EEG and MEG may serve as an efficient localizer paradigm for human face categorization.
Collapse
Affiliation(s)
- O Hauk
- MRC Cognition and Brain Sciences Unit, University of Cambridge, 15 Chaucer Road, Cambridge CB2 7EF, UK.
| | - G E Rice
- MRC Cognition and Brain Sciences Unit, University of Cambridge, 15 Chaucer Road, Cambridge CB2 7EF, UK
| | - A Volfart
- Université de Lorraine, CNRS, CRAN UMR 7039, Nancy F-54000, France; Research Institute for Psychological Science, University of Louvain, Louvain-la-Neuve, Belgium
| | - F Magnabosco
- MRC Cognition and Brain Sciences Unit, University of Cambridge, 15 Chaucer Road, Cambridge CB2 7EF, UK
| | - M A Lambon Ralph
- MRC Cognition and Brain Sciences Unit, University of Cambridge, 15 Chaucer Road, Cambridge CB2 7EF, UK
| | - B Rossion
- Université de Lorraine, CNRS, CRAN UMR 7039, Nancy F-54000, France; Université de Lorraine, CHRU-Nancy, Service de Neurologie, Nancy F-54000, France
| |
Collapse
|
38
|
Hermann P, Weiss B, Knakker B, Madurka P, Manga A, Nárai Á, Vidnyánszky Z. Neural basis of distractor resistance during visual working memory maintenance. Neuroimage 2021; 245:118650. [PMID: 34687860 DOI: 10.1016/j.neuroimage.2021.118650] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2021] [Revised: 09/10/2021] [Accepted: 10/11/2021] [Indexed: 01/12/2023] Open
Abstract
Visual working memory representations must be protected from the intervening irrelevant visual input. While it is well known that interference resistance is most challenging when distractors match the prioritised mnemonic information, its neural mechanisms remain poorly understood. Here, we identify two top-down attentional control processes that have opposing effects on distractor resistance. We reveal an early selection negativity in the EEG responses to matching as compared to non-matching distractors, the magnitude of which is negatively associated with behavioural distractor resistance. Additionally, matching distractors lead to reduced post-stimulus alpha power as well as increased fMRI responses in the object-selective visual cortical areas and the inferior frontal gyrus. However, the congruency effect found on the post-stimulus periodic alpha power and the inferior frontal gyrus fMRI responses show a positive association with distractor resistance. These findings suggest that distractor interference is enhanced by proactive memory content-guided selection processes and diminished by reactive allocation of top-down attentional resources to protect memorandum representations within visual cortical areas retaining the most selective mnemonic code.
Collapse
Affiliation(s)
- Petra Hermann
- Brain Imaging Centre, Research Centre for Natural Sciences, Budapest H-1117, Hungary
| | - Béla Weiss
- Brain Imaging Centre, Research Centre for Natural Sciences, Budapest H-1117, Hungary
| | - Balázs Knakker
- Brain Imaging Centre, Research Centre for Natural Sciences, Budapest H-1117, Hungary
| | - Petra Madurka
- Brain Imaging Centre, Research Centre for Natural Sciences, Budapest H-1117, Hungary
| | - Annamária Manga
- Brain Imaging Centre, Research Centre for Natural Sciences, Budapest H-1117, Hungary
| | - Ádám Nárai
- Brain Imaging Centre, Research Centre for Natural Sciences, Budapest H-1117, Hungary
| | - Zoltán Vidnyánszky
- Brain Imaging Centre, Research Centre for Natural Sciences, Budapest H-1117, Hungary.
| |
Collapse
|
39
|
Linear Integration of Sensory Evidence over Space and Time Underlies Face Categorization. J Neurosci 2021; 41:7876-7893. [PMID: 34326145 DOI: 10.1523/jneurosci.3055-20.2021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2020] [Revised: 07/08/2021] [Accepted: 07/21/2021] [Indexed: 11/21/2022] Open
Abstract
Visual object recognition relies on elaborate sensory processes that transform retinal inputs to object representations, but it also requires decision-making processes that read out object representations and function over prolonged time scales. The computational properties of these decision-making processes remain underexplored for object recognition. Here, we study these computations by developing a stochastic multifeature face categorization task. Using quantitative models and tight control of spatiotemporal visual information, we demonstrate that human subjects (five males, eight females) categorize faces through an integration process that first linearly adds the evidence conferred by task-relevant features over space to create aggregated momentary evidence and then linearly integrates it over time with minimum information loss. Discrimination of stimuli along different category boundaries (e.g., identity or expression of a face) is implemented by adjusting feature weights of spatial integration. This linear but flexible integration process over space and time bridges past studies on simple perceptual decisions to complex object recognition behavior.SIGNIFICANCE STATEMENT Although simple perceptual decision-making such as discrimination of random dot motion has been successfully explained as accumulation of sensory evidence, we lack rigorous experimental paradigms to study the mechanisms underlying complex perceptual decision-making such as discrimination of naturalistic faces. We develop a stochastic multifeature face categorization task as a systematic approach to quantify the properties and potential limitations of the decision-making processes during object recognition. We show that human face categorization could be modeled as a linear integration of sensory evidence over space and time. Our framework to study object recognition as a spatiotemporal integration process is broadly applicable to other object categories and bridges past studies of object recognition and perceptual decision-making.
Collapse
|
40
|
Thielgen MM, Schade S, Bosé C. Face processing in police service: the relationship between laboratory-based assessment of face processing abilities and performance in a real-world identity matching task. COGNITIVE RESEARCH-PRINCIPLES AND IMPLICATIONS 2021; 6:54. [PMID: 34351527 PMCID: PMC8342700 DOI: 10.1186/s41235-021-00317-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/29/2020] [Accepted: 07/06/2021] [Indexed: 11/10/2022]
Abstract
In the present study, we investigated whether police officers’ performance in searching for unfamiliar faces in a video-based real-world task is predicted by laboratory-based face processing tests that are typically used to assess individual differences in face processing abilities. Specifically, perceptual performance in the field was operationalized via the identification of target individuals in self-made close-circuit television (CCTV) video tapes. Police officers’ abilities in the laboratory were measured by the Cambridge Face Memory Test long form (CFMT+). We hypothesized that the CFMT+ predicts individual differences in the CCTV task performance. A total of N = 186 police officers of the Rhineland-Palatinate State Police participated in the study (i.e., N = 139 novice and advanced cadets with either 3 months, 15 months or 24 months of pre-service experience; N = 47 experienced police officers with three years of pre-service experience and at least two years of full-service experience, who participated in the assessment center of the special police forces, specifically the surveillance and technical unit). Results revealed that the CFMT+ explained variance in the CCTV task. In sample 1, CFMT+ scores predicted hits, but not false alarms. In contrast, in sample 2, CFMT+ scores were correlated with both hits and false alarms. From a theoretical perspective, we discuss factors that might explain CCTV task performance. From a practical perspective, we recommend that personnel selection processes investigating individual differences of police officers’ face processing abilities should comprise of two steps. At first, laboratory-based tests of face processing abilities should be applied. Subsequently, to validate laboratory-based individual differences in face processing abilities, we recommend that work samples such as CCTV tasks from the field should be added.
Collapse
Affiliation(s)
- Markus M Thielgen
- Department I - University Education, Area of Study VIII - Social Sciences, Rhineland-Palatinate Police University, Post Box 1111, 55482, Hahn-Airport, Germany.
| | - Stefan Schade
- Department I - University Education, Area of Study VIII - Social Sciences, Rhineland-Palatinate Police University, Post Box 1111, 55482, Hahn-Airport, Germany
| | - Carolin Bosé
- Department I - University Education, Area of Study VIII - Social Sciences, Rhineland-Palatinate Police University, Post Box 1111, 55482, Hahn-Airport, Germany
| |
Collapse
|
41
|
Mentink LJ, Guimarães JPOFT, Faber M, Sprooten E, Olde Rikkert MGM, Haak KV, Beckmann CF. Functional co-activation of the default mode network in APOE ε4-carriers: A replication study. Neuroimage 2021; 240:118304. [PMID: 34329959 DOI: 10.1016/j.neuroimage.2021.118304] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2021] [Revised: 05/27/2021] [Accepted: 06/22/2021] [Indexed: 11/19/2022] Open
Abstract
Structural and functional alterations of the brain in persons genetically at-risk for Alzheimer's disease (AD) are crucial in unravelling AD development. Filippini et al. found that the default mode network (DMN) is already affected in young APOE ε4-carriers, with increased co-activation of the DMN during rest and increased hippocampal task activation. We aimed to replicate the early findings of Filippini et al, using the APOE gene, still the principal AD risk gene, and extended this with a polygenic risk score (PRS) analysis for AD, using the Human Connectome Project dataset (HCP). We included participants from the HCP S1200 dataset (age range: 22-36 years). We studied morphometric features, functional DMN co-activation and functional task activation of recollection performance. Permutation Analysis of Linear Models (PALM) was used to test for group differences between APOE ε4-carriers and non-carriers, and to test the association with PRS. PALM controls for biases induced by the family structure of the HCP sample. Results were family-wise error rate corrected at p < 0.05. Our primary analysis did not replicate the early findings of Filippini et al. (2009). However, compared with non-carriers, APOE ε4-carriers showed increased functional activation during the encoding of subsequently recollected items in areas related to facial recognition (p<0.05, t>756.11). This increased functional activation was also positively associated with PRS (APOE variants included) (p<0.05, t>647.55). Our results are supportive for none to limited genetic effects on brain structure and function in young adults. Taking the methodological considerations of replication studies into account, the true effect of APOE ε4-carriership is likely smaller than indicated in the Filippini paper. However, it still holds that we may not yet be able to detect already present measurable effects decades before a clinical expression of AD. Since the mechanistic pathway of AD is likely to encompass many different factors, further research should be focused on the interactions of genetic risk, biomarkers, aging and lifestyle factors over the life course. Sensitive functional neuroimaging as used here may help disentangling these complex interactions.
Collapse
Affiliation(s)
- Lara J Mentink
- Department of Geriatrics, Radboudumc Alzheimer Centre, Radboud University Medical Center, Nijmegen, The Netherlands; Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - João P O F T Guimarães
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Myrthe Faber
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen, The Netherlands; Department of Communication and Cognition, Tilburg Center for Cognition and Communication, Tilburg University, Tilburg, The Netherlands.
| | - Emma Sprooten
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen, The Netherlands; Department of Human Genetics, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Marcel G M Olde Rikkert
- Department of Geriatrics, Radboudumc Alzheimer Centre, Radboud University Medical Center, Nijmegen, The Netherlands; Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Koen V Haak
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Christian F Beckmann
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen, The Netherlands; Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, United Kingdom.
| |
Collapse
|
42
|
Xu S, Zhang Y, Zhen Z, Liu J. The Face Module Emerged in a Deep Convolutional Neural Network Selectively Deprived of Face Experience. Front Comput Neurosci 2021; 15:626259. [PMID: 34093154 PMCID: PMC8173218 DOI: 10.3389/fncom.2021.626259] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Accepted: 04/14/2021] [Indexed: 12/02/2022] Open
Abstract
Can we recognize faces with zero experience on faces? This question is critical because it examines the role of experiences in the formation of domain-specific modules in the brain. Investigation with humans and non-human animals on this issue cannot easily dissociate the effect of the visual experience from that of the hardwired domain-specificity. Therefore, the present study built a model of selective deprivation of the experience on faces with a representative deep convolutional neural network, AlexNet, by removing all images containing faces from its training stimuli. This model did not show significant deficits in face categorization and discrimination, and face-selective modules automatically emerged. However, the deprivation reduced the domain-specificity of the face module. In sum, our study provides empirical evidence on the role of nature vs. nurture in developing the domain-specific modules that domain-specificity may evolve from non-specific experience without genetic predisposition, and is further fine-tuned by domain-specific experience.
Collapse
Affiliation(s)
- Shan Xu
- Beijing Key Laboratory of Applied Experimental Psychology, Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Yiyuan Zhang
- Beijing Key Laboratory of Applied Experimental Psychology, Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Zonglei Zhen
- Beijing Key Laboratory of Applied Experimental Psychology, Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Jia Liu
- Department of Psychology & Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing, China
| |
Collapse
|
43
|
Steel A, Billings MM, Silson EH, Robertson CE. A network linking scene perception and spatial memory systems in posterior cerebral cortex. Nat Commun 2021; 12:2632. [PMID: 33976141 PMCID: PMC8113503 DOI: 10.1038/s41467-021-22848-z] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2020] [Accepted: 04/05/2021] [Indexed: 02/03/2023] Open
Abstract
The neural systems supporting scene-perception and spatial-memory systems of the human brain are well-described. But how do these neural systems interact? Here, using fine-grained individual-subject fMRI, we report three cortical areas of the human brain, each lying immediately anterior to a region of the scene perception network in posterior cerebral cortex, that selectively activate when recalling familiar real-world locations. Despite their close proximity to the scene-perception areas, network analyses show that these regions constitute a distinct functional network that interfaces with spatial memory systems during naturalistic scene understanding. These "place-memory areas" offer a new framework for understanding how the brain implements memory-guided visual behaviors, including navigation.
Collapse
Affiliation(s)
- Adam Steel
- grid.254880.30000 0001 2179 2404Department of Psychology and Brain Sciences, Dartmouth College, Hanover, NH USA
| | - Madeleine M. Billings
- grid.254880.30000 0001 2179 2404Department of Psychology and Brain Sciences, Dartmouth College, Hanover, NH USA
| | - Edward H. Silson
- grid.4305.20000 0004 1936 7988Psychology, School of Philosophy, Psychology, and Language Sciences, University of Edinburgh, Edinburgh, EH8 9JZ UK
| | - Caroline E. Robertson
- grid.254880.30000 0001 2179 2404Department of Psychology and Brain Sciences, Dartmouth College, Hanover, NH USA
| |
Collapse
|
44
|
Finzi D, Gomez J, Nordt M, Rezai AA, Poltoratski S, Grill-Spector K. Differential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections. Nat Commun 2021; 12:2278. [PMID: 33859195 PMCID: PMC8050273 DOI: 10.1038/s41467-021-22524-2] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Accepted: 03/12/2021] [Indexed: 01/01/2023] Open
Abstract
Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.
Collapse
Affiliation(s)
- Dawn Finzi
- Department of Psychology, Stanford University, Stanford, CA, USA.
| | - Jesse Gomez
- Neurosciences Program, Stanford University, Stanford, CA, USA
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Marisa Nordt
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Alex A Rezai
- Department of Psychology, Stanford University, Stanford, CA, USA
| | | | - Kalanit Grill-Spector
- Department of Psychology, Stanford University, Stanford, CA, USA
- Neurosciences Program, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| |
Collapse
|
45
|
Maffei A, Sessa P. Event-related network changes unfold the dynamics of cortical integration during face processing. Psychophysiology 2021; 58:e13786. [PMID: 33550632 DOI: 10.1111/psyp.13786] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Revised: 01/18/2021] [Accepted: 01/22/2021] [Indexed: 11/28/2022]
Abstract
Face perception arises from a collective activation of brain regions in the occipital, parietal and temporal cortices. Despite the wide acknowledgment that these regions act in an intertwined network, the network behavior itself is poorly understood. Here we present a study in which time-varying connectivity estimated from EEG activity elicited by facial expressions presentation was characterized using graph-theoretical measures of node centrality and global network topology. Results revealed that face perception results from a dynamic reshaping of the network architecture, characterized by the emergence of hubs located in the occipital and temporal regions of the scalp. The importance of these nodes can be observed from the early stages of visual processing and reaches a climax in the same time-window in which the face-sensitive N170 is observed. Furthermore, using Granger causality, we found that the time-evolving centrality of these nodes is associated with ERP amplitude, providing a direct link between the network state and local neural response. Additionally, investigating global network topology by means of small-worldness and modularity, we found that face processing requires a functional network with a strong small-world organization that maximizes integration, at the cost of segregated subdivisions. Interestingly, we found that this architecture is not static, but instead, it is implemented by the network from stimulus onset to ~200 ms. Altogether, this study reveals the event-related changes underlying face processing at the network level, suggesting that a distributed processing mechanism operates through dynamically weighting the contribution of the cortical regions involved.
Collapse
Affiliation(s)
- Antonio Maffei
- Padova Neuroscience Center (PNC), University of Padova, Padova, Italy
| | - Paola Sessa
- Padova Neuroscience Center (PNC), University of Padova, Padova, Italy.,Department of Developmental and Social Psychology, University of Padova, Padova, Italy
| |
Collapse
|
46
|
Vestner T, Gray KLH, Cook R. Visual search for facing and non-facing people: The effect of actor inversion. Cognition 2020; 208:104550. [PMID: 33360076 DOI: 10.1016/j.cognition.2020.104550] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Revised: 12/08/2020] [Accepted: 12/11/2020] [Indexed: 10/22/2022]
Abstract
In recent years, there has been growing interest in how human observers perceive, attend to, and recall, social interactions viewed from third-person perspectives. One of the interesting findings to emerge from this new literature is the search advantage for facing dyads. When hidden amongst pairs of individuals facing in the same direction, pairs of individuals arranged front-to-front are found faster in visual search tasks than pairs of individuals arranged back-to-back. Interestingly, the search advantage for facing dyads appears to be sensitive to the orientation of the people depicted. While front-to-front target pairs are found faster than back-to-back targets when target and distractor pairings are shown upright, front-to-front and back-to-back targets are found equally quickly when pairings are shown upside-down. In the present study, we sought to better understand why the search advantage for facing dyads is sensitive to the orientation of the people depicted. To begin, we show that the orientation sensitivity of the search advantage is seen with dyads constructed from faces only, and from bodies with the head and face occluded. We replicate these effects using two different visual search paradigms. We go on to show that individual faces and bodies, viewed in profile, produce strong attentional cueing effects when shown upright, but not when presented upside-down. Together with recent evidence that arrows arranged front-to-front also produce the search advantage for facing dyads, these findings support the view that the search advantage is a by-product of the ability of constituent elements to direct observers' visuo-spatial attention.
Collapse
Affiliation(s)
- Tim Vestner
- Department of Psychological Sciences, Birkbeck, University of London, London, UK
| | - Katie L H Gray
- School of Psychology and Clinical Language Sciences, University of Reading, Reading, UK
| | - Richard Cook
- Department of Psychological Sciences, Birkbeck, University of London, London, UK; Department of Psychology, University of York, York, UK.
| |
Collapse
|
47
|
|
48
|
Kujala MV, Kauppi JP, Törnqvist H, Helle L, Vainio O, Kujala J, Parkkonen L. Time-resolved classification of dog brain signals reveals early processing of faces, species and emotion. Sci Rep 2020; 10:19846. [PMID: 33199715 PMCID: PMC7669855 DOI: 10.1038/s41598-020-76806-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Accepted: 10/30/2020] [Indexed: 11/22/2022] Open
Abstract
Dogs process faces and emotional expressions much like humans, but the time windows important for face processing in dogs are largely unknown. By combining our non-invasive electroencephalography (EEG) protocol on dogs with machine-learning algorithms, we show category-specific dog brain responses to pictures of human and dog facial expressions, objects, and phase-scrambled faces. We trained a support vector machine classifier with spatiotemporal EEG data to discriminate between responses to pairs of images. The classification accuracy was highest for humans or dogs vs. scrambled images, with most informative time intervals of 100–140 ms and 240–280 ms. We also detected a response sensitive to threatening dog faces at 30–40 ms; generally, responses differentiating emotional expressions were found at 130–170 ms, and differentiation of faces from objects occurred at 120–130 ms. The cortical sources underlying the highest-amplitude EEG signals were localized to the dog visual cortex.
Collapse
Affiliation(s)
- Miiamaaria V Kujala
- Department of Psychology, Faculty of Education and Psychology, University of Jyväskylä, PO Box 35, 40014, Jyväskylä, Finland. .,Department of Equine and Small Animal Medicine, Faculty of Veterinary Medicine, University of Helsinki, PL 57, 00014, Helsinki, Finland. .,Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, P.O. Box 12200, 00076, Aalto, Finland.
| | - Jukka-Pekka Kauppi
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, P.O. Box 12200, 00076, Aalto, Finland.,Faculty of Information Technology, University of Jyväskylä, P.O. Box 35, 40014, Jyväskylä, Finland
| | - Heini Törnqvist
- Department of Equine and Small Animal Medicine, Faculty of Veterinary Medicine, University of Helsinki, PL 57, 00014, Helsinki, Finland
| | - Liisa Helle
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, P.O. Box 12200, 00076, Aalto, Finland
| | - Outi Vainio
- Department of Equine and Small Animal Medicine, Faculty of Veterinary Medicine, University of Helsinki, PL 57, 00014, Helsinki, Finland
| | - Jan Kujala
- Department of Psychology, Faculty of Education and Psychology, University of Jyväskylä, PO Box 35, 40014, Jyväskylä, Finland
| | - Lauri Parkkonen
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, P.O. Box 12200, 00076, Aalto, Finland
| |
Collapse
|
49
|
Mapping neural activity patterns to contextualized fearful facial expressions onto callous-unemotional (CU) traits: intersubject representational similarity analysis reveals less variation among high-CU adolescents. PERSONALITY NEUROSCIENCE 2020; 3:e12. [PMID: 33283146 PMCID: PMC7681174 DOI: 10.1017/pen.2020.13] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/28/2020] [Revised: 07/20/2020] [Accepted: 08/10/2020] [Indexed: 12/15/2022]
Abstract
Callous-unemotional (CU) traits are early-emerging personality features characterized by deficits in empathy, concern for others, and remorse following social transgressions. One of the interpersonal deficits most consistently associated with CU traits is impaired behavioral and neurophysiological responsiveness to fearful facial expressions. However, the facial expression paradigms traditionally employed in neuroimaging are often ambiguous with respect to the nature of threat (i.e., is the perceiver the threat, or is something else in the environment?). In the present study, 30 adolescents with varying CU traits viewed fearful facial expressions cued to three different contexts ("afraid for you," "afraid of you," "afraid for self") while undergoing functional magnetic resonance imaging (fMRI). Univariate analyses found that mean right amygdala activity during the "afraid for self" context was negatively associated with CU traits. With the goal of disentangling idiosyncratic stimulus-driven neural responses, we employed intersubject representational similarity analysis to link intersubject similarities in multivoxel neural response patterns to contextualized fearful expressions with differential intersubject models of CU traits. Among low-CU adolescents, neural response patterns while viewing fearful faces were most consistently similar early in the visual processing stream and among regions implicated in affective responding, but were more idiosyncratic as emotional face information moved up the cortical processing hierarchy. By contrast, high-CU adolescents' neural response patterns consistently aligned along the entire cortical hierarchy (but diverged among low-CU youths). Observed patterns varied across contexts, suggesting that interpretations of fearful expressions depend to an extent on neural response patterns and are further shaped by levels of CU traits.
Collapse
|
50
|
Gangopadhyay P, Chawla M, Dal Monte O, Chang SWC. Prefrontal-amygdala circuits in social decision-making. Nat Neurosci 2020; 24:5-18. [PMID: 33169032 DOI: 10.1038/s41593-020-00738-9] [Citation(s) in RCA: 96] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2019] [Accepted: 10/02/2020] [Indexed: 12/20/2022]
Abstract
An increasing amount of research effort is being directed toward investigating the neural bases of social cognition from a systems neuroscience perspective. Evidence from multiple animal species is beginning to provide a mechanistic understanding of the substrates of social behaviors at multiple levels of neurobiology, ranging from those underlying high-level social constructs in humans and their more rudimentary underpinnings in monkeys to circuit-level and cell-type-specific instantiations of social behaviors in rodents. Here we review literature examining the neural mechanisms of social decision-making in humans, non-human primates and rodents, focusing on the amygdala and the medial and orbital prefrontal cortical regions and their functional interactions. We also discuss how the neuropeptide oxytocin impacts these circuits and their downstream effects on social behaviors. Overall, we conclude that regulated interactions of neuronal activity in the prefrontal-amygdala pathways critically contribute to social decision-making in the brains of primates and rodents.
Collapse
Affiliation(s)
| | - Megha Chawla
- Department of Psychology, Yale University, New Haven, CT, USA
| | - Olga Dal Monte
- Department of Psychology, Yale University, New Haven, CT, USA.,Department of Psychology, University of Turin, Torino, Italy
| | - Steve W C Chang
- Department of Psychology, Yale University, New Haven, CT, USA. .,Department of Neuroscience, Yale University School of Medicine, New Haven, CT, USA. .,Kavli Institute for Neuroscience, Yale University School of Medicine, New Haven, CT, USA.
| |
Collapse
|