1
|
Bi Z, Li H, Tian L. Top-down generation of low-resolution representations improves visual perception and imagination. Neural Netw 2024; 171:440-456. [PMID: 38150870 DOI: 10.1016/j.neunet.2023.12.030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Revised: 11/30/2023] [Accepted: 12/18/2023] [Indexed: 12/29/2023]
Abstract
Perception or imagination requires top-down signals from high-level cortex to primary visual cortex (V1) to reconstruct or simulate the representations bottom-up stimulated by the seen images. Interestingly, top-down signals in V1 have lower spatial resolution than bottom-up representations. It is unclear why the brain uses low-resolution signals to reconstruct or simulate high-resolution representations. By modeling the top-down pathway of the visual system using the decoder of a variational auto-encoder (VAE), we reveal that low-resolution top-down signals can better reconstruct or simulate the information contained in the sparse activities of V1 simple cells, which facilitates perception and imagination. This advantage of low-resolution generation is related to facilitating high-level cortex to form geometry-respecting representations observed in experiments. Furthermore, we present two findings regarding this phenomenon in the context of AI-generated sketches, a style of drawings made of lines. First, we found that the quality of the generated sketches critically depends on the thickness of the lines in the sketches: thin-line sketches are harder to generate than thick-line sketches. Second, we propose a technique to generate high-quality thin-line sketches: instead of directly using original thin-line sketches, we use blurred sketches to train VAE or GAN (generative adversarial network), and then infer the thin-line sketches from the VAE- or GAN-generated blurred sketches. Collectively, our work suggests that low-resolution top-down generation is a strategy the brain uses to improve visual perception and imagination, which inspires new sketch-generation AI techniques.
Collapse
Affiliation(s)
- Zedong Bi
- Lingang Laboratory, Shanghai 200031, China.
| | - Haoran Li
- Department of Physics, Hong Kong Baptist University, Hong Kong, China
| | - Liang Tian
- Department of Physics, Hong Kong Baptist University, Hong Kong, China; Institute of Computational and Theoretical Studies, Hong Kong Baptist University, Hong Kong, China; Institute of Systems Medicine and Health Sciences, Hong Kong Baptist University, Hong Kong, China; State Key Laboratory of Environmental and Biological Analysis, Hong Kong Baptist University, Hong Kong, China.
| |
Collapse
|
2
|
Fu Z, Wang X, Wang X, Yang H, Wang J, Wei T, Liao X, Liu Z, Chen H, Bi Y. Different computational relations in language are captured by distinct brain systems. Cereb Cortex 2023; 33:997-1013. [PMID: 35332914 DOI: 10.1093/cercor/bhac117] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2021] [Revised: 02/25/2022] [Accepted: 02/26/2022] [Indexed: 11/14/2022] Open
Abstract
A critical way for humans to acquire information is through language, yet whether and how language experience drives specific neural semantic representations is still poorly understood. We considered statistical properties captured by 3 different computational principles of language (simple co-occurrence, network-(graph)-topological relations, and neural-network-vector-embedding relations) and tested the extent to which they can explain the neural patterns of semantic representations, measured by 2 functional magnetic resonance imaging experiments that shared common semantic processes. Distinct graph-topological word relations, and not simple co-occurrence or neural-network-vector-embedding relations, had unique explanatory power for the neural patterns in the anterior temporal lobe (capturing graph-common-neighbors), inferior frontal gyrus, and posterior middle/inferior temporal gyrus (capturing graph-shortest-path). These results were relatively specific to language: they were not explained by sensory-motor similarities and the same computational relations of visual objects (based on visual image database) showed effects in the visual cortex in the picture naming experiment. That is, different topological properties within language and the same topological computations (common-neighbors) for language and visual inputs are captured by different brain regions. These findings reveal the specific neural semantic representations along graph-topological properties of language, highlighting the information type-specific and statistical property-specific manner of semantic representations in the human brain.
Collapse
Affiliation(s)
- Ze Fu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Xiaoying Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Huichao Yang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China.,School of Systems Science, Beijing Normal University, Beijing 100875, China
| | - Jiahuan Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Tao Wei
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Xuhong Liao
- School of Systems Science, Beijing Normal University, Beijing 100875, China
| | - Zhiyuan Liu
- Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
| | - Huimin Chen
- School of Journalism and Communication, Tsinghua University, Beijing 100084, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China.,Chinese Institute for Brain Research, Beijing 102206, China
| |
Collapse
|
3
|
Souter NE, Wang X, Thompson H, Krieger-Redwood K, Halai AD, Lambon Ralph MA, Thiebaut de Schotten M, Jefferies E. Mapping lesion, structural disconnection, and functional disconnection to symptoms in semantic aphasia. Brain Struct Funct 2022; 227:3043-3061. [PMID: 35786743 PMCID: PMC9653334 DOI: 10.1007/s00429-022-02526-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 06/12/2022] [Indexed: 01/03/2023]
Abstract
Patients with semantic aphasia have impaired control of semantic retrieval, often accompanied by executive dysfunction following left hemisphere stroke. Many but not all of these patients have damage to the left inferior frontal gyrus, important for semantic and cognitive control. Yet semantic and cognitive control networks are highly distributed, including posterior as well as anterior components. Accordingly, semantic aphasia might not only reflect local damage but also white matter structural and functional disconnection. Here, we characterise the lesions and predicted patterns of structural and functional disconnection in individuals with semantic aphasia and relate these effects to semantic and executive impairment. Impaired semantic cognition was associated with infarction in distributed left-hemisphere regions, including in the left anterior inferior frontal and posterior temporal cortex. Lesions were associated with executive dysfunction within a set of adjacent but distinct left frontoparietal clusters. Performance on executive tasks was also associated with interhemispheric structural disconnection across the corpus callosum. In contrast, poor semantic cognition was associated with small left-lateralized structurally disconnected clusters, including in the left posterior temporal cortex. Little insight was gained from functional disconnection symptom mapping. These results demonstrate that while left-lateralized semantic and executive control regions are often damaged together in stroke aphasia, these deficits are associated with distinct patterns of structural disconnection, consistent with the bilateral nature of executive control and the left-lateralized yet distributed semantic control network.
Collapse
Affiliation(s)
| | - Xiuyi Wang
- Department of Psychology, University of York, York, YO10 5DD, UK
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, China
| | - Hannah Thompson
- School of Psychology and Clinical Language Sciences, University of Reading, Reading, UK
| | | | - Ajay D Halai
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK
| | | | - Michel Thiebaut de Schotten
- Brain Connectivity and Behaviour Laboratory, Sorbonne Universities, Paris, France
- Groupe d'Imagerie Neurofonctionnelle, Institut des Maladies Neurodégénératives-UMR 5293, CNRS, CEA, University of Bordeaux, Bordeaux, France
| | | |
Collapse
|
4
|
Cabral L, Zubiaurre-Elorza L, Wild CJ, Linke A, Cusack R. Anatomical correlates of category-selective visual regions have distinctive signatures of connectivity in neonates. Dev Cogn Neurosci 2022; 58:101179. [PMID: 36521345 PMCID: PMC9768242 DOI: 10.1016/j.dcn.2022.101179] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Revised: 11/15/2022] [Accepted: 11/21/2022] [Indexed: 11/25/2022] Open
Abstract
The ventral visual stream is shaped during development by innate proto-organization within the visual system, such as the strong input from the fovea to the fusiform face area. In adults, category-selective regions have distinct signatures of connectivity to brain regions beyond the visual system, likely reflecting cross-modal and motoric associations. We tested if this long-range connectivity is part of the innate proto-organization, or if it develops with postnatal experience, by using diffusion-weighted imaging to characterize the connectivity of anatomical correlates of category-selective regions in neonates (N = 445), 1-9 month old infants (N = 11), and adults (N = 14). Using the HCP data we identified face- and place- selective regions and a third intermediate region with a distinct profile of selectivity. Using linear classifiers, these regions were found to have distinctive connectivity at birth, to other regions in the visual system and to those outside of it. The results support an extended proto-organization that includes long-range connectivity that shapes, and is shaped by, experience-dependent development.
Collapse
Affiliation(s)
- Laura Cabral
- Department of Radiology, University of Pittsburgh, Pittsburgh 15224, PA, USA.
| | - Leire Zubiaurre-Elorza
- Department of Psychology, Faculty of Health Sciences, University of Deusto, Bilbao 48007, Spain
| | - Conor J Wild
- Western Institute for Neuroscience, Western University, London, ON N6A 3K7, Canada; Department of Physiology and Pharmacology,Western University, London, ON N6A 3K7, Canada
| | - Annika Linke
- Brain Development Imaging Laboratories, San Diego State University, San Diego 92120, CA, USA
| | - Rhodri Cusack
- Trinity College Institute of Neuroscience, Trinity College Dublin, College Green, Dublin 2, Ireland
| |
Collapse
|
5
|
Ansteeg L, Leoné F, Dijkstra T. Characterizing the semantic and form-based similarity spaces of the mental lexicon by means of the multi-arrangement method. Front Psychol 2022; 13:945094. [PMID: 36033027 PMCID: PMC9407019 DOI: 10.3389/fpsyg.2022.945094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Accepted: 07/25/2022] [Indexed: 12/04/2022] Open
Abstract
Collecting human similarity judgments is instrumental to measuring and modeling neurocognitive representations (e.g., through representational similarity analysis) and has been made more efficient by the multi-arrangement task. While this task has been tested for collecting semantic similarity judgments, it is unclear whether it also lends itself to phonological and orthographic similarity judgments of words. We have extended the task to include these lexical modalities and compared the results between modalities and against computational models. We find that similarity judgments can be collected for all three modalities, although word forms were considered more difficult to sort and resulted in less consistent inter- and intra-rater agreement than semantics. For all three modalities we can construct stable group-level representational similarity matrices. However, these do not capture significant idiosyncratic similarity information unique to each participant. We discuss the potential underlying causes for differences between modalities and their effect on the application of the multi-arrangement task.
Collapse
|
6
|
Faskowitz J, Betzel RF, Sporns O. Edges in brain networks: Contributions to models of structure and function. Netw Neurosci 2022; 6:1-28. [PMID: 35350585 PMCID: PMC8942607 DOI: 10.1162/netn_a_00204] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Accepted: 08/02/2021] [Indexed: 11/16/2022] Open
Abstract
Network models describe the brain as sets of nodes and edges that represent its distributed organization. So far, most discoveries in network neuroscience have prioritized insights that highlight distinct groupings and specialized functional contributions of network nodes. Importantly, these functional contributions are determined and expressed by the web of their interrelationships, formed by network edges. Here, we underscore the important contributions made by brain network edges for understanding distributed brain organization. Different types of edges represent different types of relationships, including connectivity and similarity among nodes. Adopting a specific definition of edges can fundamentally alter how we analyze and interpret a brain network. Furthermore, edges can associate into collectives and higher order arrangements, describe time series, and form edge communities that provide insights into brain network topology complementary to the traditional node-centric perspective. Focusing on the edges, and the higher order or dynamic information they can provide, discloses previously underappreciated aspects of structural and functional network organization.
Collapse
Affiliation(s)
- Joshua Faskowitz
- Program in Neuroscience, Indiana University, Bloomington, IN, USA
- Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA
| | - Richard F. Betzel
- Program in Neuroscience, Indiana University, Bloomington, IN, USA
- Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA
- Indiana University Network Science Institute, Indiana University, Bloomington, IN, USA
- Cognitive Science Program, Indiana University, Bloomington, IN, USA
| | - Olaf Sporns
- Program in Neuroscience, Indiana University, Bloomington, IN, USA
- Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA
- Indiana University Network Science Institute, Indiana University, Bloomington, IN, USA
- Cognitive Science Program, Indiana University, Bloomington, IN, USA
| |
Collapse
|
7
|
Wang X, Li G, Zhao G, Li Y, Wang B, Lin CP, Liu X, Bi Y. Social and emotion dimensional organizations in the abstract semantic space: the neuropsychological evidence. Sci Rep 2021; 11:23572. [PMID: 34876617 PMCID: PMC8651696 DOI: 10.1038/s41598-021-02824-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 11/12/2021] [Indexed: 11/25/2022] Open
Abstract
An essential aspect of human cognition is supported by a rich reservoir of abstract concepts without tangible external referents (e.g., “honor”, “relationship”, “direction”). While decades of research showed that the neural organization of conceptual knowledge referring to concrete words respects domains of evolutionary salience and sensorimotor attributes, the organization principles of abstract word meanings are poorly understood. Here, we provide neuropsychological evidence for a domain (sociality) and attribute (emotion) structure in abstract word processing. Testing 34 brain-damaged patients on a word-semantic judgment task, we observed double dissociations between social and nonsocial words and a single dissociation of sparing of emotional (relative to non-emotional) words. The lesion profiles of patients with specific dissociations suggest potential neural correlates positively or negatively associated with each dimension. These results unravel a general domain-attribute architecture of word meanings and highlight the roles of the social domain and the emotional attribute in the non-object semantic space.
Collapse
Affiliation(s)
- Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China
| | - Guochao Li
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China
| | - Gang Zhao
- Department of Neurosurgery, First Hospital of Jilin University, Changchun, China
| | - Yunqian Li
- Department of Neurosurgery, First Hospital of Jilin University, Changchun, China
| | - Bijun Wang
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China
| | - Ching-Po Lin
- Department of Neurosurgery, First Hospital of Jilin University, Changchun, China.,Institute of Neuroscience, National Yang-Ming University, Taipei, Taiwan.,Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
| | - Xinrui Liu
- Department of Neurosurgery, First Hospital of Jilin University, Changchun, China.
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China. .,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China. .,Chinese Institute for Brain Research, Beijing, China.
| |
Collapse
|
8
|
Dual coding of knowledge in the human brain. Trends Cogn Sci 2021; 25:883-895. [PMID: 34509366 DOI: 10.1016/j.tics.2021.07.006] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2021] [Revised: 07/10/2021] [Accepted: 07/14/2021] [Indexed: 11/23/2022]
Abstract
How does the human brain code knowledge about the world? While disciplines such as artificial intelligence represent world knowledge based on human language, neurocognitive models of knowledge have been dominated by sensory embodiment, in which knowledge is derived from sensory/motor experience and supported by high-level sensory/motor and association cortices. The neural correlates of an alternative disembodied symbolic system had previously been difficult to establish. A recent line of studies exploring knowledge about visual properties, such as color, in visually deprived individuals converge to provide positive, compelling evidence for non-sensory, language-derived, knowledge representation in dorsal anterior temporal lobe and extended language network, in addition to the sensory-derived representations, leading to a sketch of a dual-coding knowledge neural framework.
Collapse
|
9
|
Li M, Song L, Zhang Y, Han Z. White matter network of oral word reading identified by network-based lesion-symptom mapping. iScience 2021; 24:102862. [PMID: 34386727 PMCID: PMC8346667 DOI: 10.1016/j.isci.2021.102862] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2020] [Revised: 04/29/2021] [Accepted: 07/13/2021] [Indexed: 11/19/2022] Open
Abstract
Oral word reading is supported by a neural subnetwork that includes gray matter regions and white matter tracts connected by the regions. Traditional methods typically determine the reading-relevant focal gray matter regions or white matter tracts rather than the reading-relevant global subnetwork. The present study developed a network-based lesion-symptom mapping (NLSM) method to identify the reading-relevant global white matter subnetwork in 84 brain-damaged patients. The global subnetwork was selected among all possible subnetworks because its global efficiency exhibited the best explanatory power for patients' reading scores. This reading subnetwork was left lateralized and included 7 gray matter regions and 15 white matter tracts. Moreover, the reading subnetwork had additional explanatory power for the patients' reading performance after eliminating the effects of reading-related local regions and tracts. These findings refine the reading neuroanatomical architecture and indicate that the NLSM can be a better method for revealing behavior-specific subnetworks.
Collapse
Affiliation(s)
- Mingyang Li
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Luping Song
- Shenzhen University General Hospital, Department of Rehabilitation Medicine, Shenzhen 518055, China
| | - Yumei Zhang
- Department of Rehabilitation Medicine, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| | - Zaizhu Han
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
- Corresponding author
| |
Collapse
|
10
|
Cognitive impairment after focal brain lesions is better predicted by damage to structural than functional network hubs. Proc Natl Acad Sci U S A 2021; 118:2018784118. [PMID: 33941692 DOI: 10.1073/pnas.2018784118] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Hubs are highly connected brain regions important for coordinating processing in brain networks. It is unclear, however, which measures of network "hubness" are most useful in identifying brain regions critical to human cognition. We tested how closely two measures of hubness-edge density and participation coefficient, derived from white and gray matter, respectively-were associated with general cognitive impairment after brain damage in two large cohorts of patients with focal brain lesions (N = 402 and 102, respectively) using cognitive tests spanning multiple cognitive domains. Lesions disrupting white matter regions with high edge density were associated with cognitive impairment, whereas lesions damaging gray matter regions with high participation coefficient had a weaker, less consistent association with cognitive outcomes. Similar results were observed with six other gray matter hubness measures. This suggests that damage to densely connected white matter regions is more cognitively impairing than similar damage to gray matter hubs, helping to explain interindividual differences in cognitive outcomes after brain damage.
Collapse
|
11
|
Liu C, Kang Y, Zhang L, Zhang J. Rapidly Decoding Image Categories From MEG Data Using a Multivariate Short-Time FC Pattern Analysis Approach. IEEE J Biomed Health Inform 2021; 25:1139-1150. [PMID: 32750957 DOI: 10.1109/jbhi.2020.3008731] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Recent advances in the development of multivariate analysis methods have led to the application of multivariate pattern analysis (MVPA) to investigate the interactions between brain regions using graph theory (functional connectivity, FC) and decode visual categories from functional magnetic resonance imaging (fMRI) data from a continuous multicategory paradigm. To estimate stable FC patterns from fMRI data, previous studies required long periods in the order of several minutes, in comparison to the human brain that categories visual stimuli within hundreds of milliseconds. Constructing short-time dynamic FC patterns in the order of milliseconds and decoding visual categories is a relatively novel concept. In this study, we developed a multivariate decoding algorithm based on FC patterns and applied it to magnetoencephalography (MEG) data. MEG data were recorded from participants presented with image stimuli in four categories (faces, scenes, animals and tools). MEG data from 17 participants demonstrate that short-time dynamic FC patterns yield brain activity patterns that can be used to decode visual categories with high accuracy. Our results show that FC patterns change over the time window, and FC patterns extracted in the time window of 0∼200 ms after the stimulus onset were most stable. Further, the categorizing accuracy peaked (the mean binary accuracy is above 78.6% at individual level) in the FC patterns estimated within the 0∼200 ms interval. These findings elucidate the underlying connectivity information during visual category processing on a relatively smaller time scale and demonstrate that the contribution of FC patterns to categorization fluctuates over time.
Collapse
|
12
|
Wang Y, Metoki A, Smith DV, Medaglia JD, Zang Y, Benear S, Popal H, Lin Y, Olson IR. Multimodal mapping of the face connectome. Nat Hum Behav 2020; 4:397-411. [PMID: 31988441 PMCID: PMC7167350 DOI: 10.1038/s41562-019-0811-3] [Citation(s) in RCA: 40] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2019] [Accepted: 12/09/2019] [Indexed: 01/13/2023]
Abstract
Face processing supports our ability to recognize friend from foe, form tribes and understand the emotional implications of changes in facial musculature. This skill relies on a distributed network of brain regions, but how these regions interact is poorly understood. Here we integrate anatomical and functional connectivity measurements with behavioural assays to create a global model of the face connectome. We dissect key features, such as the network topology and fibre composition. We propose a neurocognitive model with three core streams; face processing along these streams occurs in a parallel and reciprocal manner. Although long-range fibre paths are important, the face network is dominated by short-range fibres. Finally, we provide evidence that the well-known right lateralization of face processing arises from imbalanced intra- and interhemispheric connections. In summary, the face network relies on dynamic communication across highly structured fibre tracts, enabling coherent face processing that underpins behaviour and cognition.
Collapse
Affiliation(s)
- Yin Wang
- State Key Laboratory of Cognitive Neuroscience and Learning, and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.
| | - Athanasia Metoki
- Department of Psychology, Temple University, Philadelphia, PA, USA
| | - David V Smith
- Department of Psychology, Temple University, Philadelphia, PA, USA
| | - John D Medaglia
- Department of Psychology, Drexel University, Philadelphia, PA, USA
- Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Yinyin Zang
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | - Susan Benear
- Department of Psychology, Temple University, Philadelphia, PA, USA
| | - Haroon Popal
- Department of Psychology, Temple University, Philadelphia, PA, USA
| | - Ying Lin
- Department of Psychology, Temple University, Philadelphia, PA, USA
| | - Ingrid R Olson
- Department of Psychology, Temple University, Philadelphia, PA, USA.
| |
Collapse
|
13
|
Popal H, Wang Y, Olson IR. A Guide to Representational Similarity Analysis for Social Neuroscience. Soc Cogn Affect Neurosci 2019; 14:1243-1253. [PMID: 31989169 PMCID: PMC7057283 DOI: 10.1093/scan/nsz099] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2019] [Revised: 10/13/2019] [Accepted: 10/22/2019] [Indexed: 01/04/2023] Open
Abstract
Representational similarity analysis (RSA) is a computational technique that uses pairwise comparisons of stimuli to reveal their representation in higher-order space. In the context of neuroimaging, mass-univariate analyses and other multivariate analyses can provide information on what and where information is represented but have limitations in their ability to address how information is represented. Social neuroscience is a field that can particularly benefit from incorporating RSA techniques to explore hypotheses regarding the representation of multidimensional data, how representations can predict behavior, how representations differ between groups and how multimodal data can be compared to inform theories. The goal of this paper is to provide a practical as well as theoretical guide to implementing RSA in social neuroscience studies.
Collapse
Affiliation(s)
- Haroon Popal
- Department of Psychology, Temple University, Philadelphia, PA
| | | | - Ingrid R Olson
- Department of Psychology, Temple University, Philadelphia, PA
| |
Collapse
|
14
|
Liu C, Li Y, Song S, Zhang J. Decoding disparity categories in 3-dimensional images from fMRI data using functional connectivity patterns. Cogn Neurodyn 2019; 14:169-179. [PMID: 32226560 DOI: 10.1007/s11571-019-09557-6] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2018] [Revised: 09/05/2019] [Accepted: 09/29/2019] [Indexed: 02/02/2023] Open
Abstract
Humans use binocular disparity to extract depth information from two-dimensional retinal images in a process called stereopsis. Previous studies usually introduce the standard univariate analysis to describe the correlation between disparity level and brain activity within a given brain region based on functional magnetic resonance imaging (fMRI) data. Recently, multivariate pattern analysis has been developed to extract activity patterns across multiple voxels for deciphering categories of binocular disparity. However, the functional connectivity (FC) of patterns based on regions of interest or voxels and their mapping onto disparity category perception remain unknown. The present study extracted functional connectivity patterns for three disparity conditions (crossed disparity, uncrossed disparity, and zero disparity) at distinct spatial scales to decode the binocular disparity. Results of 27 subjects' fMRI data demonstrate that FC features are more discriminatory than traditional voxel activity features in binocular disparity classification. The average binary classification of the whole brain and visual areas are respectively 87% and 79% at single subject level, and thus above the chance level (50%). Our research highlights the importance of exploring functional connectivity patterns to achieve a novel understanding of 3D image processing.
Collapse
Affiliation(s)
- Chunyu Liu
- 1College of Information Science and Technology, Beijing Normal University, Beijing, China
| | - Yuan Li
- 2School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Sutao Song
- 3School of Education and Psychology, University of Jinan, Jinan, China
| | - Jiacai Zhang
- 1College of Information Science and Technology, Beijing Normal University, Beijing, China
| |
Collapse
|
15
|
Associative white matter connecting the dorsal and ventral posterior human cortex. Brain Struct Funct 2019; 224:2631-2660. [DOI: 10.1007/s00429-019-01907-8] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2018] [Accepted: 06/07/2019] [Indexed: 02/05/2023]
|
16
|
Abstract
Understanding how knowledge is represented in the human brain is a fundamental challenge in neuroscience. To date, most of the work on this topic has focused on knowledge representation in cortical areas and debated whether knowledge is represented in a distributed or localized fashion. Fang and colleagues provide evidence that brain connections and the white matter supporting such connections might play a significant role. The work opens new avenues of investigation, breaking through disciplinary boundaries across network neuroscience, computational neuroscience, cognitive science, and classical lesion studies.
Collapse
Affiliation(s)
- Franco Pestilli
- Indiana University Bloomington, Department of Psychological and Brain Sciences, Programs in Cognitive Science and Neuroscience, Indiana Network Science Institute, School of Optometry, Departments of Intelligent Systems Engineering and Computer Science, University of Indiana, Bloomington, Indiana, United States of America
- * E-mail:
| |
Collapse
|