1
|
Lahner B, Dwivedi K, Iamshchinina P, Graumann M, Lascelles A, Roig G, Gifford AT, Pan B, Jin S, Ratan Murty NA, Kay K, Oliva A, Cichy R. Modeling short visual events through the BOLD moments video fMRI dataset and metadata. Nat Commun 2024; 15:6241. [PMID: 39048577 PMCID: PMC11269733 DOI: 10.1038/s41467-024-50310-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 07/04/2024] [Indexed: 07/27/2024] Open
Abstract
Studying the neural basis of human dynamic visual perception requires extensive experimental data to evaluate the large swathes of functionally diverse brain neural networks driven by perceiving visual events. Here, we introduce the BOLD Moments Dataset (BMD), a repository of whole-brain fMRI responses to over 1000 short (3 s) naturalistic video clips of visual events across ten human subjects. We use the videos' extensive metadata to show how the brain represents word- and sentence-level descriptions of visual events and identify correlates of video memorability scores extending into the parietal cortex. Furthermore, we reveal a match in hierarchical processing between cortical regions of interest and video-computable deep neural networks, and we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. With its rich metadata, BMD offers new perspectives and accelerates research on the human brain basis of visual event perception.
Collapse
Affiliation(s)
- Benjamin Lahner
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA.
| | - Kshitij Dwivedi
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
- Department of Computer Science, Goethe University Frankfurt, Frankfurt am Main, Germany
| | - Polina Iamshchinina
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | - Monika Graumann
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | - Alex Lascelles
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
| | - Gemma Roig
- Department of Computer Science, Goethe University Frankfurt, Frankfurt am Main, Germany
- The Hessian Center for AI (hessian.AI), Darmstadt, Germany
| | | | - Bowen Pan
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
| | - SouYoung Jin
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
| | - N Apurva Ratan Murty
- Department of Brain and Cognitive Science, MIT, Cambridge, MA, USA
- School of Psychology, Georgia Institute of Technology, Atlanta, GA, USA
| | - Kendrick Kay
- Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota, Minneapolis, MN, USA
| | - Aude Oliva
- Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
| | - Radoslaw Cichy
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| |
Collapse
|
2
|
Kay K, Prince JS, Gebhart T, Tuckute G, Zhou J, Naselaris T, Schutt H. Disentangling signal and noise in neural responses through generative modeling. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.22.590510. [PMID: 38712051 PMCID: PMC11071385 DOI: 10.1101/2024.04.22.590510] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Measurements of neural responses to identically repeated experimental events often exhibit large amounts of variability. This noise is distinct from signal , operationally defined as the average expected response across repeated trials for each given event. Accurately distinguishing signal from noise is important, as each is a target that is worthy of study (many believe noise reflects important aspects of brain function) and it is important not to confuse one for the other. Here, we introduce a principled modeling approach in which response measurements are explicitly modeled as the sum of samples from multivariate signal and noise distributions. In our proposed method-termed Generative Modeling of Signal and Noise (GSN)-the signal distribution is estimated by subtracting the estimated noise distribution from the estimated data distribution. We validate GSN using ground-truth simulations and demonstrate the application of GSN to empirical fMRI data. In doing so, we illustrate a simple consequence of GSN: by disentangling signal and noise components in neural responses, GSN denoises principal components analysis and improves estimates of dimensionality. We end by discussing other situations that may benefit from GSN's characterization of signal and noise, such as estimation of noise ceilings for computational models of neural activity. A code toolbox for GSN is provided with both MATLAB and Python implementations.
Collapse
|
3
|
Stier AJ, Sajjadi S, Karimi F, Bettencourt LMA, Berman MG. Implicit racial biases are lower in more populous more diverse and less segregated US cities. Nat Commun 2024; 15:961. [PMID: 38321002 PMCID: PMC10847142 DOI: 10.1038/s41467-024-45013-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Accepted: 01/04/2024] [Indexed: 02/08/2024] Open
Abstract
Implicit biases - differential attitudes towards members of distinct groups - are pervasive in human societies and create inequities across many aspects of life. Recent research has revealed that implicit biases are generally driven by social contexts, but not whether they are systematically influenced by the ways that humans self-organize in cities. We leverage complex system modeling in the framework of urban scaling theory to predict differences in these biases between cities. Our model links spatial scales from city-wide infrastructure to individual psychology to predict that cities that are more populous, more diverse, and less segregated are less biased. We find empirical support for these predictions in U.S. cities with Implicit Association Test data spanning a decade from 2.7 million individuals and U.S. Census demographic data. Additionally, we find that changes in cities' social environments precede changes in implicit biases at short time-scales, but this relationship is bi-directional at longer time-scales. We conclude that the social organization of cities may influence the strength of these biases.
Collapse
Affiliation(s)
- Andrew J Stier
- Department of Psychology, University of Chicago, Chicago, IL, USA.
- The Santa Fe Institute, Santa Fe, NM, USA.
| | - Sina Sajjadi
- Complexity Science Hub, Vienna, Austria
- Central European University, Vienna, Austria
| | - Fariba Karimi
- Complexity Science Hub, Vienna, Austria
- Graz University of Technology, Graz, Austria
| | - Luís M A Bettencourt
- Department of Ecology and Evolution, University of Chicago, Chicago, IL, USA
- Mansueto Institute for Urban Innovation, University of Chicago, Chicago, IL, USA
| | - Marc G Berman
- Department of Psychology, University of Chicago, Chicago, IL, USA.
- The University of Chicago Neuroscience Institute, University of Chicago, Chicago, IL, USA.
| |
Collapse
|
4
|
Tuckute G, Sathe A, Srikant S, Taliaferro M, Wang M, Schrimpf M, Kay K, Fedorenko E. Driving and suppressing the human language network using large language models. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.16.537080. [PMID: 37090673 PMCID: PMC10120732 DOI: 10.1101/2023.04.16.537080] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/25/2023]
Abstract
Transformer models such as GPT generate human-like language and are highly predictive of human brain responses to language. Here, using fMRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of brain response associated with each sentence. Then, we use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also noninvasively control neural activity in higher-level cortical areas, like the language network.
Collapse
Affiliation(s)
- Greta Tuckute
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
| | - Aalok Sathe
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
| | - Shashank Srikant
- Computer Science & Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- MIT-IBM Watson AI Lab, Cambridge, MA 02142, USA
| | - Maya Taliaferro
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
| | - Mingye Wang
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
| | - Martin Schrimpf
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- Quest for Intelligence, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- Neuro-X Institute, École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland
| | - Kendrick Kay
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455 USA
| | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
- The Program in Speech and Hearing Bioscience and Technology, Harvard University, Cambridge, MA 02138 USA
| |
Collapse
|
5
|
Wang C, Yan H, Huang W, Sheng W, Wang Y, Fan YS, Liu T, Zou T, Li R, Chen H. Neural encoding with unsupervised spiking convolutional neural network. Commun Biol 2023; 6:880. [PMID: 37640808 PMCID: PMC10462614 DOI: 10.1038/s42003-023-05257-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Accepted: 08/18/2023] [Indexed: 08/31/2023] Open
Abstract
Accurately predicting the brain responses to various stimuli poses a significant challenge in neuroscience. Despite recent breakthroughs in neural encoding using convolutional neural networks (CNNs) in fMRI studies, there remain critical gaps between the computational rules of traditional artificial neurons and real biological neurons. To address this issue, a spiking CNN (SCNN)-based framework is presented in this study to achieve neural encoding in a more biologically plausible manner. The framework utilizes unsupervised SCNN to extract visual features of image stimuli and employs a receptive field-based regression algorithm to predict fMRI responses from the SCNN features. Experimental results on handwritten characters, handwritten digits and natural images demonstrate that the proposed approach can achieve remarkably good encoding performance and can be utilized for "brain reading" tasks such as image reconstruction and identification. This work suggests that SNN can serve as a promising tool for neural encoding.
Collapse
Affiliation(s)
- Chong Wang
- The Center of Psychosomatic Medicine, Sichuan Provincial Center for Mental Health, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, 611731, China
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
- MOE Key Lab for Neuroinformation; High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Hongmei Yan
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China.
- MOE Key Lab for Neuroinformation; High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, 610054, China.
| | - Wei Huang
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
- MOE Key Lab for Neuroinformation; High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Wei Sheng
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
- MOE Key Lab for Neuroinformation; High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Yuting Wang
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
- MOE Key Lab for Neuroinformation; High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Yun-Shuang Fan
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
- MOE Key Lab for Neuroinformation; High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Tao Liu
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Ting Zou
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Rong Li
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China.
- MOE Key Lab for Neuroinformation; High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, 610054, China.
| | - Huafu Chen
- The Center of Psychosomatic Medicine, Sichuan Provincial Center for Mental Health, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, 611731, China.
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China.
- MOE Key Lab for Neuroinformation; High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, 610054, China.
| |
Collapse
|
6
|
Marrazzo G, De Martino F, Lage-Castellanos A, Vaessen MJ, de Gelder B. Voxelwise encoding models of body stimuli reveal a representational gradient from low-level visual features to postural features in occipitotemporal cortex. Neuroimage 2023:120240. [PMID: 37348622 DOI: 10.1016/j.neuroimage.2023.120240] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 06/16/2023] [Accepted: 06/19/2023] [Indexed: 06/24/2023] Open
Abstract
Research on body representation in the brain has focused on category-specific representation, using fMRI to investigate the response pattern to body stimuli in occipitotemporal cortex without so far addressing the issue of the specific computations involved in body selective regions, only defined by higher order category selectivity. This study used ultra-high field fMRI and banded ridge regression to investigate the coding of body images, by comparing the performance of three encoding models in predicting brain activity in occipitotemporal cortex and specifically the extrastriate body area (EBA). Our results suggest that bodies are encoded in occipitotemporal cortex and in the EBA according to a combination of low-level visual features and postural features.
Collapse
Affiliation(s)
- Giuseppe Marrazzo
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Limburg 6200 MD, Maastricht, The Netherlands
| | - Federico De Martino
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Limburg 6200 MD, Maastricht, The Netherlands; Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota, Minneapolis, MN 55455, United States and Department of NeuroInformatics
| | - Agustin Lage-Castellanos
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Limburg 6200 MD, Maastricht, The Netherlands; Cuban Center for Neuroscience, Street 190 e/25 and 27 Cubanacán Playa Havana, CP 11600, Cuba
| | - Maarten J Vaessen
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Limburg 6200 MD, Maastricht, The Netherlands
| | - Beatrice de Gelder
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Limburg 6200 MD, Maastricht, The Netherlands.
| |
Collapse
|
7
|
Snoek L, Jack RE, Schyns PG, Garrod OG, Mittenbühler M, Chen C, Oosterwijk S, Scholte HS. Testing, explaining, and exploring models of facial expressions of emotions. SCIENCE ADVANCES 2023; 9:eabq8421. [PMID: 36763663 PMCID: PMC9916981 DOI: 10.1126/sciadv.abq8421] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 01/09/2023] [Indexed: 06/18/2023]
Abstract
Models are the hallmark of mature scientific inquiry. In psychology, this maturity has been reached in a pervasive question-what models best represent facial expressions of emotion? Several hypotheses propose different combinations of facial movements [action units (AUs)] as best representing the six basic emotions and four conversational signals across cultures. We developed a new framework to formalize such hypotheses as predictive models, compare their ability to predict human emotion categorizations in Western and East Asian cultures, explain the causal role of individual AUs, and explore updated, culture-accented models that improve performance by reducing a prevalent Western bias. Our predictive models also provide a noise ceiling to inform the explanatory power and limitations of different factors (e.g., AUs and individual differences). Thus, our framework provides a new approach to test models of social signals, explain their predictive power, and explore their optimization, with direct implications for theory development.
Collapse
Affiliation(s)
- Lukas Snoek
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | - Rachael E. Jack
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | - Philippe G. Schyns
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | | | - Maximilian Mittenbühler
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
- Department of Computer Science, University of Tübingen, Tübingen, Germany
| | - Chaona Chen
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | - Suzanne Oosterwijk
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
| | - H. Steven Scholte
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
| |
Collapse
|
8
|
Zheng YQ, Farahibozorg SR, Gong W, Rafipoor H, Jbabdi S, Smith S. Accurate predictions of individual differences in task-evoked brain activity from resting-state fMRI using a sparse ensemble learner. Neuroimage 2022; 259:119418. [PMID: 35777635 PMCID: PMC10933828 DOI: 10.1016/j.neuroimage.2022.119418] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Revised: 05/06/2022] [Accepted: 06/27/2022] [Indexed: 01/01/2023] Open
Abstract
Modelling and predicting individual differences in task-fMRI activity can have a wide range of applications from basic to clinical neuroscience. It has been shown that models based on resting-state activity can have high predictive accuracy. Here we propose several improvements to such models. Using a sparse ensemble learner, we show that (i) features extracted using Stochastic Probabilistic Functional Modes (sPROFUMO) outperform the previously proposed dual-regression approach, (ii) that the shape and overall intensity of individualised task activations can be modelled separately and explicitly, (iii) training the model on predicting residual differences in brain activity further boosts individualised predictions. These results hold for both surface-based analyses of the Human Connectome Project data as well as volumetric analyses of UK-biobank data. Overall, our model achieves state of the art prediction accuracy on par with the test-retest reliability of task-fMRI scans, suggesting that it has potential to supplement traditional task localisers.
Collapse
Affiliation(s)
- Ying-Qiu Zheng
- Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, Oxford, UK.
| | - Seyedeh-Rezvan Farahibozorg
- Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, Oxford, UK
| | - Weikang Gong
- Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, Oxford, UK
| | - Hossein Rafipoor
- Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, Oxford, UK
| | - Saad Jbabdi
- Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, Oxford, UK
| | - Stephen Smith
- Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, Oxford, UK
| |
Collapse
|
9
|
Walters J, King M, Bissett PG, Ivry RB, Diedrichsen J, Poldrack RA. Predicting brain activation maps for arbitrary tasks with cognitive encoding models. Neuroimage 2022; 263:119610. [PMID: 36064138 PMCID: PMC9981816 DOI: 10.1016/j.neuroimage.2022.119610] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Revised: 08/12/2022] [Accepted: 09/02/2022] [Indexed: 11/27/2022] Open
Abstract
A deep understanding of the neural architecture of mental function should enable the accurate prediction of a specific pattern of brain activity for any psychological task, based only on the cognitive functions known to be engaged by that task. Encoding models (EMs), which predict neural responses from known features (e.g., stimulus properties), have succeeded in circumscribed domains (e.g., visual neuroscience), but implementing domain-general EMs that predict brain-wide activity for arbitrary tasks has been limited mainly by availability of datasets that 1) sufficiently span a large space of psychological functions, and 2) are sufficiently annotated with such functions to allow robust EM specification. We examine the use of EMs based on a formal specification of psychological function, to predict cortical activation patterns across a broad range of tasks. We utilized the Multi-Domain Task Battery, a dataset in which 24 subjects completed 32 ten-minute fMRI scans, switching tasks every 35 s and engaging in 44 total conditions of diverse psychological manipulations. Conditions were annotated by a group of experts using the Cognitive Atlas ontology to identify putatively engaged functions, and region-wise cognitive EMs (CEMs) were fit, for individual subjects, on neocortical responses. We found that CEMs predicted cortical activation maps of held-out tasks with high accuracy, outperforming a permutation-based null model while approaching the noise ceiling of the data, without being driven solely by either cognitive or perceptual-motor features. Hierarchical clustering on the similarity structure of CEM generalization errors revealed relationships amongst psychological functions. Spatial distributions of feature importances systematically overlapped with large-scale resting-state functional networks (RSNs), supporting the hypothesis of functional specialization within RSNs while grounding their function in an interpretable data-driven manner. Our implementation and validation of CEMs provides a proof of principle for the utility of formal ontologies in cognitive neuroscience and motivates the use of CEMs in the further testing of cognitive theories.
Collapse
Affiliation(s)
- Jonathon Walters
- Department of Psychology, Stanford University, Stanford, CA, USA.
| | - Maedbh King
- Department of Psychology, University of California Berkeley, Berkeley, CA, USA
| | | | - Richard B. Ivry
- Department of Psychology, University of California Berkeley, Berkeley, CA, USA,Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA, USA
| | - Jörn Diedrichsen
- Brain and Mind Institute, Western University, London, Ontario, Canada,Department of Computer Science, Western University, London, Ontario, Canada
| | | |
Collapse
|
10
|
Gaziv G, Beliy R, Granot N, Hoogi A, Strappini F, Golan T, Irani M. Self-supervised Natural Image Reconstruction and Large-scale Semantic Classification from Brain Activity. Neuroimage 2022; 254:119121. [PMID: 35342004 PMCID: PMC9133799 DOI: 10.1016/j.neuroimage.2022.119121] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Revised: 01/19/2022] [Accepted: 03/19/2022] [Indexed: 11/24/2022] Open
Abstract
Reconstructing natural images and decoding their semantic category from fMRI brain recordings is challenging. Acquiring sufficient pairs of images and their corresponding fMRI responses, which span the huge space of natural images, is prohibitive. We present a novel self-supervised approach that goes well beyond the scarce paired data, for achieving both: (i) state-of-the art fMRI-to-image reconstruction, and (ii) first-ever large-scale semantic classification from fMRI responses. By imposing cycle consistency between a pair of deep neural networks (from image-to-fMRI & from fMRI-to-image), we train our image reconstruction network on a large number of "unpaired" natural images (images without fMRI recordings) from many novel semantic categories. This enables to adapt our reconstruction network to a very rich semantic coverage without requiring any explicit semantic supervision. Specifically, we find that combining our self-supervised training with high-level perceptual losses, gives rise to new reconstruction & classification capabilities. In particular, this perceptual training enables to classify well fMRIs of never-before-seen semantic classes, without requiring any class labels during training. This gives rise to: (i) Unprecedented image-reconstruction from fMRI of never-before-seen images (evaluated by image metrics and human testing), and (ii) Large-scale semantic classification of categories that were never-before-seen during network training. Such large-scale (1000-way) semantic classification from fMRI recordings has never been demonstrated before. Finally, we provide evidence for the biological consistency of our learned model.
Collapse
Affiliation(s)
- Guy Gaziv
- Dept. of Computer Science and Applied Math, Weizmann Institute of Science, Rehovot, Israel.
| | - Roman Beliy
- Dept. of Computer Science and Applied Math, Weizmann Institute of Science, Rehovot, Israel
| | - Niv Granot
- Dept. of Computer Science and Applied Math, Weizmann Institute of Science, Rehovot, Israel
| | - Assaf Hoogi
- Dept. of Computer Science and Applied Math, Weizmann Institute of Science, Rehovot, Israel
| | | | - Tal Golan
- Zuckerman Institute, Columbia University, New York, NY USA
| | - Michal Irani
- Dept. of Computer Science and Applied Math, Weizmann Institute of Science, Rehovot, Israel.
| |
Collapse
|
11
|
Merchant JS, Alkire D, Redcay E. Neural similarity between mentalizing and live social interaction during the transition to adolescence. Hum Brain Mapp 2022; 43:4074-4090. [PMID: 35545954 PMCID: PMC9374881 DOI: 10.1002/hbm.25903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Revised: 03/14/2022] [Accepted: 04/19/2022] [Indexed: 12/03/2022] Open
Abstract
Social interactions are essential for human development, yet little neuroimaging research has examined their underlying neurocognitive mechanisms using socially interactive paradigms during childhood and adolescence. Recent neuroimaging research has revealed activity in the mentalizing network when children engage with a live social partner, even when mentalizing is not required. While this finding suggests that social‐interactive contexts may spontaneously engage mentalizing, it is not a direct test of how similarly the brain responds to these two contexts. The current study used representational similarity analysis on data from 8‐ to 14‐year‐olds who made mental and nonmental judgments about an abstract character and a live interaction partner during fMRI. A within‐subject, 2 (Mental/Nonmental) × 2 (Peer/Character) design enabled us to examine response pattern similarity between conditions, and estimate fit to three conceptual models of how the two contexts relate: (1) social interaction and mentalizing about an abstract character are represented similarly; (2) interactive peers and abstract characters are represented differently regardless of the evaluation type; and (3) mental and nonmental states are represented dissimilarly regardless of target. We found that the temporal poles represent mentalizing and peer interactions similarly (Model 1), suggesting a neurocognitive link between the two in these regions. Much of the rest of the social brain exhibits different representations of interactive peers and abstract characters (Model 2). Our findings highlight the importance of studying social‐cognitive processes using interactive approaches, and the utility of pattern‐based analyses for understanding how social‐cognitive processes relate to each other.
Collapse
Affiliation(s)
- Junaid S Merchant
- Neuroscience and Cognitive Science Program, University of Maryland, College Park, Maryland, USA.,Department of Psychology, University of Maryland, College Park, Maryland, USA
| | - Diana Alkire
- Neuroscience and Cognitive Science Program, University of Maryland, College Park, Maryland, USA.,Department of Psychology, University of Maryland, College Park, Maryland, USA
| | - Elizabeth Redcay
- Neuroscience and Cognitive Science Program, University of Maryland, College Park, Maryland, USA.,Department of Psychology, University of Maryland, College Park, Maryland, USA
| |
Collapse
|
12
|
Brain networks are decoupled from external stimuli during internal cognition. Neuroimage 2022; 256:119230. [PMID: 35460919 DOI: 10.1016/j.neuroimage.2022.119230] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2022] [Accepted: 04/19/2022] [Indexed: 11/23/2022] Open
Abstract
Our cognition can be directed to external stimuli or to internal information. While there are many different forms of internal cognition (mind-wandering, recall, imagery etc.), their essential feature is independence from the immediate sensory input, conceptually referred to as perceptual decoupling. Perceptual decoupling is thought to be reflected in brain activity transitioning from a stimulus-processing to internally-processing mode, but a direct investigation of this remains outstanding. Here we present a conceptual and analysis framework that quantifies the extent to which brain networks reflect stimulus processing. We tested this framework by presenting subjects with an audiovisual stimulus and instructing them to either attend to the stimulus (external task) or engage in mental imagery, recall or arithmetic (internal tasks) while measuring the evoked brain activity using functional MRI. We found that stimulus responses were generally attenuated for the internal tasks, though they increased in a subset of tasks and brain networks. However, using our new framework, we showed that brain networks became less reflective of stimulus processing, even in the subset of tasks and brain networks in which stimulus responses increased. These results quantitatively demonstrate that during internal cognition brain networks become decoupled from the external stimuli, opening the door for a fundamental and quantitative understanding of internal cognition.
Collapse
|
13
|
Thornton MA, Wolf S, Reilly BJ, Slingerland EG, Tamir DI. The 3d Mind Model characterizes how people understand mental states across modern and historical cultures. AFFECTIVE SCIENCE 2022; 3:93-104. [PMID: 35938062 PMCID: PMC9355267 DOI: 10.1007/s42761-021-00089-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2021] [Accepted: 10/25/2021] [Indexed: 06/15/2023]
Abstract
Humans rely on social interaction to achieve many important goals. These interactions rely in turn on people's capacity to understand others' mental states: their thoughts and feelings. Do different cultures understand minds in different ways, or do widely shared principles describe how different cultures understand mental states? Extensive data suggest that the mind organizes mental state concepts using the 3d Mind Model, composed of the psychological dimensions: rationality (vs. emotionality), social impact (states which affect others more vs. less), and valence (positive vs. negative states). However, this evidence comes primarily from English-speaking individuals in the United States. Here we investigated mental state representation in 57 contemporary countries, using 163 million English language tweets; in 17 languages, using billions of words of text from internet webpages; and across more than 2000 years of history, using curated texts from four historical societies. We quantified mental state meaning by analyzing the text produced by each culture using word embeddings. We then tested whether the 3d Mind Model could explain which mental states were similar in meaning within each culture. We found that the 3d Mind Model significantly explained mental state meaning in every country, language, and historical society that we examined. These results suggest that rationality, social impact, and valence form a generalizable conceptual backbone for mental state representation.
Collapse
Affiliation(s)
- Mark A. Thornton
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755 USA
| | - Sarah Wolf
- The Jewish Theological Seminary, New York City, NY 10027 USA
| | - Brian J. Reilly
- Department of Modern Languages and Literatures, Fordham University, Bronx, NY 10458 USA
| | - Edward G. Slingerland
- Department of Philosophy, University of British Columbia, Vancouver, BC V6T 1Z2 Canada
| | - Diana I. Tamir
- Department of Psychology, Princeton University, Princeton, NJ 08540 USA
| |
Collapse
|
14
|
Merrick CM, Dixon TC, Breska A, Lin J, Chang EF, King-Stephens D, Laxer KD, Weber PB, Carmena J, Thomas Knight R, Ivry RB. Left hemisphere dominance for bilateral kinematic encoding in the human brain. eLife 2022; 11:e69977. [PMID: 35227374 PMCID: PMC8887902 DOI: 10.7554/elife.69977] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Accepted: 01/19/2022] [Indexed: 11/29/2022] Open
Abstract
Neurophysiological studies in humans and nonhuman primates have revealed movement representations in both the contralateral and ipsilateral hemispheres. Inspired by clinical observations, we ask if this bilateral representation differs for the left and right hemispheres. Electrocorticography was recorded in human participants during an instructed-delay reaching task, with movements produced with either the contralateral or ipsilateral arm. Using a cross-validated kinematic encoding model, we found stronger bilateral encoding in the left hemisphere, an effect that was present during preparation and was amplified during execution. Consistent with this asymmetry, we also observed better across-arm generalization in the left hemisphere, indicating similar neural representations for right and left arm movements. Notably, these left hemisphere electrodes were centered over premotor and parietal regions. The more extensive bilateral encoding in the left hemisphere adds a new perspective to the pervasive neuropsychological finding that the left hemisphere plays a dominant role in praxis.
Collapse
Affiliation(s)
- Christina M Merrick
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
| | - Tanner C Dixon
- UC Berkeley – UCSF Graduate Program in Bioengineering, University of California, BerkeleyBerkeleyUnited States
| | - Assaf Breska
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
| | - Jack Lin
- Department of Neurology, University of California at IrvineIrvineUnited States
| | - Edward F Chang
- Department of Neurological Surgery, University of California San Francisco, San FranciscoSan FranciscoUnited States
| | - David King-Stephens
- Department of Neurology and Neurosurgery, California Pacific Medical CenterSan FranciscoUnited States
| | - Kenneth D Laxer
- Department of Neurology and Neurosurgery, California Pacific Medical CenterSan FranciscoUnited States
| | - Peter B Weber
- Department of Neurology and Neurosurgery, California Pacific Medical CenterSan FranciscoUnited States
| | - Jose Carmena
- UC Berkeley – UCSF Graduate Program in Bioengineering, University of California, BerkeleyBerkeleyUnited States
- Department of Electrical Engineering and Computer Sciences, University of California, BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| | - Robert Thomas Knight
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- UC Berkeley – UCSF Graduate Program in Bioengineering, University of California, BerkeleyBerkeleyUnited States
- Department of Neurological Surgery, University of California San Francisco, San FranciscoSan FranciscoUnited States
- Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| | - Richard B Ivry
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- UC Berkeley – UCSF Graduate Program in Bioengineering, University of California, BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| |
Collapse
|
15
|
Ashton K, Zinszer BD, Cichy RM, Nelson CA, Aslin RN, Bayet L. Time-resolved multivariate pattern analysis of infant EEG data: A practical tutorial. Dev Cogn Neurosci 2022; 54:101094. [PMID: 35248819 PMCID: PMC8897621 DOI: 10.1016/j.dcn.2022.101094] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2021] [Revised: 10/22/2021] [Accepted: 02/24/2022] [Indexed: 01/27/2023] Open
Abstract
Time-resolved multivariate pattern analysis (MVPA), a popular technique for analyzing magneto- and electro-encephalography (M/EEG) neuroimaging data, quantifies the extent and time-course by which neural representations support the discrimination of relevant stimuli dimensions. As EEG is widely used for infant neuroimaging, time-resolved MVPA of infant EEG data is a particularly promising tool for infant cognitive neuroscience. MVPA has recently been applied to common infant imaging methods such as EEG and fNIRS. In this tutorial, we provide and describe code to implement time-resolved, within-subject MVPA with infant EEG data. An example implementation of time-resolved MVPA based on linear SVM classification is described, with accompanying code in Matlab and Python. Results from a test dataset indicated that in both infants and adults this method reliably produced above-chance accuracy for classifying stimuli images. Extensions of the classification analysis are presented including both geometric- and accuracy-based representational similarity analysis, implemented in Python. Common choices of implementation are presented and discussed. As the amount of artifact-free EEG data contributed by each participant is lower in studies of infants than in studies of children and adults, we also explore and discuss the impact of varying participant-level inclusion thresholds on resulting MVPA findings in these datasets.
Collapse
Affiliation(s)
- Kira Ashton
- Department of Neuroscience, American University, Washington, DC 20016, USA; Center for Neuroscience and Behavior, American University, Washington, DC 20016, USA.
| | | | - Radoslaw M Cichy
- Department of Education and Psychology, Freie Universität Berlin, 14195 Berlin, Germany
| | - Charles A Nelson
- Boston Children's Hospital, Boston, MA 02115, USA; Department of Pediatrics, Harvard Medical School, Boston, MA 02115, USA; Graduate School of Education, Harvard, Cambridge, MA 02138, USA
| | - Richard N Aslin
- Haskins Laboratories, 300 George Street, New Haven, CT 06511, USA; Psychological Sciences Department, University of Connecticut, Storrs, CT 06269, USA; Department of Psychology, Yale University, New Haven, CT 06511, USA; Yale Child Study Center, School of Medicine, New Haven, CT 06519, USA
| | - Laurie Bayet
- Department of Neuroscience, American University, Washington, DC 20016, USA; Center for Neuroscience and Behavior, American University, Washington, DC 20016, USA
| |
Collapse
|
16
|
Allen EJ, St-Yves G, Wu Y, Breedlove JL, Prince JS, Dowdle LT, Nau M, Caron B, Pestilli F, Charest I, Hutchinson JB, Naselaris T, Kay K. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nat Neurosci 2022; 25:116-126. [PMID: 34916659 DOI: 10.1038/s41593-021-00962-x] [Citation(s) in RCA: 81] [Impact Index Per Article: 40.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Accepted: 10/12/2021] [Indexed: 11/09/2022]
Abstract
Extensive sampling of neural activity during rich cognitive phenomena is critical for robust understanding of brain function. Here we present the Natural Scenes Dataset (NSD), in which high-resolution functional magnetic resonance imaging responses to tens of thousands of richly annotated natural scenes were measured while participants performed a continuous recognition task. To optimize data quality, we developed and applied novel estimation and denoising techniques. Simple visual inspections of the NSD data reveal clear representational transformations along the ventral visual pathway. Further exemplifying the inferential power of the dataset, we used NSD to build and train deep neural network models that predict brain activity more accurately than state-of-the-art models from computer vision. NSD also includes substantial resting-state and diffusion data, enabling network neuroscience perspectives to constrain and enhance models of perception and memory. Given its unprecedented scale, quality and breadth, NSD opens new avenues of inquiry in cognitive neuroscience and artificial intelligence.
Collapse
Affiliation(s)
- Emily J Allen
- Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota, Minneapolis, MN, USA
- Department of Psychology, University of Minnesota, Minneapolis, MN, USA
| | - Ghislain St-Yves
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, USA
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Yihan Wu
- Graduate Program in Cognitive Science, University of Minnesota, Minneapolis, MN, USA
| | - Jesse L Breedlove
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, USA
- Department of Psychology, University of Minnesota, Minneapolis, MN, USA
| | - Jacob S Prince
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, USA
- Department of Psychology, Harvard University, Cambridge, MA, USA
| | - Logan T Dowdle
- Department of Neuroscience, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Minneapolis, MN, USA
- Department of Neurosurgery, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Minneapolis, MN, USA
| | - Matthias Nau
- National Institute of Mental Health (NIMH), Bethesda MD, USA
| | - Brad Caron
- Program in Neuroscience, Indiana University, Bloomington IN, USA
- Program in Vision Science, Indiana University, Bloomington IN, USA
| | - Franco Pestilli
- Department of Psychology, University of Texas at Austin, Austin, TX, USA
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
- Institute for Neuroscience, University of Texas at Austin, Austin, TX, USA
| | - Ian Charest
- Center for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- cerebrUM, Département de Psychologie, Université de Montréal, Montréal QC, Canada
| | | | - Thomas Naselaris
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, USA
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Kendrick Kay
- Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota, Minneapolis, MN, USA.
| |
Collapse
|
17
|
Cai Y, Hofstetter S, van der Zwaag W, Zuiderbaan W, Dumoulin SO. Individualized cognitive neuroscience needs 7T: Comparing numerosity maps at 3T and 7T MRI. Neuroimage 2021; 237:118184. [PMID: 34023448 DOI: 10.1016/j.neuroimage.2021.118184] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2020] [Revised: 05/14/2021] [Accepted: 05/16/2021] [Indexed: 02/06/2023] Open
Abstract
The field of cognitive neuroscience is weighing evidence about whether to move from the current standard field strength of 3 Tesla (3T) to ultra-high field (UHF) of 7T and above. The present study contributes to the evidence by comparing a computational cognitive neuroscience paradigm at 3T and 7T. The goal was to evaluate the practical effects, i.e. model predictive power, of field strength on a numerosity task using accessible pre-processing and analysis tools. Previously, using 7T functional magnetic resonance imaging and biologically-inspired analyses, i.e. population receptive field modelling, we discovered topographical organization of numerosity-selective neural populations in human parietal cortex. Here we show that these topographic maps are also detectable at 3T. However, averaging of many more functional runs was required at 3T to reliably reconstruct numerosity maps. On average, one 7T run had about four times the model predictive power of one 3T run. We believe that this amount of scanning would have made the initial discovery of the numerosity maps on 3T highly infeasible in practice. Therefore, we suggest that the higher signal-to-noise ratio and signal sensitivity of UHF MRI is necessary to build mechanistic models of the organization and function of our cognitive abilities in individual participants.
Collapse
Affiliation(s)
- Yuxuan Cai
- Spinoza Centre for Neuroimaging, Amsterdam, Netherlands; Experimental and Applied Psychology, VU University Amsterdam, Amsterdam, Netherlands.
| | | | | | | | - Serge O Dumoulin
- Spinoza Centre for Neuroimaging, Amsterdam, Netherlands; Experimental and Applied Psychology, VU University Amsterdam, Amsterdam, Netherlands; Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, Netherlands.
| |
Collapse
|
18
|
Seeliger K, Ambrogioni L, Güçlütürk Y, van den Bulk LM, Güçlü U, van Gerven MAJ. End-to-end neural system identification with neural information flow. PLoS Comput Biol 2021; 17:e1008558. [PMID: 33539366 PMCID: PMC7888598 DOI: 10.1371/journal.pcbi.1008558] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2019] [Revised: 02/17/2021] [Accepted: 11/24/2020] [Indexed: 11/19/2022] Open
Abstract
Neural information flow (NIF) provides a novel approach for system identification in neuroscience. It models the neural computations in multiple brain regions and can be trained end-to-end via stochastic gradient descent from noninvasive data. NIF models represent neural information processing via a network of coupled tensors, each encoding the representation of the sensory input contained in a brain region. The elements of these tensors can be interpreted as cortical columns whose activity encodes the presence of a specific feature in a spatiotemporal location. Each tensor is coupled to the measured data specific to a brain region via low-rank observation models that can be decomposed into the spatial, temporal and feature receptive fields of a localized neuronal population. Both these observation models and the convolutional weights defining the information processing within regions are learned end-to-end by predicting the neural signal during sensory stimulation. We trained a NIF model on the activity of early visual areas using a large-scale fMRI dataset recorded in a single participant. We show that we can recover plausible visual representations and population receptive fields that are consistent with empirical findings.
Collapse
Affiliation(s)
- K. Seeliger
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - L. Ambrogioni
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Y. Güçlütürk
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - L. M. van den Bulk
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - U. Güçlü
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - M. A. J. van Gerven
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| |
Collapse
|
19
|
Zhang H, Hao S, Lee A, Eickhoff SB, Pecheva D, Cai S, Meaney M, Chong YS, Broekman BFP, Fortier MV, Qiu A. Do intrinsic brain functional networks predict working memory from childhood to adulthood? Hum Brain Mapp 2021; 41:4574-4586. [PMID: 33463860 PMCID: PMC7555072 DOI: 10.1002/hbm.25143] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2020] [Revised: 07/07/2020] [Accepted: 07/10/2020] [Indexed: 12/21/2022] Open
Abstract
Working memory (WM) is defined as the ability to maintain a representation online to guide goal‐directed behavior. Its capacity in early childhood predicts academic achievements in late childhood and its deficits are found in various neurodevelopmental disorders. We employed resting‐state fMRI (rs‐fMRI) of 468 participants aged from 4 to 55 years and connectome‐based predictive modeling (CPM) to explore the potential predictive power of intrinsic functional networks to WM in preschoolers, early and late school‐age children, adolescents, and adults. We defined intrinsic functional networks among brain regions identified by activation likelihood estimation (ALE) meta‐analysis on existing WM functional studies (ALE‐based intrinsic functional networks) and intrinsic functional networks generated based on the whole brain (whole‐brain intrinsic functional networks). We employed the CPM on these networks to predict WM in each age group. The CPM using the ALE‐based and whole‐brain intrinsic functional networks predicted WM of individual adults, while the prediction power of the ALE‐based intrinsic functional networks was superior to that of the whole‐brain intrinsic functional networks. Nevertheless, the CPM using the whole‐brain but not the ALE‐based intrinsic functional networks predicted WM in adolescents. And, the CPM using neither the ALE‐based nor whole‐brain networks predicted WM in any of the children groups. Our findings showed the trend of the prediction power of the intrinsic functional networks to cognition in individuals from early childhood to adulthood.
Collapse
Affiliation(s)
- Han Zhang
- Department of Biomedical Engineering and Clinical Imaging Research Center, National University of Singapore, Singapore, Singapore.,School of Computer Engineering and Science, Shanghai University, Shanghai, China
| | - Shuji Hao
- School of Computer Engineering and Science, Shanghai University, Shanghai, China
| | - Annie Lee
- Department of Biomedical Engineering and Clinical Imaging Research Center, National University of Singapore, Singapore, Singapore
| | - Simon B Eickhoff
- Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University, Düsseldorf, Germany.,Institute of Neuroscience and Medicine (INM-7), Research Center Jülich, Jülich, Germany
| | - Diliana Pecheva
- Department of Biomedical Engineering and Clinical Imaging Research Center, National University of Singapore, Singapore, Singapore
| | - Shirong Cai
- Singapore Institute for Clinical Sciences, Singapore, Singapore
| | - Michael Meaney
- Singapore Institute for Clinical Sciences, Singapore, Singapore
| | - Yap-Seng Chong
- Singapore Institute for Clinical Sciences, Singapore, Singapore.,Department of Obstetrics and Gynaecology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Birit F P Broekman
- Department of Psychiatry, Amsterdam UMC, Location VU Medical Centre, VU University, Amsterdam, The Netherlands
| | - Marielle V Fortier
- Department of Diagnostic and Interventional Imaging, KK Women's and Children's Hospital, Singapore, Singapore
| | - Anqi Qiu
- Department of Biomedical Engineering and Clinical Imaging Research Center, National University of Singapore, Singapore, Singapore.,The N.1 Institute for Health, National University of Singapore, Singapore, Singapore
| |
Collapse
|
20
|
Bayet L, Zinszer BD, Reilly E, Cataldo JK, Pruitt Z, Cichy RM, Nelson CA, Aslin RN. Temporal dynamics of visual representations in the infant brain. Dev Cogn Neurosci 2020; 45:100860. [PMID: 32932205 PMCID: PMC7498752 DOI: 10.1016/j.dcn.2020.100860] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2020] [Revised: 08/21/2020] [Accepted: 09/04/2020] [Indexed: 12/12/2022] Open
Abstract
Tools from computational neuroscience have facilitated the investigation of the neural correlates of mental representations. However, access to the representational content of neural activations early in life has remained limited. We asked whether patterns of neural activity elicited by complex visual stimuli (animals, human body) could be decoded from EEG data gathered from 12-15-month-old infants and adult controls. We assessed pairwise classification accuracy at each time-point after stimulus onset, for individual infants and adults. Classification accuracies rose above chance in both groups, within 500 ms. In contrast to adults, neural representations in infants were not linearly separable across visual domains. Representations were similar within, but not across, age groups. These findings suggest a developmental reorganization of visual representations between the second year of life and adulthood and provide a promising proof-of-concept for the feasibility of decoding EEG data within-subject to assess how the infant brain dynamically represents visual objects.
Collapse
Affiliation(s)
- Laurie Bayet
- Department of Psychology, American University, Washington, DC, 20016, USA; Center for Neuroscience and Behavior, American University, Washington, DC, 20016, USA.
| | - Benjamin D Zinszer
- Department of Psychology, Princeton University, Princeton, NJ, 08540, USA; Department of Linguistics and Cognitive Science, University of Delaware, Newark, DE, 19716, USA
| | - Emily Reilly
- Boston Children's Hospital, Boston, MA, 02115, USA
| | | | - Zoe Pruitt
- Department of Brain and Cognitive Science, University of Rochester, Rochester, NY, 14627, USA
| | - Radoslaw M Cichy
- Department of Education and Psychology, Freie Universität Berlin, 14195, Berlin, Germany
| | - Charles A Nelson
- Boston Children's Hospital, Boston, MA, 02115, USA; Department of Pediatrics, Harvard Medical School, Boston, MA, 02115, USA; Graduate School of Education, Harvard, Cambridge, MA, 02138, USA
| | - Richard N Aslin
- Haskins Laboratories, 300 George Street, New Haven, CT, 06511, USA; Psychological Sciences Department, University of Connecticut, Storrs, CT, 06269, USA; Department of Psychology, Yale University, New Haven, CT, 06511, USA; Yale Child Study Center, School of Medicine, New Haven, CT, 06519, USA
| |
Collapse
|
21
|
Lage-Castellanos A, Valente G, Senden M, De Martino F. Investigating the Reliability of Population Receptive Field Size Estimates Using fMRI. Front Neurosci 2020; 14:825. [PMID: 32848580 PMCID: PMC7408704 DOI: 10.3389/fnins.2020.00825] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2020] [Accepted: 07/15/2020] [Indexed: 01/17/2023] Open
Abstract
In functional MRI (fMRI), population receptive field (pRF) models allow a quantitative description of the response as a function of the features of the stimuli that are relevant for each voxel. The most popular pRF model used in fMRI assumes a Gaussian shape in the features space (e.g., the visual field) reducing the description of the voxel’s pRF to the Gaussian mean (the pRF preferred feature) and standard deviation (the pRF size). The estimation of the pRF mean has been proven to be highly reliable. However, the estimate of the pRF size has been shown not to be consistent within and between subjects. While this issue has been noted experimentally, here we use an optimization theory perspective to describe how the inconsistency in estimating the pRF size is linked to an inherent property of the Gaussian pRF model. When fitting such models, the goodness of fit is less sensitive to variations in the pRF size than to variations in the pRF mean. We also show how the same issue can be considered from a bias-variance perspective. We compare different estimation procedures in terms of the reliability of their estimates using simulated and real fMRI data in the visual (using the Human Connectome Project database) and auditory domain. We show that, the reliability of the estimate of the pRF size can be improved considering a linear combination of those pRF models with similar goodness of fit or a permutation based approach. This increase in reliability of the pRF size estimate does not affect the reliability of the estimate of the pRF mean and the prediction accuracy.
Collapse
Affiliation(s)
- Agustin Lage-Castellanos
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands.,Department of NeuroInformatics, Cuban Center for Neuroscience, Havana, Cuba
| | - Giancarlo Valente
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Mario Senden
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Federico De Martino
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands.,Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota, Minneapolis, MN, United States
| |
Collapse
|