1
|
Mochalski LN, Friedrich P, Li X, Kröll J, Eickhoff SB, Weis S. Inter- and intra-subject similarity in network functional connectivity across a full narrative movie. Hum Brain Mapp 2024; 45:e26802. [PMID: 39086203 PMCID: PMC11291869 DOI: 10.1002/hbm.26802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 07/11/2024] [Accepted: 07/17/2024] [Indexed: 08/02/2024] Open
Abstract
Naturalistic paradigms, such as watching movies during functional magnetic resonance imaging, are thought to prompt the emotional and cognitive processes typically elicited in real life situations. Therefore, naturalistic viewing (NV) holds great potential for studying individual differences. Previous studies have primarily focused on using shorter movie clips, geared toward eliciting specific and often isolated emotions, while the potential behind using full narratives depicted in commercial movies as a proxy for real-life experiences has barely been explored. Here, we offer preliminary evidence that a full narrative movie (FNM), that is, a movie covering a complete narrative arc, can capture complex socio-affective dynamics and their links to individual differences. Using the studyforrest dataset, we investigated inter- and intra-subject similarity in network functional connectivity (NFC) of 14 meta-analytically defined networks across a full narrative, audio-visual movie split into eight consecutive movie segments. We characterized the movie segments by valence and arousal portrayed within the sequences, before utilizing a linear mixed model to analyze which factors explain inter- and intra-subject similarity. Our results show that the model best explaining inter-subject similarity comprised network, movie segment, valence and a movie segment by valence interaction. Intra-subject similarity was influenced significantly by the same factors and an additional three-way interaction between movie segment, valence and arousal. Overall, inter- and intra-subject similarity in NFC were sensitive to the ongoing narrative and emotions in the movie. We conclude that FNMs offer complex content and dynamics that might be particularly valuable for studying individual differences. Further characterization of movie features, such as the overarching narratives, that enhance individual differences is needed for advancing the potential of NV research.
Collapse
Affiliation(s)
- Lisa N. Mochalski
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
- Institute of Systems Neuroscience, Medical FacultyHeinrich Heine University DüsseldorfDüsseldorfGermany
| | - Patrick Friedrich
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
| | - Xuan Li
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
| | - Jean‐Philippe Kröll
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
| | - Simon B. Eickhoff
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
- Institute of Systems Neuroscience, Medical FacultyHeinrich Heine University DüsseldorfDüsseldorfGermany
| | - Susanne Weis
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
- Institute of Systems Neuroscience, Medical FacultyHeinrich Heine University DüsseldorfDüsseldorfGermany
| |
Collapse
|
2
|
Thieu MK, Ayzenberg V, Lourenco SF, Kragel PA. Visual looming is a primitive for human emotion. iScience 2024; 27:109886. [PMID: 38799577 PMCID: PMC11126809 DOI: 10.1016/j.isci.2024.109886] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Revised: 03/11/2024] [Accepted: 04/30/2024] [Indexed: 05/29/2024] Open
Abstract
The neural computations for looming detection are strikingly similar across species. In mammals, information about approaching threats is conveyed from the retina to the midbrain superior colliculus, where approach variables are computed to enable defensive behavior. Although neuroscientific theories posit that midbrain representations contribute to emotion through connectivity with distributed brain systems, it remains unknown whether a computational system for looming detection can predict both defensive behavior and phenomenal experience in humans. Here, we show that a shallow convolutional neural network based on the Drosophila visual system predicts defensive blinking to looming objects in infants and superior colliculus responses to optical expansion in adults. Further, the neural network's responses to naturalistic video clips predict self-reported emotion largely by way of subjective arousal. These findings illustrate how a simple neural network architecture optimized for a species-general task relevant for survival explains motor and experiential components of human emotion.
Collapse
Affiliation(s)
| | - Vladislav Ayzenberg
- Emory University, Atlanta, GA, USA
- University of Pennsylvania, Philadelphia, PA, USA
| | | | | |
Collapse
|
3
|
Pinho AL, Richard H, Ponce AF, Eickenberg M, Amadon A, Dohmatob E, Denghien I, Torre JJ, Shankar S, Aggarwal H, Thual A, Chapalain T, Ginisty C, Becuwe-Desmidt S, Roger S, Lecomte Y, Berland V, Laurier L, Joly-Testault V, Médiouni-Cloarec G, Doublé C, Martins B, Varoquaux G, Dehaene S, Hertz-Pannier L, Thirion B. Individual Brain Charting dataset extension, third release for movie watching and retinotopy data. Sci Data 2024; 11:590. [PMID: 38839770 PMCID: PMC11153490 DOI: 10.1038/s41597-024-03390-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Accepted: 05/20/2024] [Indexed: 06/07/2024] Open
Abstract
The Individual Brain Charting (IBC) is a multi-task functional Magnetic Resonance Imaging dataset acquired at high spatial-resolution and dedicated to the cognitive mapping of the human brain. It consists in the deep phenotyping of twelve individuals, covering a broad range of psychological domains suitable for functional-atlasing applications. Here, we present the inclusion of task data from both naturalistic stimuli and trial-based designs, to uncover structures of brain activation. We rely on the Fast Shared Response Model (FastSRM) to provide a data-driven solution for modelling naturalistic stimuli, typically containing many features. We show that data from left-out runs can be reconstructed using FastSRM, enabling the extraction of networks from the visual, auditory and language systems. We also present the topographic organization of the visual system through retinotopy. In total, six new tasks were added to IBC, wherein four trial-based retinotopic tasks contributed with a mapping of the visual field to the cortex. IBC is open access: source plus derivatives imaging data and meta-data are available in public repositories.
Collapse
Affiliation(s)
- Ana Luísa Pinho
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France.
- Department of Computer Science, Western University, London, Ontario, Canada.
- Western Centre for Brain and Mind, Western University, London, Ontario, Canada.
| | - Hugo Richard
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France
- Criteo AI Labs, Paris, France
- FAIRPLAY - IA coopérative: équité, vie privée, incitations, Paris, France
| | | | - Michael Eickenberg
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France
- Flatiron Institute, New York, USA
| | - Alexis Amadon
- Université Paris-Saclay, CEA, CNRS, BAOBAB, NeuroSpin, 91191, Gif-sur-Yvette, France
| | - Elvis Dohmatob
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France
- Meta FAIR, Paris, France
| | - Isabelle Denghien
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, 91191, Gif-sur-Yvette, France
| | | | - Swetha Shankar
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France
| | | | - Alexis Thual
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, 91191, Gif-sur-Yvette, France
- Collège de France, Paris, France
| | | | | | | | | | - Yann Lecomte
- CEA Saclay/DRF/IFJ/NeuroSpin/UNIACT, Paris, France
| | | | | | | | | | | | | | - Gaël Varoquaux
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, 91191, Gif-sur-Yvette, France
- Collège de France, Paris, France
| | - Lucie Hertz-Pannier
- CEA Saclay/DRF/IFJ/NeuroSpin/UNIACT, Paris, France
- UMR 1141, NeuroDiderot, Université de Paris, Paris, France
| | | |
Collapse
|
4
|
Keles U, Dubois J, Le KJM, Tyszka JM, Kahn DA, Reed CM, Chung JM, Mamelak AN, Adolphs R, Rutishauser U. Multimodal single-neuron, intracranial EEG, and fMRI brain responses during movie watching in human patients. Sci Data 2024; 11:214. [PMID: 38365977 PMCID: PMC10873379 DOI: 10.1038/s41597-024-03029-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 01/31/2024] [Indexed: 02/18/2024] Open
Abstract
We present a multimodal dataset of intracranial recordings, fMRI, and eye tracking in 20 participants during movie watching. Recordings consist of single neurons, local field potential, and intracranial EEG activity acquired from depth electrodes targeting the amygdala, hippocampus, and medial frontal cortex implanted for monitoring of epileptic seizures. Participants watched an 8-min long excerpt from the video "Bang! You're Dead" and performed a recognition memory test for movie content. 3 T fMRI activity was recorded prior to surgery in 11 of these participants while performing the same task. This NWB- and BIDS-formatted dataset includes spike times, field potential activity, behavior, eye tracking, electrode locations, demographics, and functional and structural MRI scans. For technical validation, we provide signal quality metrics, assess eye tracking quality, behavior, the tuning of cells and high-frequency broadband power field potentials to familiarity and event boundaries, and show brain-wide inter-subject correlations for fMRI. This dataset will facilitate the investigation of brain activity during movie watching, recognition memory, and the neural basis of the fMRI-BOLD signal.
Collapse
Affiliation(s)
- Umit Keles
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Julien Dubois
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Kevin J M Le
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
| | - J Michael Tyszka
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - David A Kahn
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Chrystal M Reed
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Jeffrey M Chung
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Adam N Mamelak
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Ralph Adolphs
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
| | - Ueli Rutishauser
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Center for Neural Science and Medicine, Department of Biomedical Sciences, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
| |
Collapse
|
5
|
Watson DM, Andrews TJ. Mapping the functional and structural connectivity of the scene network. Hum Brain Mapp 2024; 45:e26628. [PMID: 38376190 PMCID: PMC10878195 DOI: 10.1002/hbm.26628] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Revised: 01/19/2024] [Accepted: 02/05/2024] [Indexed: 02/21/2024] Open
Abstract
The recognition and perception of places has been linked to a network of scene-selective regions in the human brain. While previous studies have focussed on functional connectivity between scene-selective regions themselves, less is known about their connectivity with other cortical and subcortical regions in the brain. Here, we determine the functional and structural connectivity profile of the scene network. We used fMRI to examine functional connectivity between scene regions and across the whole brain during rest and movie-watching. Connectivity within the scene network revealed a bias between posterior and anterior scene regions implicated in perceptual and mnemonic aspects of scene perception respectively. Differences between posterior and anterior scene regions were also evident in the connectivity with cortical and subcortical regions across the brain. For example, the Occipital Place Area (OPA) and posterior Parahippocampal Place Area (PPA) showed greater connectivity with visual and dorsal attention networks, while anterior PPA and Retrosplenial Complex showed preferential connectivity with default mode and frontoparietal control networks and the hippocampus. We further measured the structural connectivity of the scene network using diffusion tractography. This indicated both similarities and differences with the functional connectivity, highlighting biases between posterior and anterior regions, but also between ventral and dorsal scene regions. Finally, we quantified the structural connectivity between the scene network and major white matter tracts throughout the brain. These findings provide a map of the functional and structural connectivity of scene-selective regions to each other and the rest of the brain.
Collapse
Affiliation(s)
- David M. Watson
- Department of Psychology and York Neuroimaging CentreUniversity of YorkYorkUK
| | - Timothy J. Andrews
- Department of Psychology and York Neuroimaging CentreUniversity of YorkYorkUK
| |
Collapse
|
6
|
Thieu MK, Ayzenberg V, Lourenco SF, Kragel PA. Visual looming is a primitive for human emotion. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.08.29.555380. [PMID: 37693448 PMCID: PMC10491236 DOI: 10.1101/2023.08.29.555380] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/12/2023]
Abstract
Looming objects afford threat of collision across the animal kingdom. Defensive responses to looming and neural computations for looming detection are strikingly conserved across species. In mammals, information about rapidly approaching threats is conveyed from the retina to the midbrain superior colliculus, where variables that indicate the position and velocity of approach are computed to enable defensive behavior. Although neuroscientific theories posit that midbrain representations contribute to emotion through connectivity with distributed brain systems, it remains unknown whether a computational system for looming detection can predict both defensive behavior and phenomenal experience in humans. Here, we show that a shallow convolutional neural network based on the Drosophila visual system predicts defensive blinking to looming objects in infants and superior colliculus responses to optical expansion in adults. Further, the responses of the convolutional network to a broad array of naturalistic video clips predict self-reported emotion largely on the basis of subjective arousal. Our findings illustrate how motor and experiential components of human emotion relate to species-general systems for survival in unpredictable environments.
Collapse
|
7
|
Jiahui G, Feilong M, Nastase SA, Haxby JV, Gobbini MI. Cross-movie prediction of individualized functional topography. eLife 2023; 12:e86037. [PMID: 37994909 PMCID: PMC10666932 DOI: 10.7554/elife.86037] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 11/09/2023] [Indexed: 11/24/2023] Open
Abstract
Participant-specific, functionally defined brain areas are usually mapped with functional localizers and estimated by making contrasts between responses to single categories of input. Naturalistic stimuli engage multiple brain systems in parallel, provide more ecologically plausible estimates of real-world statistics, and are friendly to special populations. The current study shows that cortical functional topographies in individual participants can be estimated with high fidelity from naturalistic stimuli. Importantly, we demonstrate that robust, individualized estimates can be obtained even when participants watched different movies, were scanned with different parameters/scanners, and were sampled from different institutes across the world. Our results create a foundation for future studies that allow researchers to estimate a broad range of functional topographies based on naturalistic movies and a normative database, making it possible to integrate high-level cognitive functions across datasets from laboratories worldwide.
Collapse
Affiliation(s)
- Guo Jiahui
- Center for Cognitive Neuroscience, Dartmouth CollegeHanoverUnited States
| | - Ma Feilong
- Center for Cognitive Neuroscience, Dartmouth CollegeHanoverUnited States
| | - Samuel A Nastase
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - James V Haxby
- Center for Cognitive Neuroscience, Dartmouth CollegeHanoverUnited States
| | - M Ida Gobbini
- Department of Medical and Surgical Sciences (DIMEC), University of BolognaBolognaItaly
- IRCCS, Istituto delle Scienze Neurologiche di BolognaBolognaItaly
| |
Collapse
|
8
|
Lv X, Funahashi S, Li C, Wu J. Variational relevance evaluation of individual fMRI data enables deconstruction of task-dependent neural dynamics. Commun Biol 2023; 6:491. [PMID: 37147471 PMCID: PMC10163018 DOI: 10.1038/s42003-023-04804-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2023] [Accepted: 04/04/2023] [Indexed: 05/07/2023] Open
Abstract
In neuroimaging research, univariate analysis has always been used to localize "representations" at the microscale, whereas network approaches have been applied to characterize transregional "operations". How are representations and operations linked through dynamic interactions? We developed the variational relevance evaluation (VRE) method to analyze individual task fMRI data, which selects informative voxels during model training to localize the "representation", and quantifies the dynamic contributions of single voxels across the whole-brain to different cognitive functions to characterize the "operation". Using 15 individual fMRI data files for higher visual area localizers, we evaluated the characterization of selected voxel positions of VRE and revealed different object-selective regions functioning in similar dynamics. Using another 15 individual fMRI data files for memory retrieval after offline learning, we found similar task-related regions working in different neural dynamics for tasks with diverse familiarities. VRE demonstrates a promising horizon in individual fMRI research.
Collapse
Affiliation(s)
- Xiaoyu Lv
- School of Mechatronical Engineering, Beijing Institute of Technology, Beijing, China
| | - Shintaro Funahashi
- Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology, Beijing, China
| | - Chunlin Li
- School of Biomedical Engineering, Capital Medical University, Beijing, China.
- Beijing Key Laboratory of Fundamental Research on Biomechanics in Clinical Application, Capital Medical University, Beijing, China.
| | - Jinglong Wu
- School of Medical Technology, Beijing Institute of Technology, Beijing, China.
- Researh Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Science, Shenzhen, Guangdong, China.
| |
Collapse
|
9
|
Fang M, Aglinskas A, Li Y, Anzellotti S. Angular Gyrus Responses Show Joint Statistical Dependence with Brain Regions Selective for Different Categories. J Neurosci 2023; 43:2756-2766. [PMID: 36894316 PMCID: PMC10089240 DOI: 10.1523/jneurosci.1283-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2022] [Revised: 02/13/2023] [Accepted: 02/20/2023] [Indexed: 03/11/2023] Open
Abstract
Category selectivity is a fundamental principle of organization of perceptual brain regions. Human occipitotemporal cortex is subdivided into areas that respond preferentially to faces, bodies, artifacts, and scenes. However, observers need to combine information about objects from different categories to form a coherent understanding of the world. How is this multicategory information encoded in the brain? Studying the multivariate interactions between brain regions of male and female human subjects with fMRI and artificial neural networks, we found that the angular gyrus shows joint statistical dependence with multiple category-selective regions. Adjacent regions show effects for the combination of scenes and each other category, suggesting that scenes provide a context to combine information about the world. Additional analyses revealed a cortical map of areas that encode information across different subsets of categories, indicating that multicategory information is not encoded in a single centralized location, but in multiple distinct brain regions.SIGNIFICANCE STATEMENT Many cognitive tasks require combining information about entities from different categories. However, visual information about different categorical objects is processed by separate, specialized brain regions. How is the joint representation from multiple category-selective regions implemented in the brain? Using fMRI movie data and state-of-the-art multivariate statistical dependence based on artificial neural networks, we identified the angular gyrus encoding responses across face-, body-, artifact-, and scene-selective regions. Further, we showed a cortical map of areas that encode information across different subsets of categories. These findings suggest that multicategory information is not encoded in a single centralized location, but at multiple cortical sites which might contribute to distinct cognitive functions, offering insights to understand integration in a variety of domains.
Collapse
Affiliation(s)
- Mengting Fang
- Department of Psychology and Neuroscience, Boston College, Boston, Massachusetts 02467
- Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104
| | - Aidas Aglinskas
- Department of Psychology and Neuroscience, Boston College, Boston, Massachusetts 02467
| | - Yichen Li
- Department of Psychology, Harvard University, Cambridge, Massachusetts 02138
| | - Stefano Anzellotti
- Department of Psychology and Neuroscience, Boston College, Boston, Massachusetts 02467
| |
Collapse
|
10
|
Busch EL, Huang J, Benz A, Wallenstein T, Lajoie G, Wolf G, Krishnaswamy S, Turk-Browne NB. Multi-view manifold learning of human brain-state trajectories. NATURE COMPUTATIONAL SCIENCE 2023; 3:240-253. [PMID: 37693659 PMCID: PMC10487346 DOI: 10.1038/s43588-023-00419-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Accepted: 02/14/2023] [Indexed: 09/12/2023]
Abstract
The complexity of the human brain gives the illusion that brain activity is intrinsically high-dimensional. Nonlinear dimensionality-reduction methods such as uniform manifold approximation and t-distributed stochastic neighbor embedding have been used for high-throughput biomedical data. However, they have not been used extensively for brain activity data such as those from functional magnetic resonance imaging (fMRI), primarily due to their inability to maintain dynamic structure. Here we introduce a nonlinear manifold learning method for time-series data-including those from fMRI-called temporal potential of heat-diffusion for affinity-based transition embedding (T-PHATE). In addition to recovering a low-dimensional intrinsic manifold geometry from time-series data, T-PHATE exploits the data's autocorrelative structure to faithfully denoise and unveil dynamic trajectories. We empirically validate T-PHATE on three fMRI datasets, showing that it greatly improves data visualization, classification, and segmentation of the data relative to several other state-of-the-art dimensionality-reduction benchmarks. These improvements suggest many potential applications of T-PHATE to other high-dimensional datasets of temporally diffuse processes.
Collapse
Affiliation(s)
- Erica L. Busch
- Department of Psychology, Yale University, New Haven, CT, USA
| | - Jessie Huang
- Department of Computer Science, Yale University, New Haven, CT, USA
| | - Andrew Benz
- Department of Mathematics, Yale University, New Haven, CT, USA
| | - Tom Wallenstein
- Department of Computer Science, Yale University, New Haven, CT, USA
| | - Guillaume Lajoie
- Department of Mathematics and Statistics, Université de Montréal, Montreal, Canada
- Mila—Quebec Artificial Intelligence Institute, Montreal, Canada
| | - Guy Wolf
- Department of Mathematics and Statistics, Université de Montréal, Montreal, Canada
- Mila—Quebec Artificial Intelligence Institute, Montreal, Canada
| | - Smita Krishnaswamy
- Department of Computer Science, Yale University, New Haven, CT, USA
- Department of Genetics, Yale University, New Haven, CT, USA
- Program in Applied Mathematics, Yale University, New Haven, CT, USA
- Wu Tsai Institute, Yale University, New Haven, CT, USA
- These authors contributed equally: Smita Krishnaswamy and Nicholas B. Turk-Browne
| | - Nicholas B. Turk-Browne
- Department of Psychology, Yale University, New Haven, CT, USA
- Wu Tsai Institute, Yale University, New Haven, CT, USA
- These authors contributed equally: Smita Krishnaswamy and Nicholas B. Turk-Browne
| |
Collapse
|
11
|
Poskanzer C, Anzellotti S. Functional coordinates: Modeling interactions between brain regions as points in a function space. Netw Neurosci 2022; 6:1296-1315. [PMID: 38800459 PMCID: PMC11117108 DOI: 10.1162/netn_a_00264] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 06/22/2022] [Indexed: 05/29/2024] Open
Abstract
Here, we propose a novel technique to investigate nonlinear interactions between brain regions that captures both the strength and type of the functional relationship. Inspired by the field of functional analysis, we propose that the relationship between activity in separate brain areas can be viewed as a point in function space, identified by coordinates along an infinite set of basis functions. Using Hermite polynomials as bases, we estimate a subset of these values that serve as "functional coordinates," characterizing the interaction between BOLD activity across brain areas. We provide a proof of the convergence of the estimates in the limit, and we validate the method with simulations in which the ground truth is known, additionally showing that functional coordinates detect statistical dependence even when correlations ("functional connectivity") approach zero. We then use functional coordinates to examine neural interactions with a chosen seed region: the fusiform face area (FFA). Using k-means clustering across each voxel's functional coordinates, we illustrate that adding nonlinear basis functions allows for the discrimination of interregional interactions that are otherwise grouped together when using only linear dependence. Finally, we show that regions in V5 and medial occipital and temporal lobes exhibit significant nonlinear interactions with the FFA.
Collapse
Affiliation(s)
- Craig Poskanzer
- Department of Psychology, Columbia University, New York City, NY, USA
- Department of Psychology and Neuroscience, Boston College, Boston, MA, USA
| | - Stefano Anzellotti
- Department of Psychology and Neuroscience, Boston College, Boston, MA, USA
| |
Collapse
|
12
|
An evaluation of how connectopic mapping reveals visual field maps in V1. Sci Rep 2022; 12:16249. [PMID: 36171242 PMCID: PMC9519585 DOI: 10.1038/s41598-022-20322-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2022] [Accepted: 09/09/2022] [Indexed: 11/25/2022] Open
Abstract
Abstract Functional gradients, in which response properties change gradually across the cortical surface, have been proposed as a key organising principle of the brain. However, the presence of these gradients remains undetermined in many brain regions. Resting-state neuroimaging studies have suggested these gradients can be reconstructed from patterns of functional connectivity. Here we investigate the accuracy of these reconstructions and establish whether it is connectivity or the functional properties within a region that determine these “connectopic maps”. Different manifold learning techniques were used to recover visual field maps while participants were at rest or engaged in natural viewing. We benchmarked these reconstructions against maps measured by traditional visual field mapping. We report an initial exploratory experiment of a publicly available naturalistic imaging dataset, followed by a preregistered replication using larger resting-state and naturalistic imaging datasets from the Human Connectome Project. Connectopic mapping accurately predicted visual field maps in primary visual cortex, with better predictions for eccentricity than polar angle maps. Non-linear manifold learning methods outperformed simpler linear embeddings. We also found more accurate predictions during natural viewing compared to resting-state. Varying the source of the connectivity estimates had minimal impact on the connectopic maps, suggesting the key factor is the functional topography within a brain region. The application of these standardised methods for connectopic mapping will allow the discovery of functional gradients across the brain. Protocol registration The stage 1 protocol for this Registered Report was accepted in
principle on 19 April 2022. The protocol, as accepted by the journal, can be found at 10.6084/m9.figshare.19771717.
Collapse
|
13
|
Poskanzer C, Fang M, Aglinskas A, Anzellotti S. Controlling for Spurious Nonlinear Dependence in Connectivity Analyses. Neuroinformatics 2022; 20:599-611. [PMID: 34519963 DOI: 10.1007/s12021-021-09540-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/03/2021] [Indexed: 12/31/2022]
Abstract
Recent analysis methods can capture nonlinear interactions between brain regions. However, noise sources might induce spurious nonlinear relationships between the responses in different regions. Previous research has demonstrated that traditional denoising techniques effectively remove noise-induced linear relationships between brain areas, but it is unknown whether these techniques can remove spurious nonlinear relationships. To address this question, we analyzed fMRI responses while participants watched the film Forrest Gump. We tested whether nonlinear Multivariate Pattern Dependence Networks (MVPN) outperform linear MVPN in non-denoised data, and whether this difference is reduced after CompCor denoising. Whereas nonlinear MVPN outperformed linear MVPN in the non-denoised data, denoising removed these nonlinear interactions. We replicated our results using different neural network architectures as the bases of MVPN, different activation functions (ReLU and sigmoid), different dimensionality reduction techniques for CompCor (PCA and ICA), and multiple datasets, demonstrating that CompCor's ability to remove nonlinear interactions is robust across these analysis choices and across different groups of participants. Finally, we asked whether information contributing to the removal of nonlinear interactions is localized to specific anatomical regions of no interest or to specific principal components. We denoised the data 8 separate times by regressing out 5 principal components extracted from combined white matter (WM) and cerebrospinal fluid (CSF), each of the 5 components separately, 5 components extracted from WM only, and 5 components extracted solely from CSF. In all cases, denoising was sufficient to remove the observed nonlinear interactions.
Collapse
Affiliation(s)
- Craig Poskanzer
- Department of Psychology and Neuroscience, Boston College, Boston, 02467, MA, USA.
| | - Mengting Fang
- Department of Psychology and Neuroscience, Boston College, Boston, 02467, MA, USA
| | - Aidas Aglinskas
- Department of Psychology and Neuroscience, Boston College, Boston, 02467, MA, USA
| | - Stefano Anzellotti
- Department of Psychology and Neuroscience, Boston College, Boston, 02467, MA, USA
| |
Collapse
|
14
|
Fang M, Poskanzer C, Anzellotti S. PyMVPD: A Toolbox for Multivariate Pattern Dependence. Front Neuroinform 2022; 16:835772. [PMID: 35811995 PMCID: PMC9262406 DOI: 10.3389/fninf.2022.835772] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Accepted: 05/17/2022] [Indexed: 11/13/2022] Open
Abstract
Cognitive tasks engage multiple brain regions. Studying how these regions interact is key to understand the neural bases of cognition. Standard approaches to model the interactions between brain regions rely on univariate statistical dependence. However, newly developed methods can capture multivariate dependence. Multivariate pattern dependence (MVPD) is a powerful and flexible approach that trains and tests multivariate models of the interactions between brain regions using independent data. In this article, we introduce PyMVPD: an open source toolbox for multivariate pattern dependence. The toolbox includes linear regression models and artificial neural network models of the interactions between regions. It is designed to be easily customizable. We demonstrate example applications of PyMVPD using well-studied seed regions such as the fusiform face area (FFA) and the parahippocampal place area (PPA). Next, we compare the performance of different model architectures. Overall, artificial neural networks outperform linear regression. Importantly, the best performing architecture is region-dependent: MVPD subdivides cortex in distinct, contiguous regions whose interaction with FFA and PPA is best captured by different models.
Collapse
|
15
|
Häusler CO, Eickhoff SB, Hanke M. Processing of visual and non-visual naturalistic spatial information in the "parahippocampal place area". Sci Data 2022; 9:147. [PMID: 35365659 PMCID: PMC8975992 DOI: 10.1038/s41597-022-01250-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Accepted: 02/14/2022] [Indexed: 11/09/2022] Open
Abstract
The "parahippocampal place area" (PPA) in the human ventral visual stream exhibits increased hemodynamic activity correlated with the perception of landscape photos compared to faces or objects. Here, we investigate the perception of scene-related, spatial information embedded in two naturalistic stimuli. The same 14 participants were watching a Hollywood movie and listening to its audio-description as part of the open-data resource studyforrest.org. We model hemodynamic activity based on annotations of selected stimulus features, and compare results to a block-design visual localizer. On a group level, increased activation correlating with visual spatial information occurring in the movie is overlapping with a traditionally localized PPA. Activation correlating with semantic spatial information occurring in the audio-description is more restricted to the anterior PPA. On an individual level, we find significant bilateral activity in the PPA of nine individuals and unilateral activity in one individual. Results suggest that activation in the PPA generalizes to spatial information embedded in a movie and an auditory narrative, and may call for considering a functional subdivision of the PPA.
Collapse
Affiliation(s)
- Christian O Häusler
- Psychoinformatics Lab, Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany. .,Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany.
| | - Simon B Eickhoff
- Psychoinformatics Lab, Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.,Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany
| | - Michael Hanke
- Psychoinformatics Lab, Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.,Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany
| |
Collapse
|
16
|
Tu Y, Li X, Lu ZL, Wang Y. Diffeomorphic registration for retinotopic maps of multiple visual regions. Brain Struct Funct 2022; 227:1507-1522. [PMID: 35325293 DOI: 10.1007/s00429-022-02480-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2021] [Accepted: 02/23/2022] [Indexed: 11/29/2022]
Abstract
Retinotopic map, the mapping between visual inputs on the retina and neuronal responses on the cortical surface, is one of the central topics in vision science. Typically, human retinotopic maps are constructed by analyzing functional magnetic resonance responses to designed visual stimuli on the cortical surface. Although it is widely used in visual neuroscience, retinotopic maps are limited by the signal-to-noise ratio and spatial resolution of fMRI. One promising approach to improve the quality of retinotopic maps is to register individual subject's retinotopic maps to a retinotopic template. However, none of the existing retinotopic registration methods has explicitly quantified the diffeomorphic condition, that is, retinotopic maps shall be aligned by stretching/compressing without tearing up the cortical surface. Here, we developed Diffeomorphic Registration for Retinotopic Maps (DRRM) to simultaneously align retinotopic maps in multiple visual regions under the diffeomorphic condition. Specifically, we used the Beltrami coefficient to model the diffeomorphic condition and performed surface registration based on retinotopic coordinates. The overall framework preserves the topological condition defined in the template. We further developed a unique evaluation protocol and compared the performance of the new method with several existing registration methods on both synthetic and real datasets. The results showed that DRRM is superior to the existing methods in achieving diffeomorphic registration in synthetic and empirical data from 3T and 7T MRI systems. DRRM may improve the interpretation of low-quality retinotopic maps and facilitate applications of retinotopic maps in clinical settings.
Collapse
Affiliation(s)
- Yanshuai Tu
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA
| | - Xin Li
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA
| | - Zhong-Lin Lu
- Division of Arts and Sciences, NYU Shanghai, Shanghai, China.,Center for Neural Science and Department of Psychology, New York University, New York, USA.,NYU-ECNU Institute of Brain and Cognitive Science, NYU Shanghai, Shanghai, China
| | - Yalin Wang
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA.
| |
Collapse
|
17
|
Berezutskaya J, Vansteensel MJ, Aarnoutse EJ, Freudenburg ZV, Piantoni G, Branco MP, Ramsey NF. Open multimodal iEEG-fMRI dataset from naturalistic stimulation with a short audiovisual film. Sci Data 2022; 9:91. [PMID: 35314718 PMCID: PMC8938409 DOI: 10.1038/s41597-022-01173-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Accepted: 01/24/2022] [Indexed: 12/19/2022] Open
Abstract
Intracranial human recordings are a valuable and rare resource of information about the brain. Making such data publicly available not only helps tackle reproducibility issues in science, it helps make more use of these valuable data. This is especially true for data collected using naturalistic tasks. Here, we describe a dataset collected from a large group of human subjects while they watched a short audiovisual film. The dataset has several unique features. First, it includes a large amount of intracranial electroencephalography (iEEG) data (51 participants, age range of 5-55 years, who all performed the same task). Second, it includes functional magnetic resonance imaging (fMRI) recordings (30 participants, age range of 7-47) during the same task. Eighteen participants performed both iEEG and fMRI versions of the task, non-simultaneously. Third, the data were acquired using a rich audiovisual stimulus, for which we provide detailed speech and video annotations. This dataset can be used to study neural mechanisms of multimodal perception and language comprehension, and similarity of neural signals across brain recording modalities.
Collapse
Affiliation(s)
- Julia Berezutskaya
- Brain Center, Department of Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, the Netherlands.
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands.
| | - Mariska J Vansteensel
- Brain Center, Department of Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Erik J Aarnoutse
- Brain Center, Department of Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Zachary V Freudenburg
- Brain Center, Department of Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Giovanni Piantoni
- Brain Center, Department of Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Mariana P Branco
- Brain Center, Department of Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Nick F Ramsey
- Brain Center, Department of Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, the Netherlands
| |
Collapse
|
18
|
Big Data in Cognitive Neuroscience: Opportunities and Challenges. BIG DATA ANALYTICS 2022. [DOI: 10.1007/978-3-031-24094-2_2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023] Open
|
19
|
Abstract
We describe a collection of T1-, diffusion- and functional T2*-weighted magnetic resonance imaging data from human individuals with albinism and achiasma. This repository can be used as a test-bed to develop and validate tractography methods like diffusion-signal modeling and fiber tracking as well as to investigate the properties of the human visual system in individuals with congenital abnormalities. The MRI data is provided together with tools and files allowing for its preprocessing and analysis, along with the data derivatives such as manually curated masks and regions of interest for performing tractography.
Collapse
|
20
|
|
21
|
Isherwood SJS, Bazin PL, Alkemade A, Forstmann BU. Quantity and quality: Normative open-access neuroimaging databases. PLoS One 2021; 16:e0248341. [PMID: 33705468 PMCID: PMC7951909 DOI: 10.1371/journal.pone.0248341] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Accepted: 02/24/2021] [Indexed: 11/19/2022] Open
Abstract
The focus of this article is to compare twenty normative and open-access neuroimaging databases based on quantitative measures of image quality, namely, signal-to-noise (SNR) and contrast-to-noise ratios (CNR). We further the analysis through discussing to what extent these databases can be used for the visualization of deeper regions of the brain, such as the subcortex, as well as provide an overview of the types of inferences that can be drawn. A quantitative comparison of contrasts including T1-weighted (T1w) and T2-weighted (T2w) images are summarized, providing evidence for the benefit of ultra-high field MRI. Our analysis suggests a decline in SNR in the caudate nuclei with increasing age, in T1w, T2w, qT1 and qT2* contrasts, potentially indicative of complex structural age-dependent changes. A similar decline was found in the corpus callosum of the T1w, qT1 and qT2* contrasts, though this relationship is not as extensive as within the caudate nuclei. These declines were accompanied by a declining CNR over age in all image contrasts. A positive correlation was found between scan time and the estimated SNR as well as a negative correlation between scan time and spatial resolution. Image quality as well as the number and types of contrasts acquired by these databases are important factors to take into account when selecting structural data for reuse. This article highlights the opportunities and pitfalls associated with sampling existing databases, and provides a quantitative backing for their usage.
Collapse
Affiliation(s)
- Scott Jie Shen Isherwood
- Integrative Model-Based Cognitive Neuroscience Research Unit, University of Amsterdam, Amsterdam, The Netherlands
| | - Pierre-Louis Bazin
- Integrative Model-Based Cognitive Neuroscience Research Unit, University of Amsterdam, Amsterdam, The Netherlands
- Department of Neurophysics, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Anneke Alkemade
- Integrative Model-Based Cognitive Neuroscience Research Unit, University of Amsterdam, Amsterdam, The Netherlands
| | - Birte Uta Forstmann
- Integrative Model-Based Cognitive Neuroscience Research Unit, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
22
|
Pinho AL, Amadon A, Fabre M, Dohmatob E, Denghien I, Torre JJ, Ginisty C, Becuwe-Desmidt S, Roger S, Laurier L, Joly-Testault V, Médiouni-Cloarec G, Doublé C, Martins B, Pinel P, Eger E, Varoquaux G, Pallier C, Dehaene S, Hertz-Pannier L, Thirion B. Subject-specific segregation of functional territories based on deep phenotyping. Hum Brain Mapp 2020; 42:841-870. [PMID: 33368868 PMCID: PMC7856658 DOI: 10.1002/hbm.25189] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2020] [Revised: 07/11/2020] [Accepted: 08/04/2020] [Indexed: 11/08/2022] Open
Abstract
Functional magnetic resonance imaging (fMRI) has opened the possibility to investigate how brain activity is modulated by behavior. Most studies so far are bound to one single task, in which functional responses to a handful of contrasts are analyzed and reported as a group average brain map. Contrariwise, recent data-collection efforts have started to target a systematic spatial representation of multiple mental functions. In this paper, we leverage the Individual Brain Charting (IBC) dataset-a high-resolution task-fMRI dataset acquired in a fixed environment-in order to study the feasibility of individual mapping. First, we verify that the IBC brain maps reproduce those obtained from previous, large-scale datasets using the same tasks. Second, we confirm that the elementary spatial components, inferred across all tasks, are consistently mapped within and, to a lesser extent, across participants. Third, we demonstrate the relevance of the topographic information of the individual contrast maps, showing that contrasts from one task can be predicted by contrasts from other tasks. At last, we showcase the benefit of contrast accumulation for the fine functional characterization of brain regions within a prespecified network. To this end, we analyze the cognitive profile of functional territories pertaining to the language network and prove that these profiles generalize across participants.
Collapse
Affiliation(s)
| | - Alexis Amadon
- Université Paris-Saclay, CEA, CNRS, BAOBAB, NeuroSpin, Gif-sur-Yvette, France
| | - Murielle Fabre
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | - Elvis Dohmatob
- Université Paris-Saclay, Inria, CEA, Palaiseau, France.,Criteo AI Lab, Paris, France
| | - Isabelle Denghien
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | | | | | | | | | | | | | | | | | | | - Philippe Pinel
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | - Evelyn Eger
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | | | - Christophe Pallier
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France.,Collège de France, Paris, France
| | - Lucie Hertz-Pannier
- CEA Saclay/DRF/IFJ/NeuroSpin/UNIACT, Paris, France.,UMR 1141, NeuroDiderot, Université de Paris, Paris, France
| | | |
Collapse
|
23
|
Kumar S, Ellis CT, O'Connell TP, Chun MM, Turk-Browne NB. Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain. PLoS Comput Biol 2020; 16:e1008457. [PMID: 33270655 PMCID: PMC7738169 DOI: 10.1371/journal.pcbi.1008457] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2020] [Revised: 12/15/2020] [Accepted: 10/21/2020] [Indexed: 11/18/2022] Open
Abstract
The extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model, as well as object representations, are more widely distributed across the brain than previously acknowledged and that functional searchlight can improve model-based similarity and decoding accuracy. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation.
Collapse
Affiliation(s)
- Sreejan Kumar
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, United States of America.,Department of Psychology, Yale University, New Haven, Connecticut, United States of America
| | - Cameron T Ellis
- Department of Psychology, Yale University, New Haven, Connecticut, United States of America
| | - Thomas P O'Connell
- Department of Brain and Cognitive Sciences, MIT, Cambridge, Massachusetts, United States of America
| | - Marvin M Chun
- Department of Psychology, Yale University, New Haven, Connecticut, United States of America
| | - Nicholas B Turk-Browne
- Department of Psychology, Yale University, New Haven, Connecticut, United States of America
| |
Collapse
|
24
|
Visconti di Oleggio Castello M, Chauhan V, Jiahui G, Gobbini MI. An fMRI dataset in response to "The Grand Budapest Hotel", a socially-rich, naturalistic movie. Sci Data 2020; 7:383. [PMID: 33177526 PMCID: PMC7658985 DOI: 10.1038/s41597-020-00735-4] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Accepted: 10/27/2020] [Indexed: 11/18/2022] Open
Abstract
Naturalistic stimuli evoke strong, consistent, and information-rich patterns of brain activity, and engage large extents of the human brain. They allow researchers to compare highly similar brain responses across subjects, and to study how complex representations are encoded in brain activity. Here, we describe and share a dataset where 25 subjects watched part of the feature film "The Grand Budapest Hotel" by Wes Anderson. The movie has a large cast with many famous actors. Throughout the story, the camera shots highlight faces and expressions, which are fundamental to understand the complex narrative of the movie. This movie was chosen to sample brain activity specifically related to social interactions and face processing. This dataset provides researchers with fMRI data that can be used to explore social cognitive processes and face processing, adding to the existing neuroimaging datasets that sample brain activity with naturalistic movies.
Collapse
Affiliation(s)
| | - Vassiki Chauhan
- Center for Cognitive Neuroscience, Dartmouth College, Hanover, USA
| | - Guo Jiahui
- Center for Cognitive Neuroscience, Dartmouth College, Hanover, USA
| | - M Ida Gobbini
- Cognitive Science Program, Dartmouth College, Hanover, USA.
- Dipartimento di Medicina Specialistica, Diagnostica e Sperimentale, University of Bologna, Bologna, Italy.
| |
Collapse
|
25
|
Pinho AL, Amadon A, Gauthier B, Clairis N, Knops A, Genon S, Dohmatob E, Torre JJ, Ginisty C, Becuwe-Desmidt S, Roger S, Lecomte Y, Berland V, Laurier L, Joly-Testault V, Médiouni-Cloarec G, Doublé C, Martins B, Salmon E, Piazza M, Melcher D, Pessiglione M, van Wassenhove V, Eger E, Varoquaux G, Dehaene S, Hertz-Pannier L, Thirion B. Individual Brain Charting dataset extension, second release of high-resolution fMRI data for cognitive mapping. Sci Data 2020; 7:353. [PMID: 33067452 PMCID: PMC7567863 DOI: 10.1038/s41597-020-00670-4] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 09/01/2020] [Indexed: 11/09/2022] Open
Abstract
We present an extension of the Individual Brain Charting dataset -a high spatial-resolution, multi-task, functional Magnetic Resonance Imaging dataset, intended to support the investigation on the functional principles governing cognition in the human brain. The concomitant data acquisition from the same 12 participants, in the same environment, allows to obtain in the long run finer cognitive topographies, free from inter-subject and inter-site variability. This second release provides more data from psychological domains present in the first release, and also yields data featuring new ones. It includes tasks on e.g. mental time travel, reward, theory-of-mind, pain, numerosity, self-reference effect and speech recognition. In total, 13 tasks with 86 contrasts were added to the dataset and 63 new components were included in the cognitive description of the ensuing contrasts. As the dataset becomes larger, the collection of the corresponding topographies becomes more comprehensive, leading to better brain-atlasing frameworks. This dataset is an open-access facility; raw data and derivatives are publicly available in neuroimaging repositories.
Collapse
Affiliation(s)
- Ana Luísa Pinho
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France.
| | - Alexis Amadon
- Université Paris-Saclay, CEA, CNRS, BAOBAB, NeuroSpin, 91191, Gif-sur-Yvette, France
| | - Baptiste Gauthier
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, 91191, Gif/Yvette, France
- Laboratory of Cognitive Neuroscience, Brain Mind Institute, School of Life Sciences and Center for Neuroprosthetics, Swiss Federal Institute of Technology (EPFL), Campus Biotech, Geneva, Switzerland
| | - Nicolas Clairis
- Motivation, Brain and Behavior (MBB) team, Institut du Cerveau (ICM), Inserm UMRS 1127, CNS UMR 7225, Sorbonne Université, Paris, France
| | - André Knops
- Center for Mind/Brain Sciences, University of Trento, I-38068, Rovereto, Italy
- LaPsyDÉ, UMR CNRS 8240, Université de Paris, Paris, France
| | - Sarah Genon
- GIGA-CRC In vivo Imaging, University of Liège, Liège, Belgium
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7) Research Centre Jülich, Jülich, Germany
| | - Elvis Dohmatob
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France
- Criteo AI Lab, Paris, France
| | | | - Chantal Ginisty
- Université Paris-Saclay, CEA, UNIACT, NeuroSpin, 91191 Gif-sur-Yvette, France
| | | | - Séverine Roger
- Université Paris-Saclay, CEA, UNIACT, NeuroSpin, 91191 Gif-sur-Yvette, France
| | - Yann Lecomte
- Université Paris-Saclay, CEA, UNIACT, NeuroSpin, 91191 Gif-sur-Yvette, France
| | - Valérie Berland
- Université Paris-Saclay, CEA, UNIACT, NeuroSpin, 91191 Gif-sur-Yvette, France
| | - Laurence Laurier
- Université Paris-Saclay, CEA, UNIACT, NeuroSpin, 91191 Gif-sur-Yvette, France
| | | | | | - Christine Doublé
- Université Paris-Saclay, CEA, UNIACT, NeuroSpin, 91191 Gif-sur-Yvette, France
| | - Bernadette Martins
- Université Paris-Saclay, CEA, UNIACT, NeuroSpin, 91191 Gif-sur-Yvette, France
| | - Eric Salmon
- GIGA-CRC In vivo Imaging, University of Liège, Liège, Belgium
| | - Manuela Piazza
- Center for Mind/Brain Sciences, University of Trento, I-38068, Rovereto, Italy
| | - David Melcher
- Center for Mind/Brain Sciences, University of Trento, I-38068, Rovereto, Italy
| | - Mathias Pessiglione
- Motivation, Brain and Behavior (MBB) team, Institut du Cerveau (ICM), Inserm UMRS 1127, CNS UMR 7225, Sorbonne Université, Paris, France
| | - Virginie van Wassenhove
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, 91191, Gif/Yvette, France
| | - Evelyn Eger
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, 91191, Gif/Yvette, France
| | - Gaël Varoquaux
- Université Paris-Saclay, Inria, CEA, Palaiseau, 91120, France
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, 91191, Gif/Yvette, France
- Collège de France, Université Paris-Sciences-Lettres, Paris, France
| | - Lucie Hertz-Pannier
- Université Paris-Saclay, CEA, UNIACT, NeuroSpin, 91191 Gif-sur-Yvette, France
- UMR 1141, NeuroDiderot, Université de Paris, Paris, France
| | | |
Collapse
|
26
|
Jiahui G, Feilong M, Visconti di Oleggio Castello M, Guntupalli JS, Chauhan V, Haxby JV, Gobbini MI. Predicting individual face-selective topography using naturalistic stimuli. Neuroimage 2019; 216:116458. [PMID: 31843709 DOI: 10.1016/j.neuroimage.2019.116458] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2019] [Revised: 11/16/2019] [Accepted: 12/09/2019] [Indexed: 01/28/2023] Open
Abstract
Subject-specific, functionally defined areas are conventionally estimated with functional localizers and a simple contrast analysis between responses to different stimulus categories. Compared with functional localizers, naturalistic stimuli provide several advantages such as stronger and widespread brain activation, greater engagement, and increased subject compliance. In this study we demonstrate that a subject's idiosyncratic functional topography can be estimated with high fidelity from that subject's fMRI data obtained while watching a naturalistic movie using hyperalignment to project other subjects' localizer data into that subject's idiosyncratic cortical anatomy. These findings lay the foundation for developing an efficient tool for mapping functional topographies for a wide range of perceptual and cognitive functions in new subjects based only on fMRI data collected while watching an engaging, naturalistic stimulus and other subjects' localizer data from a normative sample.
Collapse
Affiliation(s)
- Guo Jiahui
- Center for Cognitive Neuroscience, Dartmouth College, NH, USA
| | - Ma Feilong
- Center for Cognitive Neuroscience, Dartmouth College, NH, USA
| | | | | | - Vassiki Chauhan
- Center for Cognitive Neuroscience, Dartmouth College, NH, USA
| | - James V Haxby
- Center for Cognitive Neuroscience, Dartmouth College, NH, USA
| | - M Ida Gobbini
- Cognitive Science, Dartmouth College, NH, USA; Dipartimento di Medicina Specialistica, Diagnostica e Sperimentale, Università di Bologna, Bologna, 40126, Italy.
| |
Collapse
|
27
|
Wang L, Baumgartner F, Kaule FR, Hanke M, Pollmann S. Individual face- and house-related eye movement patterns distinctively activate FFA and PPA. Nat Commun 2019; 10:5532. [PMID: 31797874 PMCID: PMC6892816 DOI: 10.1038/s41467-019-13541-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2019] [Accepted: 11/12/2019] [Indexed: 11/23/2022] Open
Abstract
We investigated if the fusiform face area (FFA) and the parahippocampal place area (PPA) contain a representation of fixation sequences that are typically used when looking at faces or houses. Here, we instructed observers to follow a dot presented on a uniform background. The dot’s movements represented gaze paths acquired separately from observers looking at face or house pictures. Even when gaze dispersion differences were controlled, face- and house-associated gaze patterns could be discriminated by fMRI multivariate pattern analysis in FFA and PPA, more so for the current observer’s own gazes than for another observer’s gaze. The discrimination of the observer’s own gaze patterns was not observed in early visual areas (V1 – V4) or superior parietal lobule and frontal eye fields. These findings indicate a link between perception and action—the complex gaze patterns that are used to explore faces and houses—in the FFA and PPA. The fusiform face area and parahippocampal place area respond to face and scene stimuli respectively. Here, the authors show using fMRI that these brain areas are also preferentially activated by eye movements associated with looking at faces and scenes even when no images are shown.
Collapse
Affiliation(s)
- Lihui Wang
- Department of Experimental Psychology, Otto-von-Guericke University, Magdeburg, Germany.,Center for Behavioral Brain Sciences, Magdeburg, Germany.,Institute of Psychology and Behavioral Science, Shanghai Jiao Tong University, Shanghai, China.,Shanghai Key Laboratory of Psychotic Disorders, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Florian Baumgartner
- Department of Experimental Psychology, Otto-von-Guericke University, Magdeburg, Germany
| | - Falko R Kaule
- Department of Experimental Psychology, Otto-von-Guericke University, Magdeburg, Germany
| | - Michael Hanke
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.,Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Stefan Pollmann
- Department of Experimental Psychology, Otto-von-Guericke University, Magdeburg, Germany. .,Center for Behavioral Brain Sciences, Magdeburg, Germany. .,Beijing Key Laboratory of Learning and Cognition and School of Psychology, Capital Normal University, Beijing, China.
| |
Collapse
|
28
|
Li Y, Saxe R, Anzellotti S. Intersubject MVPD: Empirical comparison of fMRI denoising methods for connectivity analysis. PLoS One 2019; 14:e0222914. [PMID: 31550276 PMCID: PMC6759145 DOI: 10.1371/journal.pone.0222914] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2019] [Accepted: 09/10/2019] [Indexed: 11/18/2022] Open
Abstract
Noise is a major challenge for the analysis of fMRI data in general and for connectivity analyses in particular. As researchers develop increasingly sophisticated tools to model statistical dependence between the fMRI signal in different brain regions, there is a risk that these models may increasingly capture artifactual relationships between regions, that are the result of noise. Thus, choosing optimal denoising methods is a crucial step to maximize the accuracy and reproducibility of connectivity models. Most comparisons between denoising methods require knowledge of the ground truth: of what is the ‘real signal’. For this reason, they are usually based on simulated fMRI data. However, simulated data may not match the statistical properties of real data, limiting the generalizability of the conclusions. In this article, we propose an approach to evaluate denoising methods using real (non-simulated) fMRI data. First, we introduce an intersubject version of multivariate pattern dependence (iMVPD) that computes the statistical dependence between a brain region in one participant, and another brain region in a different participant. iMVPD has the following advantages: 1) it is multivariate, 2) it trains and tests models on independent partitions of the real fMRI data, and 3) it generates predictions that are both between subjects and between regions. Since whole-brain sources of noise are more strongly correlated within subject than between subjects, we can use the difference between standard MVPD and iMVPD as a ‘discrepancy metric’ to evaluate denoising techniques (where more effective techniques should yield smaller differences). As predicted, the difference is the greatest in the absence of denoising methods. Furthermore, a combination of removal of the global signal and CompCorr optimizes denoising (among the set of denoising options tested).
Collapse
Affiliation(s)
- Yichen Li
- Courant Institute of Mathematical Sciences, New York University, New York, NY, United States of America
- Department of Computer Science, New York University, New York, NY, United States of America
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, United States of America
| | - Rebecca Saxe
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, United States of America
| | - Stefano Anzellotti
- Department of Psychology, Boston College, Chestnut Hill, MA, United States of America
- * E-mail:
| |
Collapse
|
29
|
The Hippocampal Film Editor: Sensitivity and Specificity to Event Boundaries in Continuous Experience. J Neurosci 2018; 38:10057-10068. [PMID: 30301758 PMCID: PMC6246887 DOI: 10.1523/jneurosci.0524-18.2018] [Citation(s) in RCA: 96] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2018] [Revised: 07/02/2018] [Accepted: 07/03/2018] [Indexed: 11/21/2022] Open
Abstract
The function of the human hippocampus is normally investigated by experimental manipulation of discrete events. Less is known about what triggers hippocampal activity during more naturalistic, continuous experience. We hypothesized that the hippocampus would be sensitive to the occurrence of event boundaries, that is, moments in time identified by observers as a transition between events. To address this, we analyzed functional MRI data from two groups: one (n = 253, 131 female) who viewed an 8.5 min film and another (n = 15, 6 female) who viewed a 120 min film. We observed a strong hippocampal response at boundaries defined by independent observers, which was modulated by boundary salience (the number of observers that identified each boundary). In the longer film, there were sufficient boundaries to show that this modulation remained after covarying out a large number of perceptual factors. This hypothesis-driven approach was complemented by a data-driven approach, in which we identified hippocampal events as moments in time with the strongest hippocampal activity. The correspondence between these hippocampal events and event boundaries was highly significant, revealing that the hippocampal response is not only sensitive, but also specific to event boundaries. We conclude that event boundaries play a key role in shaping hippocampal activity during encoding of naturalistic events.SIGNIFICANCE STATEMENT Recent years have seen the field of human neuroscience research transitioning from experiments with simple stimuli to the study of more complex and naturalistic experience. Nonetheless, our understanding of the function of many brain regions, such as the hippocampus, is based primarily on the study of brief, discrete events. As a result, we know little of what triggers hippocampal activity in real-life settings when we are exposed to a continuous stream of information. When does the hippocampus "decide" to respond during the encoding of naturalistic experience? We reveal here that hippocampal activity measured by fMRI during film watching is both sensitive and specific to event boundaries, identifying a potential mechanism whereby event boundaries shape experience by modulation of hippocampal activity.
Collapse
|
30
|
Pinho AL, Amadon A, Ruest T, Fabre M, Dohmatob E, Denghien I, Ginisty C, Becuwe-Desmidt S, Roger S, Laurier L, Joly-Testault V, Médiouni-Cloarec G, Doublé C, Martins B, Pinel P, Eger E, Varoquaux G, Pallier C, Dehaene S, Hertz-Pannier L, Thirion B. Individual Brain Charting, a high-resolution fMRI dataset for cognitive mapping. Sci Data 2018; 5:180105. [PMID: 29893753 PMCID: PMC5996851 DOI: 10.1038/sdata.2018.105] [Citation(s) in RCA: 79] [Impact Index Per Article: 13.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2017] [Accepted: 02/23/2018] [Indexed: 01/11/2023] Open
Abstract
Functional Magnetic Resonance Imaging (fMRI) has furthered brain mapping on perceptual, motor, as well as higher-level cognitive functions. However, to date, no data collection has systematically addressed the functional mapping of cognitive mechanisms at a fine spatial scale. The Individual Brain Charting (IBC) project stands for a high-resolution multi-task fMRI dataset that intends to provide the objective basis toward a comprehensive functional atlas of the human brain. The data refer to a cohort of 12 participants performing many different tasks. The large amount of task-fMRI data on the same subjects yields a precise mapping of the underlying functions, free from both inter-subject and inter-site variability. The present article gives a detailed description of the first release of the IBC dataset. It comprises a dozen of tasks, addressing both low- and high- level cognitive functions. This openly available dataset is thus intended to become a reference for cognitive brain mapping.
Collapse
Affiliation(s)
- Ana Luísa Pinho
- Parietal Team, Inria, Saclay, France
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
| | | | - Torsten Ruest
- Parietal Team, Inria, Saclay, France
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
| | - Murielle Fabre
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
- Cognitive Neuroimaging Unit, Saclay, France
- INSERM, Paris, France
- Paris-Sud University, Paris, France
| | - Elvis Dohmatob
- Parietal Team, Inria, Saclay, France
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
| | - Isabelle Denghien
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
- Cognitive Neuroimaging Unit, Saclay, France
- INSERM, Paris, France
- Paris-Sud University, Paris, France
| | | | | | - Séverine Roger
- Neurospin, CEA, Saclay, France
- UNIACT-U1129, Paris, France
| | | | | | | | | | | | | | - Evelyn Eger
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
- Cognitive Neuroimaging Unit, Saclay, France
- INSERM, Paris, France
- Paris-Sud University, Paris, France
| | - Gaël Varoquaux
- Parietal Team, Inria, Saclay, France
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
| | - Christophe Pallier
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
- Cognitive Neuroimaging Unit, Saclay, France
- INSERM, Paris, France
- Paris-Sud University, Paris, France
| | - Stanislas Dehaene
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
- Cognitive Neuroimaging Unit, Saclay, France
- INSERM, Paris, France
- Paris-Sud University, Paris, France
- Collège de France, Paris, France
| | - Lucie Hertz-Pannier
- Neurospin, CEA, Saclay, France
- INSERM, Paris, France
- UNIACT-U1129, Paris, France
- Paris Descartes University, Paris, France
| | - Bertrand Thirion
- Parietal Team, Inria, Saclay, France
- Neurospin, CEA, Saclay, France
- Paris-Saclay University, Paris, France
| |
Collapse
|
31
|
Lu J, Wang X, Qing Z, Li Z, Zhang W, Liu Y, Yuan L, Cheng L, Li M, Zhu B, Zhang X, Yang QX, Zhang B. Detectability and reproducibility of the olfactory fMRI signal under the influence of magnetic susceptibility artifacts in the primary olfactory cortex. Neuroimage 2018; 178:613-621. [PMID: 29885483 DOI: 10.1016/j.neuroimage.2018.06.008] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2018] [Revised: 06/03/2018] [Accepted: 06/04/2018] [Indexed: 10/14/2022] Open
Abstract
For human olfactory functional MRI studies, the primary olfactory cortex (POC) suffers severe magnetic susceptibility artifacts, which adversely influences the detectability and reproducibility of the olfactory fMRI data and its clinical applications. The goal of this work is to assess the impacts of the image artifacts on the detectability and reproducibility of the olfactory activation in the POC. The severity of artifacts in the POC were classified into three levels using a Subjective Artifact score (SA_score). The mean temporal signal-to-noise ratio (tSNR) of the fMRI data acquired by a given MRI sequence and olfactory activation (β value) in POC were evaluated and compared to the concurrent activations in the primary visual cortex (Brodmann area 17, BA17) by an odor-visual association paradigm using ninety-nine normal human subjects. Our study revealed that the mean tSNR in POC was above the threshold for reliable detection of the functional activation signal, and, consequently, the mean olfactory activations in the POC were not significantly different from those in BA17. The reproducibility of the activation in the POC was assessed by a random half-split stimulation of a test-retest experiment. The overlap of the activation maps for all the trials (n = 1000) in the POC were not statistically different from that observed in BA17. These results show that the detectability and reproducibility of olfactory activation in the presence of susceptibility artifacts in the POC was at similar level of that in the visual cortex.
Collapse
Affiliation(s)
- Jiaming Lu
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Xin Wang
- Department of Radiology, Drum Tower Hospital, Clinical College of Nanjing Medical University, Nanjing, 210008, China
| | - Zhao Qing
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Zhu Li
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Wen Zhang
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Ying Liu
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Lihua Yuan
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Le Cheng
- Department of Radiology, Drum Tower Hospital, Clinical College of Nanjing Medical University, Nanjing, 210008, China
| | - Ming Li
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Bin Zhu
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Xin Zhang
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China
| | - Qing X Yang
- Radiology, Center for NMR Research, Penn State University College of Medicine, Hershey, PA, 17033, USA
| | - Bing Zhang
- Department of Radiology, Drum Tower Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, 210008, China.
| |
Collapse
|
32
|
Nastase SA, Halchenko YO, Connolly AC, Gobbini MI, Haxby JV. Neural Responses to Naturalistic Clips of Behaving Animals in Two Different Task Contexts. Front Neurosci 2018; 12:316. [PMID: 29867327 PMCID: PMC5962655 DOI: 10.3389/fnins.2018.00316] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2018] [Accepted: 04/24/2018] [Indexed: 12/30/2022] Open
Affiliation(s)
- Samuel A Nastase
- Department of Psychological and Brain Sciences, Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH, United States.,Princeton Neuroscience Institute, Princeton University, Princeton, NJ, United States
| | - Yaroslav O Halchenko
- Department of Psychological and Brain Sciences, Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH, United States
| | - Andrew C Connolly
- Department of Neurology, Geisel School of Medicine at Dartmouth, Hanover, NH, United States
| | - M Ida Gobbini
- Department of Psychological and Brain Sciences, Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH, United States.,Dipartimento di Medicina Specialistica, Diagnostica e Sperimentale, Medical School, University of Bologna, Bologna, Italy
| | - James V Haxby
- Department of Psychological and Brain Sciences, Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH, United States
| |
Collapse
|
33
|
Guntupalli JS, Wheeler KG, Gobbini MI. Disentangling the Representation of Identity from Head View Along the Human Face Processing Pathway. Cereb Cortex 2018; 27:46-53. [PMID: 28051770 PMCID: PMC5939212 DOI: 10.1093/cercor/bhw344] [Citation(s) in RCA: 62] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2016] [Accepted: 10/19/2016] [Indexed: 11/20/2022] Open
Abstract
Neural models of a distributed system for face perception implicate a network of regions in the ventral visual stream for recognition of identity. Here, we report a functional magnetic resonance imaging (fMRI) neural decoding study in humans that shows that this pathway culminates in the right inferior frontal cortex face area (rIFFA) with a representation of individual identities that has been disentangled from variable visual features in different images of the same person. At earlier stages in the pathway, processing begins in early visual cortex and the occipital face area with representations of head view that are invariant across identities, and proceeds to an intermediate level of representation in the fusiform face area in which identity is emerging but still entangled with head view. Three-dimensional, view-invariant representation of identities in the rIFFA may be the critical link to the extended system for face perception, affording activation of person knowledge and emotional responses to familiar faces.
Collapse
Affiliation(s)
- J Swaroop Guntupalli
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, USA.,Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH, USA
| | - Kelsey G Wheeler
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, USA.,Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH, USA
| | - M Ida Gobbini
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, USA.,Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH, USA.,Department of Specialized, Diagnostic, and Experimental Medicine (DIMES), Medical School, University of Bologna, Bologna 40127, Italy
| |
Collapse
|
34
|
Garcia-Diaz AI, Segura B, Baggio HC, Marti MJ, Valldeoriola F, Compta Y, Bargallo N, Uribe C, Campabadal A, Abos A, Junque C. Structural Brain Correlations of Visuospatial and Visuoperceptual Tests in Parkinson's Disease. J Int Neuropsychol Soc 2018; 24:33-44. [PMID: 28714429 PMCID: PMC5851059 DOI: 10.1017/s1355617717000583] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/02/2017] [Revised: 05/18/2017] [Accepted: 05/30/2017] [Indexed: 12/22/2022]
Abstract
BACKGROUND Diagnosis of mild cognitive impairment in Parkinson's disease (PD) is relevant because it is a marker for evolution to dementia. However, the selection of suitable tests to evaluate separate cognitive domains in mild cognitive impairment related to PD remains an open question. The current work aims to investigate the neuroanatomical correlates of several visuospatial/visuoperceptual tests using the same sample and a multimodal MRI approach. METHODS The study included 36 PD patients and 20 healthy subjects matched for age, sex, and education. The visuospatial/visuoperceptual tests selected were: Pentagon Copying Test (PCT), Judgment of Line Orientation Test (JLOT), Visual Form Discrimination Test (VFDT), Facial Recognition Test (FRT), Symbol Digit Modalities Test (SMDT), and clock copying task (CLOX2). FreeSurfer was used to assess cortical thickness, and tract-based spatial statistics was used for fractional anisotropy analysis. RESULTS Lower performance in the PCT, JLOT, and SDMT was associated with extensive cortical thickness reductions in lateral parietal and temporal regions. VFDT and CLOX2 did not show this common pattern and correlated with more limited medial occipito-temporal and occipito-parietal regions. Performance in all visuospatial/visuoperceptual tests correlated with fractional anisotropy in the corpus callosum. CONCLUSIONS Our findings show that JLOT, SDMT, and PCT, in addition to differentiating patients from controls, are suitable visuospatial/visuoperceptual tests to reflect cortical thinning in lateral temporo-parietal regions in PD patients. We did not observe the dissociation between dorsal and ventral streams that was expected according to the neuropsychological classification of visuospatial and visuoperceptual tests. (JINS, 2018, 24, 33-44).
Collapse
Affiliation(s)
- Anna Isabel Garcia-Diaz
- Department of Medicine, Faculty of Medicine and Health Science, University of Barcelona, Barcelona, Catalonia, Spain
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
| | - Barbara Segura
- Department of Medicine, Faculty of Medicine and Health Science, University of Barcelona, Barcelona, Catalonia, Spain
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
| | - Hugo Cesar Baggio
- Department of Medicine, Faculty of Medicine and Health Science, University of Barcelona, Barcelona, Catalonia, Spain
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
| | - Maria Jose Marti
- Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Catalonia, Spain
- Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas (CIBERNED), Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain
- Movement Disorders Unit, Neurology Service, Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain
| | - Francesc Valldeoriola
- Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Catalonia, Spain
- Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas (CIBERNED), Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain
- Movement Disorders Unit, Neurology Service, Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain
| | - Yaroslau Compta
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
- Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Catalonia, Spain
- Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas (CIBERNED), Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain
- Movement Disorders Unit, Neurology Service, Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain
| | - Nuria Bargallo
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
- Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Catalonia, Spain
- Centre de Diagnòstic per la Imatge, Hospital Clínic, Barcelona, Catalonia, Spain
| | - Carme Uribe
- Department of Medicine, Faculty of Medicine and Health Science, University of Barcelona, Barcelona, Catalonia, Spain
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
| | - Anna Campabadal
- Department of Medicine, Faculty of Medicine and Health Science, University of Barcelona, Barcelona, Catalonia, Spain
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
- Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Catalonia, Spain
| | - Alexandra Abos
- Department of Medicine, Faculty of Medicine and Health Science, University of Barcelona, Barcelona, Catalonia, Spain
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
| | - Carme Junque
- Department of Medicine, Faculty of Medicine and Health Science, University of Barcelona, Barcelona, Catalonia, Spain
- Neuroscience Institute, University of Barcelona, Barcelona, Catalonia, Spain
- Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Catalonia, Spain
- Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas (CIBERNED), Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain
| |
Collapse
|
35
|
Sengupta A, Yakupov R, Speck O, Pollmann S, Hanke M. Ultra high-field (7 T) multi-resolution fMRI data for orientation decoding in visual cortex. Data Brief 2017; 13:219-222. [PMID: 28616455 PMCID: PMC5459569 DOI: 10.1016/j.dib.2017.05.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2017] [Revised: 04/03/2017] [Accepted: 05/04/2017] [Indexed: 11/23/2022] Open
Abstract
Multivariate pattern classification methods have been successfully applied to decode orientation of visual grating stimuli from BOLD fMRI activity recorded in human visual cortex (Kamitani and Tong, 2005; Haynes and Rees, 2005) [12], [10]. Though there has been extensive research investigating the true spatial scale of the orientation specific signals (Op de Beeck, 2010; Swisher et al., 2010; Alink et al., 2013; Freeman et al., 2011, 2013) [2], [15], [1], [4], [5], it remained inconclusive what spatial acquisition resolution is required, or is optimal, for decoding analyses. The research article entitled "The effect of acquisition resolution on orientation decoding from V1 BOLD fMRI at 7 T" Sengupta et al. (2017) [14] studied the effect of spatial acquisition resolution and also analyzed the strength and spatial scale of orientation discriminating signals. In this article, for the first time, we present empirical ultra high-field fMRI data, obtained as a part of the aforementioned study, which were recorded at four spatial resolutions (0.8 mm, 1.4 mm, 2 mm, and 3 mm isotropic voxel size) for orientation decoding in visual cortex. The dataset is compliant with the BIDS (Brain Imaging Data Structure) format, and freely available from the OpenfMRI portal (dataset accession number: http://openfmri.org/dataset/ds000113c ds000113c).
Collapse
Affiliation(s)
- Ayan Sengupta
- Department of Experimental Psychology, Institute of Psychology II, Otto-von-Guericke University, Universitätsplatz 2, 39016 Magdeburg, Germany
- Psychoinformatics lab, Institute of Psychology II, Otto-von-Guericke University, Universitätsplatz 2, 39016 Magdeburg, Germany
- Sir Peter Mansfield Imaging Centre, School of Physics & Astronomy, University of Nottingham, UK
| | - Renat Yakupov
- Department of Biomedical Magnetic Resonance, Institute for Experimental Physics, Otto-von-Guericke University, Leipziger Str. 44, 39120 Magdeburg, Germany
| | - Oliver Speck
- Department of Biomedical Magnetic Resonance, Institute for Experimental Physics, Otto-von-Guericke University, Leipziger Str. 44, 39120 Magdeburg, Germany
- Center for Behavioral Brain Sciences, Universitätsplatz 2, 39016 Magdeburg, Germany
- Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany
- German Center for Neurodegenerative Disease (DZNE), site Magdeburg, Leipziger Straße 44, Germany
| | - Stefan Pollmann
- Department of Experimental Psychology, Institute of Psychology II, Otto-von-Guericke University, Universitätsplatz 2, 39016 Magdeburg, Germany
- Center for Behavioral Brain Sciences, Universitätsplatz 2, 39016 Magdeburg, Germany
| | - Michael Hanke
- Center for Behavioral Brain Sciences, Universitätsplatz 2, 39016 Magdeburg, Germany
- Psychoinformatics lab, Institute of Psychology II, Otto-von-Guericke University, Universitätsplatz 2, 39016 Magdeburg, Germany
| |
Collapse
|
36
|
Sengupta A, Yakupov R, Speck O, Pollmann S, Hanke M. The effect of acquisition resolution on orientation decoding from V1 BOLD fMRI at 7T. Neuroimage 2017; 148:64-76. [PMID: 28063973 DOI: 10.1016/j.neuroimage.2016.12.040] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2015] [Revised: 12/14/2016] [Accepted: 12/15/2016] [Indexed: 12/28/2022] Open
Abstract
A decade after it was shown that the orientation of visual grating stimuli can be decoded from human visual cortex activity by means of multivariate pattern classification of BOLD fMRI data, numerous studies have investigated which aspects of neuronal activity are reflected in BOLD response patterns and are accessible for decoding. However, it remains inconclusive what the effect of acquisition resolution on BOLD fMRI decoding analyses is. The present study is the first to provide empirical ultra high-field fMRI data recorded at four spatial resolutions (0.8mm, 1.4mm, 2mm, and 3mm isotropic voxel size) on this topic - in order to test hypotheses on the strength and spatial scale of orientation discriminating signals. We present detailed analysis, in line with predictions from previous simulation studies, about how the performance of orientation decoding varies with different acquisition resolutions. Moreover, we also examine different spatial filtering procedures and its effects on orientation decoding. Here we show that higher-resolution scans with subsequent down-sampling or low-pass filtering yield no benefit over scans natively recorded in the corresponding lower resolution regarding decoding accuracy. The orientation-related signal in the BOLD fMRI data is spatially broadband in nature, includes both high spatial frequency components, as well as large-scale biases previously proposed in the literature. Moreover, we found above chance-level contribution from large draining veins to orientation decoding. Acquired raw data were publicly released to facilitate further investigation.
Collapse
Affiliation(s)
- Ayan Sengupta
- Department of Experimental Psychology, Institute of Psychology II, Otto-von-Guericke University, Universitätsplatz 2, 39016 Magdeburg, Germany.
| | - Renat Yakupov
- Department of Biomedical Magnetic Resonance, Institute for Experimental Physics, Otto-von-Guericke University, Leipziger Str. 44, 39120 Magdeburg, Germany
| | - Oliver Speck
- Department of Biomedical Magnetic Resonance, Institute for Experimental Physics, Otto-von-Guericke University, Leipziger Str. 44, 39120 Magdeburg, Germany; Center for Behavioral Brain Sciences, Universitätsplatz 2, 39016 Magdeburg, Germany; Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany; German Center for Neurodegenerative Disease (DZNE), Site Magdeburg, Leipziger Straße 44, Germany
| | - Stefan Pollmann
- Department of Experimental Psychology, Institute of Psychology II, Otto-von-Guericke University, Universitätsplatz 2, 39016 Magdeburg, Germany; Center for Behavioral Brain Sciences, Universitätsplatz 2, 39016 Magdeburg, Germany
| | - Michael Hanke
- Center for Behavioral Brain Sciences, Universitätsplatz 2, 39016 Magdeburg, Germany; Psychoinformatics Lab, Institute of Psychology II, Otto-von-Guericke University, Universitätsplatz 2, 39016 Magdeburg, Germany
| |
Collapse
|
37
|
Hanke M, Adelhöfer N, Kottke D, Iacovella V, Sengupta A, Kaule FR, Nigbur R, Waite AQ, Baumgartner F, Stadler J. A studyforrest extension, simultaneous fMRI and eye gaze recordings during prolonged natural stimulation. Sci Data 2016; 3:160092. [PMID: 27779621 PMCID: PMC5079121 DOI: 10.1038/sdata.2016.92] [Citation(s) in RCA: 62] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2016] [Accepted: 09/13/2016] [Indexed: 11/28/2022] Open
Abstract
Here we present an update of the studyforrest (http://studyforrest.org) dataset that complements the previously released functional magnetic resonance imaging (fMRI) data for natural language processing with a new two-hour 3 Tesla fMRI acquisition while 15 of the original participants were shown an audio-visual version of the stimulus motion picture. We demonstrate with two validation analyses that these new data support modeling specific properties of the complex natural stimulus, as well as a substantial within-subject BOLD response congruency in brain areas related to the processing of auditory inputs, speech, and narrative when compared to the existing fMRI data for audio-only stimulation. In addition, we provide participants' eye gaze location as recorded simultaneously with fMRI, and an additional sample of 15 control participants whose eye gaze trajectories for the entire movie were recorded in a lab setting—to enable studies on attentional processes and comparative investigations on the potential impact of the stimulation setting on these processes.
Collapse
Affiliation(s)
- Michael Hanke
- Psychoinformatics Lab, Institute of Psychology, Otto-von-Guericke University, Magdeburg D-39016, Germany.,Center for Behavioral Brain Sciences, Magdeburg D-39016, Germany
| | - Nico Adelhöfer
- Psychoinformatics Lab, Institute of Psychology, Otto-von-Guericke University, Magdeburg D-39016, Germany
| | - Daniel Kottke
- Knowledge Management and Discovery Lab, Otto-von-Guericke University, Magdeburg D-39016, Germany
| | | | - Ayan Sengupta
- Experimental Psychology Lab, Institute of Psychology, Otto-von-Guericke University, Magdeburg D-39016, Germany
| | - Falko R Kaule
- Psychoinformatics Lab, Institute of Psychology, Otto-von-Guericke University, Magdeburg D-39016, Germany.,Visual Processing Laboratory, Department of Ophthalmology, Otto-von-Guericke University, Magdeburg D-39016, Germany
| | - Roland Nigbur
- Department of Neuropsychology, Institute of Psychology, Otto-von-Guericke University, Magdeburg D-39016, Germany
| | - Alexander Q Waite
- Psychoinformatics Lab, Institute of Psychology, Otto-von-Guericke University, Magdeburg D-39016, Germany
| | - Florian Baumgartner
- Experimental Psychology Lab, Institute of Psychology, Otto-von-Guericke University, Magdeburg D-39016, Germany
| | - Jörg Stadler
- Leibniz Institute for Neurobiology, Magdeburg D-39118, Germany
| |
Collapse
|
38
|
Häusler CO, Hanke M. An annotation of cuts, depicted locations, and temporal progression in the motion picture "Forrest Gump". F1000Res 2016; 5:2273. [PMID: 27781092 PMCID: PMC5034791 DOI: 10.12688/f1000research.9536.1] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/05/2016] [Indexed: 12/04/2022] Open
Abstract
Here we present an annotation of locations and temporal progression depicted in the movie “Forrest Gump”, as an addition to a large public functional brain imaging dataset (
http://studyforrest.org). The annotation provides information about the exact timing of each of the 870 shots, and the depicted location after every cut with a high, medium, and low level of abstraction. Additionally, four classes are used to distinguish the differences of the depicted time between shots. Each shot is also annotated regarding the type of location (interior/exterior) and time of day. This annotation enables further studies of visual perception, memory of locations, and the perception of time under conditions of real-life complexity using the studyforrest dataset.
Collapse
Affiliation(s)
- Christian O Häusler
- Psychoinformatics Lab, Department of Psychology, University of Magdeburg, Magdeburg, Germany
| | - Michael Hanke
- Psychoinformatics Lab, Department of Psychology, University of Magdeburg, Magdeburg, Germany; Center for Behavioral Brain Sciences, Magdeburg, Germany
| |
Collapse
|