1
|
Liu C, Jing J, Jiang J, Wen W, Zhu W, Li Z, Pan Y, Cai X, Liu H, Zhou Y, Meng X, Zhang J, Wang Y, Li H, Jiang Y, Zheng H, Wang S, Niu H, Kochan N, Brodaty H, Wei T, Sachdev P, Liu T, Wang Y. Relationships between brain structure-function coupling in normal aging and cognition: A cross-ethnicity population-based study. Neuroimage 2024; 299:120847. [PMID: 39265959 DOI: 10.1016/j.neuroimage.2024.120847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2024] [Revised: 08/19/2024] [Accepted: 09/09/2024] [Indexed: 09/14/2024] Open
Abstract
Increased efforts in neuroscience seek to understand how macro-anatomical and physiological connectomes cooperatively work to generate cognitive behaviors. However, the structure-function coupling characteristics in normal aging individuals remain unclear. Here, we developed an index, the Coupling in Brain Structural connectome and Functional connectome (C-BSF) index, to quantify regional structure-function coupling in a large community-based cohort. C-BSF used diffusion tensor imaging (DTI) and resting-state functional magnetic resonance imaging (fMRI) data from the Polyvascular Evaluation for Cognitive Impairment and Vascular Events study (PRECISE) cohort (2007 individuals, age: 61.15 ± 6.49 years) and the Sydney Memory and Ageing Study (MAS) cohort (254 individuals, age: 83.45 ± 4.33 years). We observed that structure-function coupling was the strongest in the visual network and the weakest in the ventral attention network. We also observed that the weaker structure-function coupling was associated with increased age and worse cognitive level of the participant. Meanwhile, the structure-function coupling in the visual network was associated with the visuospatial performance and partially mediated the connections between age and the visuospatial function. This work contributes to our understanding of the underlying brain mechanisms by which aging affects cognition and also help establish early diagnosis and treatment approaches for neurological diseases in the elderly.
Collapse
Affiliation(s)
- Chang Liu
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Jing Jing
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
| | - Jiyang Jiang
- Neuropsychiatric Institute, Prince of Wales Hospital, Sydney, NSW, Australia
| | - Wei Wen
- Neuropsychiatric Institute, Prince of Wales Hospital, Sydney, NSW, Australia; Centre for Healthy Brain Ageing (CHeBA), Discipline of Psychiatry and Mental Health, School of Clinical Medicine, UNSW Medicine, Sydney NSW 2052, Australia
| | - Wanlin Zhu
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Zixiao Li
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Yuesong Pan
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Xueli Cai
- Department of Neurology, Lishui Hospital, Zhejiang University School of Medicine, Lishui, Zhejiang, China
| | - Hao Liu
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Yijun Zhou
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Xia Meng
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Jicong Zhang
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Yilong Wang
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Hao Li
- China National Clinical Research Center for Neurological Diseases, Beijing, China
| | - Yong Jiang
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Huaguang Zheng
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Suying Wang
- Cerebrovascular Research Lab, Lishui Hospital, Zhejiang University School of Medicine, Lishui, Zhejiang, China
| | - Haijun Niu
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China
| | - Nicole Kochan
- Neuropsychiatric Institute, Prince of Wales Hospital, Sydney, NSW, Australia; Centre for Healthy Brain Ageing (CHeBA), Discipline of Psychiatry and Mental Health, School of Clinical Medicine, UNSW Medicine, Sydney NSW 2052, Australia
| | - Henry Brodaty
- Neuropsychiatric Institute, Prince of Wales Hospital, Sydney, NSW, Australia; Centre for Healthy Brain Ageing (CHeBA), Discipline of Psychiatry and Mental Health, School of Clinical Medicine, UNSW Medicine, Sydney NSW 2052, Australia
| | - Tiemin Wei
- Department of Cardiology, Lishui Hospital, Zhejiang University School of Medicine, Lishui, Zhejiang, China
| | - Perminder Sachdev
- Neuropsychiatric Institute, Prince of Wales Hospital, Sydney, NSW, Australia; Centre for Healthy Brain Ageing (CHeBA), Discipline of Psychiatry and Mental Health, School of Clinical Medicine, UNSW Medicine, Sydney NSW 2052, Australia
| | - Tao Liu
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, China.
| | - Yongjun Wang
- China National Clinical Research Center for Neurological Diseases, Beijing, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
| |
Collapse
|
2
|
Wen X, Cao Q, Zhao Y, Wu X, Zhang D. D-MHGCN: An End-to-End Individual Behavioral Prediction Model Using Dual Multi-Hop Graph Convolutional Network. IEEE J Biomed Health Inform 2024; 28:6130-6140. [PMID: 38935468 DOI: 10.1109/jbhi.2024.3420134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/29/2024]
Abstract
Predicting individual behavior is a crucial area of research in neuroscience. Graph Neural Networks (GNNs), as powerful tools for extracting graph-structured features, are increasingly being utilized in various functional connectivity (FC) based behavioral prediction tasks. However, current predictive models primarily focus on enhancing GNNs' ability to extract features from FC networks while neglecting the importance of upstream individual network construction quality. This oversight results in constructed functional networks that fail to adequately represent individual behavioral capacity, thereby affecting the subsequent prediction accuracy. To address this issue, we proposed a new GNN-based behavioral prediction framework, named Dual Multi-Hop Graph Convolutional Network (D-MHGCN). Through the joint training of two GCNs, this framework integrates individual functional network construction and behavioral prediction into a unified optimization model. It allows the model to dynamically adjust the individual functional cortical parcellation according to the downstream tasks, thus creating task-aware, individual-specific FCNs that largely enhance its ability to predict behavior scores. Additionally, we employed multi-hop graph convolution layers instead of traditional single-hop methods in GCN to capture complex hierarchical connectivity patterns in brain networks. Our experimental evaluations, conducted on the large, public Human Connectome Project dataset, demonstrate that our proposed method outperforms existing methods in various behavioral prediction tasks. Moreover, it produces more functionally homogeneous cortical parcellation, showcasing its practical utility and effectiveness. Our work not only enhances the accuracy of individual behavioral prediction but also provides deeper insights into the neural mechanisms underlying individual differences in behavior.
Collapse
|
3
|
Tehrani-Saleh A, McAuley JD, Adami C. Mechanism of Duration Perception in Artificial Brains Suggests New Model of Attentional Entrainment. Neural Comput 2024; 36:2170-2200. [PMID: 39177952 DOI: 10.1162/neco_a_01699] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Accepted: 05/28/2024] [Indexed: 08/24/2024]
Abstract
While cognitive theory has advanced several candidate frameworks to explain attentional entrainment, the neural basis for the temporal allocation of attention is unknown. Here we present a new model of attentional entrainment guided by empirical evidence obtained using a cohort of 50 artificial brains. These brains were evolved in silico to perform a duration judgment task similar to one where human subjects perform duration judgments in auditory oddball paradigms. We found that the artificial brains display psychometric characteristics remarkably similar to those of human listeners and exhibit similar patterns of distortions of perception when presented with out-of-rhythm oddballs. A detailed analysis of mechanisms behind the duration distortion suggests that attention peaks at the end of the tone, which is inconsistent with previous attentional entrainment models. Instead, the new model of entrainment emphasizes increased attention to those aspects of the stimulus that the brain expects to be highly informative.
Collapse
Affiliation(s)
- Ali Tehrani-Saleh
- Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824, U.S.A.
| | - J Devin McAuley
- Department of Psychology, Michigan State University, East Lansing, MI 48824, U.S.A.
| | - Christoph Adami
- Department of Microbiology, Genetics, and Immunology
- Program in Ecology, Evolution, and Behavior
- Department of Physics and Astronomy, Michigan State University, East Lansing, MI 48824, U.S.A.
| |
Collapse
|
4
|
Wei H, Feng C, Li F. Modeling biological memory network by an autonomous and adaptive multi-agent system. Brain Inform 2024; 11:23. [PMID: 39277566 PMCID: PMC11401808 DOI: 10.1186/s40708-024-00237-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2024] [Accepted: 09/02/2024] [Indexed: 09/17/2024] Open
Abstract
At the intersection of computation and cognitive science, graph theory is utilized as a formalized description of complex relationships description of complex relationships and structures, but traditional graph models are static, lack the dynamic and autonomous behaviors of biological neural networks, rely on algorithms with a global view. This study introduces a multi-agent system (MAS) model based on the graph theory, each agent equipped with adaptive learning and decision-making capabilities, thereby facilitating decentralized dynamic information memory, modeling and simulation of the brain's memory process. This decentralized approach transforms memory storage into the management of MAS paths, with each agent utilizing localized information for the dynamic formation and modification of these paths, different path refers to different memory instance. The model's unique memory algorithm avoids a global view, instead relying on neighborhood-based interactions to enhance resource utilization. Emulating neuron electrophysiology, each agent's adaptive learning behavior is represented through a microcircuit centered around a variable resistor. Using principles of Ohm's and Kirchhoff's laws, we validated the model's efficacy in memorizing and retrieving data through computer simulations. This approach offers a plausible neurobiological explanation for memory realization and validates the memory trace theory at a system level.
Collapse
Affiliation(s)
- Hui Wei
- Laboratory of Algorithms for Cognitive Models, School of Computer Science, Fudan University, No. 2005 Songhu Rd, Yangpu District, Shanghai, 200438, China.
| | - Chenyue Feng
- Laboratory of Algorithms for Cognitive Models, School of Computer Science, Fudan University, No. 2005 Songhu Rd, Yangpu District, Shanghai, 200438, China
| | - Fushun Li
- Laboratory of Algorithms for Cognitive Models, School of Computer Science, Fudan University, No. 2005 Songhu Rd, Yangpu District, Shanghai, 200438, China
| |
Collapse
|
5
|
Wood JN, Pandey L, Wood SMW. Digital Twin Studies for Reverse Engineering the Origins of Visual Intelligence. Annu Rev Vis Sci 2024; 10:145-170. [PMID: 39292554 DOI: 10.1146/annurev-vision-101322-103628] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/20/2024]
Abstract
What are the core learning algorithms in brains? Nativists propose that intelligence emerges from innate domain-specific knowledge systems, whereas empiricists propose that intelligence emerges from domain-general systems that learn domain-specific knowledge from experience. We address this debate by reviewing digital twin studies designed to reverse engineer the learning algorithms in newborn brains. In digital twin studies, newborn animals and artificial agents are raised in the same environments and tested with the same tasks, permitting direct comparison of their learning abilities. Supporting empiricism, digital twin studies show that domain-general algorithms learn animal-like object perception when trained on the first-person visual experiences of newborn animals. Supporting nativism, digital twin studies show that domain-general algorithms produce innate domain-specific knowledge when trained on prenatal experiences (retinal waves). We argue that learning across humans, animals, and machines can be explained by a universal principle, which we call space-time fitting. Space-time fitting explains both empiricist and nativist phenomena, providing a unified framework for understanding the origins of intelligence.
Collapse
Affiliation(s)
- Justin N Wood
- Informatics Department, Indiana University Bloomington, Bloomington, Indiana, USA; , ,
- Cognitive Science Program, Indiana University Bloomington, Bloomington, Indiana, USA
- Neuroscience Department, Indiana University Bloomington, Bloomington, Indiana, USA
| | - Lalit Pandey
- Informatics Department, Indiana University Bloomington, Bloomington, Indiana, USA; , ,
| | - Samantha M W Wood
- Informatics Department, Indiana University Bloomington, Bloomington, Indiana, USA; , ,
- Cognitive Science Program, Indiana University Bloomington, Bloomington, Indiana, USA
- Neuroscience Department, Indiana University Bloomington, Bloomington, Indiana, USA
| |
Collapse
|
6
|
Clapp M, Bahuguna J, Giossi C, Rubin JE, Verstynen T, Vich C. CBGTPy: An extensible cortico-basal ganglia-thalamic framework for modeling biological decision making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.09.05.556301. [PMID: 37732280 PMCID: PMC10508778 DOI: 10.1101/2023.09.05.556301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/22/2023]
Abstract
Here we introduce CBGTPy, a virtual environment for designing and testing goal-directed agents with internal dynamics that are modeled on the cortico-basal-ganglia-thalamic (CBGT) pathways in the mammalian brain. CBGTPy enables researchers to investigate the internal dynamics of the CBGT system during a variety of tasks, allowing for the formation of testable predictions about animal behavior and neural activity. The framework has been designed around the principle of flexibility, such that many experimental parameters in a decision making paradigm can be easily defined and modified. Here we demonstrate the capabilities of CBGTPy across a range of single and multi-choice tasks, highlighting the ease of set up and the biologically realistic behavior that it produces. We show that CBGTPy is extensible enough to apply to a range of experimental protocols and to allow for the implementation of model extensions with minimal developmental effort.
Collapse
Affiliation(s)
- Matthew Clapp
- Department of Psychology & Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States of America
| | - Jyotika Bahuguna
- Department of Psychology & Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States of America
| | - Cristina Giossi
- Departament de Ciències Matemàtiques i Informàtica, Universitat de les Illes Balears, Palma, Spain
- Institute of Applied Computing and Community Code, Palma, Spain
| | - Jonathan E. Rubin
- Center for the Neural Basis of Cognition, Pittsburgh, Pennsylvania, United States of America
- Department of Mathematics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
| | - Timothy Verstynen
- Department of Psychology & Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States of America
- Center for the Neural Basis of Cognition, Pittsburgh, Pennsylvania, United States of America
| | - Catalina Vich
- Departament de Ciències Matemàtiques i Informàtica, Universitat de les Illes Balears, Palma, Spain
- Institute of Applied Computing and Community Code, Palma, Spain
| |
Collapse
|
7
|
Stoll A, Maier A, Krauss P, Gerum R, Schilling A. Coincidence detection and integration behavior in spiking neural networks. Cogn Neurodyn 2024; 18:1753-1765. [PMID: 39104689 PMCID: PMC11297875 DOI: 10.1007/s11571-023-10038-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 09/11/2023] [Accepted: 11/09/2023] [Indexed: 08/07/2024] Open
Abstract
Recently, the interest in spiking neural networks (SNNs) remarkably increased, as up to now some key advances of biological neural networks are still out of reach. Thus, the energy efficiency and the ability to dynamically react and adapt to input stimuli as observed in biological neurons is still difficult to achieve. One neuron model commonly used in SNNs is the leaky-integrate-and-fire (LIF) neuron. LIF neurons already show interesting dynamics and can be run in two operation modes: coincidence detectors for low and integrators for high membrane decay times, respectively. However, the emergence of these modes in SNNs and the consequence on network performance and information processing ability is still elusive. In this study, we examine the effect of different decay times in SNNs trained with a surrogate-gradient-based approach. We propose two measures that allow to determine the operation mode of LIF neurons: the number of contributing input spikes and the effective integration interval. We show that coincidence detection is characterized by a low number of input spikes as well as short integration intervals, whereas integration behavior is related to many input spikes over long integration intervals. We find the two measures to linearly correlate via a correlation factor that depends on the decay time. Thus, the correlation factor as function of the decay time shows a powerlaw behavior, which could be an intrinsic property of LIF networks. We argue that our work could be a starting point to further explore the operation modes in SNNs to boost efficiency and biological plausibility. Supplementary Information The online version of this article (10.1007/s11571-023-10038-0) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Andreas Stoll
- Pattern Recognition Lab, University Erlangen-Nürnberg, Erlangen, Germany
| | - Andreas Maier
- Pattern Recognition Lab, University Erlangen-Nürnberg, Erlangen, Germany
| | - Patrick Krauss
- Pattern Recognition Lab, University Erlangen-Nürnberg, Erlangen, Germany
- Neuroscience Lab, University Hospital Erlangen, Erlangen, Germany
| | - Richard Gerum
- Department of Physics and Astronomy, York University, Toronto, Canada
| | - Achim Schilling
- Pattern Recognition Lab, University Erlangen-Nürnberg, Erlangen, Germany
- Neuroscience Lab, University Hospital Erlangen, Erlangen, Germany
| |
Collapse
|
8
|
Schilling A, Krauss P. The Bayesian brain: world models and conscious dimensions of auditory phantom perception. J Neurophysiol 2024; 132:317-318. [PMID: 38958284 DOI: 10.1152/jn.00263.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2024] [Accepted: 06/21/2024] [Indexed: 07/04/2024] Open
Affiliation(s)
- Achim Schilling
- Neuroscience Lab, University Hospital Erlangen, Erlangen, Germany
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Erlangen, Germany
| | - Patrick Krauss
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
9
|
Dingil AE, Pribyl O. Understanding state-of-the-art situation of transport planning strategies in earthquake-prone areas by using AI-supported literature review methodology. Heliyon 2024; 10:e33645. [PMID: 39040344 PMCID: PMC11261024 DOI: 10.1016/j.heliyon.2024.e33645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Revised: 06/13/2024] [Accepted: 06/25/2024] [Indexed: 07/24/2024] Open
Abstract
Aim This review aims to explore earthquake-based transport strategies in seismic areas, providing state-of-the-art insights into the components necessary to guide urban planners and policymakers in their decision-making processes. Outputs The review provides a variety of methodologies and approaches employed for the reinforcement planning and emergency demand management to analyze and evaluate the impact of seismic events on transportation systems, in turn to develop strategies for preparedness, mitigation, response, and recovery phases. The selection of the appropriate approach depends on factors such as the specific transport system, urbanization level and type, built environment, and critical components involved. Originality and value Besides providing a distinctive illustration of the integration of transportation and seismic literature as a valuable consolidated resource, this article introduces a novel methodology named ALARM for conducting state-of-the-art reviews on any topic, incorporating AI through the utilization of large language models (LLMs) built upon transformer deep neural networks, along with indexing data structures (in this study mainly OPEN-AI DAVINCI-003 model and vector-storing index). Hence, it is of paramount significance as the first instance of implementing LLMs within academic review standards. This paves the way for the potential integration of AI and human collaboration to become a standard practice under enhanced criteria for comprehending and analyzing specific information.
Collapse
Affiliation(s)
- Ali Enes Dingil
- LAMbDA Lab. (λ), Faculty of Transportation Sciences, Czech Technical University, 110 00 Prague, Czech Republic
| | - Ondrej Pribyl
- LAMbDA Lab. (λ), Faculty of Transportation Sciences, Czech Technical University, 110 00 Prague, Czech Republic
| |
Collapse
|
10
|
Huang H, Li R, Qiao X, Li X, Li Z, Chen S, Yao Y, Wang F, Zhang X, Lin K, Zhang J. Attentional control influence habituation through modulation of connectivity patterns within the prefrontal cortex: Insights from stereo-EEG. Neuroimage 2024; 294:120640. [PMID: 38719154 DOI: 10.1016/j.neuroimage.2024.120640] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2023] [Revised: 04/28/2024] [Accepted: 05/06/2024] [Indexed: 05/13/2024] Open
Abstract
Attentional control, guided by top-down processes, enables selective focus on pertinent information, while habituation, influenced by bottom-up factors and prior experiences, shapes cognitive responses by emphasizing stimulus relevance. These two fundamental processes collaborate to regulate cognitive behavior, with the prefrontal cortex and its subregions playing a pivotal role. Nevertheless, the intricate neural mechanisms underlying the interaction between attentional control and habituation are still a subject of ongoing exploration. To our knowledge, there is a dearth of comprehensive studies on the functional connectivity between subsystems within the prefrontal cortex during attentional control processes in both primates and humans. Utilizing stereo-electroencephalogram (SEEG) recordings during the Stroop task, we observed top-down dominance effects and corresponding connectivity patterns among the orbitofrontal cortex (OFC), the middle frontal gyrus (MFG), and the inferior frontal gyrus (IFG) during heightened attentional control. These findings highlighting the involvement of OFC in habituation through top-down attention. Our study unveils unique connectivity profiles, shedding light on the neural interplay between top-down and bottom-up attentional control processes, shaping goal-directed attention.
Collapse
Affiliation(s)
- Huimin Huang
- Brain Cognition and Computing Lab, National Engineering Research Center for E-learning, Central China Normal University, Wuhan, Hubei, China; Brain Cognition and Intelligent Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen University, Xiamen, Fujian, China
| | - Rui Li
- Brain Cognition and Computing Lab, National Engineering Research Center for E-learning, Central China Normal University, Wuhan, Hubei, China
| | - Xiaojun Qiao
- Brain Cognition and Computing Lab, National Engineering Research Center for E-learning, Central China Normal University, Wuhan, Hubei, China
| | - Xiaoran Li
- Brain Cognition and Intelligent Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen University, Xiamen, Fujian, China
| | - Ziyue Li
- Brain Cognition and Intelligent Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen University, Xiamen, Fujian, China
| | - Siyi Chen
- Brain Cognition and Intelligent Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen University, Xiamen, Fujian, China
| | - Yi Yao
- Epilepsy Center, Xiamen Humanity Hospital, Xiamen, Fujian, China
| | - Fengpeng Wang
- Epilepsy Center, Xiamen Humanity Hospital, Xiamen, Fujian, China
| | - Xiaobin Zhang
- Epilepsy Center, Xiamen Humanity Hospital, Xiamen, Fujian, China
| | - Kaomin Lin
- Epilepsy Center, Xiamen Humanity Hospital, Xiamen, Fujian, China
| | - Junsong Zhang
- Brain Cognition and Intelligent Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen University, Xiamen, Fujian, China.
| |
Collapse
|
11
|
Li Y, Yang H, Gu S. Enhancing neural encoding models for naturalistic perception with a multi-level integration of deep neural networks and cortical networks. Sci Bull (Beijing) 2024; 69:1738-1747. [PMID: 38490889 DOI: 10.1016/j.scib.2024.02.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2023] [Revised: 06/27/2023] [Accepted: 02/23/2024] [Indexed: 03/17/2024]
Abstract
Cognitive neuroscience aims to develop computational models that can accurately predict and explain neural responses to sensory inputs in the cortex. Recent studies attempt to leverage the representation power of deep neural networks (DNNs) to predict the brain response and suggest a correspondence between artificial and biological neural networks in their feature representations. However, typical voxel-wise encoding models tend to rely on specific networks designed for computer vision tasks, leading to suboptimal brain-wide correspondence during cognitive tasks. To address this challenge, this work proposes a novel approach that upgrades voxel-wise encoding models through multi-level integration of features from DNNs and information from brain networks. Our approach combines DNN feature-level ensemble learning and brain atlas-level model integration, resulting in significant improvements in predicting whole-brain neural activity during naturalistic video perception. Furthermore, this multi-level integration framework enables a deeper understanding of the brain's neural representation mechanism, accurately predicting the neural response to complex visual concepts. We demonstrate that neural encoding models can be optimized by leveraging a framework that integrates both data-driven approaches and theoretical insights into the functional structure of the cortical networks.
Collapse
Affiliation(s)
- Yuanning Li
- School of Biomedical Engineering & State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University, Shanghai 201210, China.
| | - Huzheng Yang
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Shi Gu
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen 518110, China.
| |
Collapse
|
12
|
Toba MN, Malkinson TS, Howells H, Mackie MA, Spagna A. Same, Same but Different? A Multi-Method Review of the Processes Underlying Executive Control. Neuropsychol Rev 2024; 34:418-454. [PMID: 36967445 DOI: 10.1007/s11065-023-09577-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Accepted: 09/26/2022] [Indexed: 03/29/2023]
Abstract
Attention, working memory, and executive control are commonly considered distinct cognitive functions with important reciprocal interactions. Yet, longstanding evidence from lesion studies has demonstrated both overlap and dissociation in their behavioural expression and anatomical underpinnings, suggesting that a lower dimensional framework could be employed to further identify processes supporting goal-directed behaviour. Here, we describe the anatomical and functional correspondence between attention, working memory, and executive control by providing an overview of cognitive models, as well as recent data from lesion studies, invasive and non-invasive multimodal neuroimaging and brain stimulation. We emphasize the benefits of considering converging evidence from multiple methodologies centred on the identification of brain mechanisms supporting goal-driven behaviour. We propose that expanding on this approach should enable the construction of a comprehensive anatomo-functional framework with testable new hypotheses, and aid clinical neuroscience to intervene on impairments of executive functions.
Collapse
Affiliation(s)
- Monica N Toba
- Laboratory of Functional Neurosciences (UR UPJV 4559), University Hospital of Amiens and University of Picardie Jules Verne, Amiens, France.
- CHU Amiens Picardie - Site Sud, Centre Universitaire de Recherche en Santé, Avenue René Laënnec, 80054, Amiens Cedex 1, France.
| | - Tal Seidel Malkinson
- Paris Brain Institute, ICM, Hôpital de La Pitié-Salpêtrière, Sorbonne Université, Inserm U 1127, CNRS UMR 7225, 75013, Paris, France
- Université de Lorraine, CRAN, F-54000, Nancy, France
| | - Henrietta Howells
- Laboratory of Motor Control, Department of Medical Biotechnologies and Translational Medicine, Humanitas Research Hospital, IRCCS, Università Degli Studi Di Milano, Milan, Italy
| | - Melissa-Ann Mackie
- Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Alfredo Spagna
- Department of Psychology, Columbia University, New York, NY, 10025, USA.
| |
Collapse
|
13
|
Caplette L, Turk-Browne NB. Computational reconstruction of mental representations using human behavior. Nat Commun 2024; 15:4183. [PMID: 38760341 PMCID: PMC11101448 DOI: 10.1038/s41467-024-48114-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Accepted: 04/19/2024] [Indexed: 05/19/2024] Open
Abstract
Revealing how the mind represents information is a longstanding goal of cognitive science. However, there is currently no framework for reconstructing the broad range of mental representations that humans possess. Here, we ask participants to indicate what they perceive in images made of random visual features in a deep neural network. We then infer associations between the semantic features of their responses and the visual features of the images. This allows us to reconstruct the mental representations of multiple visual concepts, both those supplied by participants and other concepts extrapolated from the same semantic space. We validate these reconstructions in separate participants and further generalize our approach to predict behavior for new stimuli and in a new task. Finally, we reconstruct the mental representations of individual observers and of a neural network. This framework enables a large-scale investigation of conceptual representations.
Collapse
Affiliation(s)
| | - Nicholas B Turk-Browne
- Department of Psychology, Yale University, New Haven, CT, USA
- Wu Tsai Institute, Yale University, New Haven, CT, USA
| |
Collapse
|
14
|
Turk-Browne NB, Aslin RN. Infant neuroscience: how to measure brain activity in the youngest minds. Trends Neurosci 2024; 47:338-354. [PMID: 38570212 DOI: 10.1016/j.tins.2024.02.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 01/08/2024] [Accepted: 02/09/2024] [Indexed: 04/05/2024]
Abstract
The functional properties of the infant brain are poorly understood. Recent advances in cognitive neuroscience are opening new avenues for measuring brain activity in human infants. These include novel uses of existing technologies such as electroencephalography (EEG) and magnetoencephalography (MEG), the availability of newer technologies including functional near-infrared spectroscopy (fNIRS) and optically pumped magnetometry (OPM), and innovative applications of functional magnetic resonance imaging (fMRI) in awake infants during cognitive tasks. In this review article we catalog these available non-invasive methods, discuss the challenges and opportunities encountered when applying them to human infants, and highlight the potential they may ultimately hold for advancing our understanding of the youngest minds.
Collapse
Affiliation(s)
- Nicholas B Turk-Browne
- Department of Psychology, Yale University, New Haven, CT 06520, USA; Wu Tsai Institute, Yale University, New Haven, CT 06510, USA.
| | - Richard N Aslin
- Department of Psychology, Yale University, New Haven, CT 06520, USA; Child Study Center, Yale School of Medicine, New Haven, CT 06520, USA
| |
Collapse
|
15
|
Held M. Multitasking While Driving: Central Bottleneck or Problem State Interference? HUMAN FACTORS 2024; 66:1564-1582. [PMID: 36472950 PMCID: PMC10943624 DOI: 10.1177/00187208221143857] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Accepted: 11/04/2022] [Indexed: 06/17/2023]
Abstract
OBJECTIVE The objective of this work was to investigate if visuospatial attention and working memory load interact at a central control resource or at a task-specific, information processing resource during driving. BACKGROUND In previous multitasking driving experiments, interactions between different cognitive concepts (e.g., attention and working memory) have been found. These interactions have been attributed to a central bottleneck or to the so-called problem-state bottleneck, related to working memory usage. METHOD We developed two different cognitive models in the cognitive architecture ACT-R, which implement the central vs. problem-state bottleneck. The models performed a driving task, during which we varied visuospatial attention and working memory load. We evaluated the model by conducting an experiment with human participants and compared the behavioral data to the model's behavior. RESULTS The problem-state-bottleneck model could account for decreased driving performance due to working memory load as well as increased visuospatial attentional demands as compared to the central-bottleneck model, which could not account for effects of increased working memory load. CONCLUSION The interaction between working memory and visuospatial attention in our dual tasking experiment can be best characterized by a bottleneck in the working memory. The model results suggest that as working memory load becomes higher, drivers manage to perform fewer control actions, which leads to decreasing driving performance. APPLICATION Predictions about the effect of different mental loads can be used to quantify the contribution of each subtask allowing for precise assessments of the current overall mental load, which automated driving systems may adapt to.
Collapse
Affiliation(s)
- Moritz Held
- Moritz Held, Carl von Ossietzky Universität Oldenburg, Küpkersweg 74, Oldenburg 26129, Germany; e‐mail:
| |
Collapse
|
16
|
Herber CS, Pratt KJ, Shea JM, Villeda SA, Giocomo LM. Spatial Coding Dysfunction and Network Instability in the Aging Medial Entorhinal Cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.12.588890. [PMID: 38659809 PMCID: PMC11042240 DOI: 10.1101/2024.04.12.588890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
Across species, spatial memory declines with age, possibly reflecting altered hippocampal and medial entorhinal cortex (MEC) function. However, the integrity of cellular and network-level spatial coding in aged MEC is unknown. Here, we leveraged in vivo electrophysiology to assess MEC function in young, middle-aged, and aged mice navigating virtual environments. In aged grid cells, we observed impaired stabilization of context-specific spatial firing, correlated with spatial memory deficits. Additionally, aged grid networks shifted firing patterns often but with poor alignment to context changes. Aged spatial firing was also unstable in an unchanging environment. In these same mice, we identified 458 genes differentially expressed with age in MEC, 61 of which had expression correlated with spatial firing stability. These genes were enriched among interneurons and related to synaptic transmission. Together, these findings identify coordinated transcriptomic, cellular, and network changes in MEC implicated in impaired spatial memory in aging.
Collapse
Affiliation(s)
- Charlotte S. Herber
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA, 94305, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, 94305, USA
| | - Karishma J.B. Pratt
- Department of Anatomy, University of California San Francisco, 513 Parnassus Avenue, Box 0452, San Francisco, CA, 94143, USA
- These authors contributed equally
| | - Jeremy M. Shea
- Department of Anatomy, University of California San Francisco, 513 Parnassus Avenue, Box 0452, San Francisco, CA, 94143, USA
- These authors contributed equally
| | - Saul A. Villeda
- Department of Anatomy, University of California San Francisco, 513 Parnassus Avenue, Box 0452, San Francisco, CA, 94143, USA
- Bakar Aging Research Institute, San Francisco, CA, 94143, USA
| | - Lisa M. Giocomo
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA, 94305, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, 94305, USA
- Lead contact
| |
Collapse
|
17
|
Roelofs A. Wernicke's functional neuroanatomy model of language turns 150: what became of its psychological reflex arcs? Brain Struct Funct 2024:10.1007/s00429-024-02785-5. [PMID: 38581582 DOI: 10.1007/s00429-024-02785-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 03/05/2024] [Indexed: 04/08/2024]
Abstract
Wernicke (Der aphasische Symptomencomplex: Eine psychologische Studie auf anatomischer Basis. Cohn und Weigert, Breslau. https://wellcomecollection.org/works/dwv5w9rw , 1874) proposed a model of the functional neuroanatomy of spoken word repetition, production, and comprehension. At the heart of this epoch-making model are psychological reflex arcs underpinned by fiber tracts connecting sensory to motor areas. Here, I evaluate the central assumption of psychological reflex arcs in light of what we have learned about language in the brain during the past 150 years. I first describe Wernicke's 1874 model and the evidence he presented for it. Next, I discuss his updates of the model published in 1886 and posthumously in 1906. Although the model had an enormous immediate impact, it lost influence after the First World War. Unresolved issues included the anatomical underpinnings of the psychological reflex arcs, the role of auditory images in word production, and the sufficiency of psychological reflex arcs, which was questioned by Wundt (Grundzüge der physiologischen Psychologie. Engelmann, Leipzig. http://vlp.mpiwg-berlin.mpg.de/references?id=lit46 , 1874; Grundzüge der physiologischen Psychologie (Vol. 1, 5th ed.). Engelmann, Leipzig. http://vlp.mpiwg-berlin.mpg.de/references?id=lit806 , 1902). After a long dormant period, Wernicke's model was revived by Geschwind (Science 170:940-944. https://doi.org/10.1126/science.170.3961.940 , 1970; Selected papers on language and the brain. Reidel, Dordrecht, 1974), who proposed a version of it that differed in several important respects from Wernicke's original. Finally, I describe how new evidence from modern research has led to a novel view on language in the brain, supplementing contemporary equivalents of psychological reflex arcs by other mechanisms such as attentional control and assuming different neuroanatomical underpinnings. In support of this novel view, I report new analyses of patient data and computer simulations using the WEAVER++/ARC model (Roelofs 2014, 2022) that incorporates attentional control and integrates the new evidence.
Collapse
Affiliation(s)
- Ardi Roelofs
- Donders Institute for Brain, Cognition and Behaviour, Centre for Cognition, Radboud University, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands.
| |
Collapse
|
18
|
Lyu B, Marslen-Wilson WD, Fang Y, Tyler LK. Finding structure during incremental speech comprehension. eLife 2024; 12:RP89311. [PMID: 38577982 PMCID: PMC10997333 DOI: 10.7554/elife.89311] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/06/2024] Open
Abstract
A core aspect of human speech comprehension is the ability to incrementally integrate consecutive words into a structured and coherent interpretation, aligning with the speaker's intended meaning. This rapid process is subject to multidimensional probabilistic constraints, including both linguistic knowledge and non-linguistic information within specific contexts, and it is their interpretative coherence that drives successful comprehension. To study the neural substrates of this process, we extract word-by-word measures of sentential structure from BERT, a deep language model, which effectively approximates the coherent outcomes of the dynamic interplay among various types of constraints. Using representational similarity analysis, we tested BERT parse depths and relevant corpus-based measures against the spatiotemporally resolved brain activity recorded by electro-/magnetoencephalography when participants were listening to the same sentences. Our results provide a detailed picture of the neurobiological processes involved in the incremental construction of structured interpretations. These findings show when and where coherent interpretations emerge through the evaluation and integration of multifaceted constraints in the brain, which engages bilateral brain regions extending beyond the classical fronto-temporal language system. Furthermore, this study provides empirical evidence supporting the use of artificial neural networks as computational models for revealing the neural dynamics underpinning complex cognitive processes in the brain.
Collapse
Affiliation(s)
| | - William D Marslen-Wilson
- Centre for Speech, Language and the Brain, Department of Psychology, University of CambridgeCambridgeUnited Kingdom
| | - Yuxing Fang
- Centre for Speech, Language and the Brain, Department of Psychology, University of CambridgeCambridgeUnited Kingdom
| | - Lorraine K Tyler
- Centre for Speech, Language and the Brain, Department of Psychology, University of CambridgeCambridgeUnited Kingdom
| |
Collapse
|
19
|
Lopopolo A, Fedorenko E, Levy R, Rabovsky M. Cognitive Computational Neuroscience of Language: Using Computational Models to Investigate Language Processing in the Brain. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2024; 5:1-6. [PMID: 38645621 PMCID: PMC11025655 DOI: 10.1162/nol_e_00131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/23/2024]
Affiliation(s)
| | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Roger Levy
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Milena Rabovsky
- Department of Psychology, University of Potsdam, Potsdam, Germany
| |
Collapse
|
20
|
Farrugia C, Galdi P, Irazu IA, Scerri K, Bajada CJ. Local gradient analysis of human brain function using the Vogt-Bailey Index. Brain Struct Funct 2024; 229:497-512. [PMID: 38294531 PMCID: PMC10917869 DOI: 10.1007/s00429-023-02751-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Accepted: 12/09/2023] [Indexed: 02/01/2024]
Abstract
In this work, we take a closer look at the Vogt-Bailey (VB) index, proposed in Bajada et al. (NeuroImage 221:117140, 2020) as a tool for studying local functional homogeneity in the human cortex. We interpret the VB index in terms of the minimum ratio cut, a scaled cut-set weight that indicates whether a network can easily be disconnected into two parts having a comparable number of nodes. In our case, the nodes of the network consist of a brain vertex/voxel and its neighbours, and a given edge is weighted according to the affinity of the nodes it connects (as reflected by the modified Pearson correlation between their fMRI time series). Consequently, the minimum ratio cut quantifies the degree of small-scale similarity in brain activity: the greater the similarity, the 'heavier' the edges and the more difficult it is to disconnect the network, hence the higher the value of the minimum ratio cut. We compare the performance of the VB index with that of the Regional Homogeneity (ReHo) algorithm, commonly used to assess whether voxels in close proximity have synchronised fMRI signals, and find that the VB index is uniquely placed to detect sharp changes in the (local) functional organization of the human cortex.
Collapse
Affiliation(s)
- Christine Farrugia
- Faculty of Engineering, L-Università ta' Malta, Msida, Malta.
- University of Malta Magnetic Resonance Imaging Platform (UMRI), L-Università ta' Malta, Msida, Malta.
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK.
| | - Paola Galdi
- School of Informatics, The University of Edinburgh, Edinburgh, UK
| | | | - Kenneth Scerri
- Faculty of Engineering, L-Università ta' Malta, Msida, Malta
| | - Claude J Bajada
- University of Malta Magnetic Resonance Imaging Platform (UMRI), L-Università ta' Malta, Msida, Malta.
- Faculty of Medicine and Surgery, L-Università ta' Malta, Msida, Malta.
| |
Collapse
|
21
|
Prasad R, Tarai S, Bit A. Hybrid computational model depicts the contribution of non-significant lobes of human brain during the perception of emotional stimuli. Comput Methods Biomech Biomed Engin 2024:1-27. [PMID: 38328832 DOI: 10.1080/10255842.2024.2311876] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Accepted: 11/03/2023] [Indexed: 02/09/2024]
Abstract
Emotions are synchronizing responses of human brain while executing cognitive tasks. Earlier studies had revealed strong correlation between specific lobes of the brain to different types of emotional valence. In the current study, a comprehensive three-dimensional mapping of human brain for executing emotion specific tasks had been formulated. A hybrid computational machine learning model customized from Custom Weight Allocation Model (CWAM) and defined as Custom Rank Allocation Model (CRAM). This regression-based hybrid computational model computes the allocated tasks to different lobes of the brain during their respective executive stage. Event Related Potentials (ERP) were obtained with significant effect at P1, P2, P3, N170, N2, and N4. These ERPs were configured at Pz, Cz, F3, and T8 regions of the brain with maximal responses; while regions like Cz, C4 and F4 were also found to make effective contributions to elevate the responses of the brain, and thus these regions were configured as augmented source regions of the brain. In another circumstance of frequent -deviant - equal (FDE) presentation of the emotional stimuli, it was observed that the brain channels C3, C4, P3, P4, O1, O2, and Oz were contributing their emotional quotient to the overall response of the brain regions; whereas, the interaction effect was found presentable at O2, Oz, P3, P4, T8 and C3 regions of brain. The proposed computational model had identified the potential neural pathways during the execution of emotional task.
Collapse
Affiliation(s)
| | | | - Arindam Bit
- Department of Biomedical Engineering, NIT Raipur
| |
Collapse
|
22
|
Wise T, Emery K, Radulescu A. Naturalistic reinforcement learning. Trends Cogn Sci 2024; 28:144-158. [PMID: 37777463 PMCID: PMC10878983 DOI: 10.1016/j.tics.2023.08.016] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 08/23/2023] [Accepted: 08/24/2023] [Indexed: 10/02/2023]
Abstract
Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.
Collapse
Affiliation(s)
- Toby Wise
- Department of Neuroimaging, King's College London, London, UK.
| | - Kara Emery
- Center for Data Science, New York University, New York, NY, USA
| | - Angela Radulescu
- Center for Computational Psychiatry, Icahn School of Medicine at Mt. Sinai, New York, NY, USA
| |
Collapse
|
23
|
Misaghian K, Lugo JE, Faubert J. "Extended Descriptive Risk-Averse Bayesian Model" a More Comprehensive Approach in Simulating Complex Biological Motion Perception. Biomimetics (Basel) 2024; 9:27. [PMID: 38248601 PMCID: PMC10813264 DOI: 10.3390/biomimetics9010027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 12/08/2023] [Accepted: 12/28/2023] [Indexed: 01/23/2024] Open
Abstract
The ability to perceive biological motion is crucial for human survival, social interactions, and communication. Over the years, researchers have studied the mechanisms and neurobiological substrates that enable this ability. In a previous study, we proposed a descriptive Bayesian simulation model to represent the dorsal pathway of the visual system, which processes motion information. The model was inspired by recent studies that questioned the impact of dynamic form cues in biological motion perception and was trained to distinguish the direction of a soccer ball from a set of complex biological motion soccer-kick stimuli. However, the model was unable to simulate the reaction times of the athletes in a credible manner, and a few subjects could not be simulated. In this current work, we implemented a novel disremembering strategy to incorporate neural adaptation at the decision-making level, which improved the model's ability to simulate the athletes' reaction times. We also introduced receptive fields to detect rotational optic flow patterns not considered in the previous model to simulate a new subject and improve the correlation between the simulation and experimental data. The findings suggest that rotational optic flow plays a critical role in the decision-making process and sheds light on how different individuals perform at different levels. The correlation analysis of human versus simulation data shows a significant, almost perfect correlation between experimental and simulated angular thresholds and slopes, respectively. The analysis also reveals a strong relation between the average reaction times of the athletes and the simulations.
Collapse
Affiliation(s)
- Khashayar Misaghian
- Sage-Sentinel Smart Solutions, 1919-1 Tancha, Onna-son, Kunigami-gun, Okinawa 904-0495, Japan;
- Faubert Lab, School of Optometry, Université de Montréal, C.P. 6128, Montreal, QC H3C 3J7, Canada
| | - J. Eduardo Lugo
- Sage-Sentinel Smart Solutions, 1919-1 Tancha, Onna-son, Kunigami-gun, Okinawa 904-0495, Japan;
- Faubert Lab, School of Optometry, Université de Montréal, C.P. 6128, Montreal, QC H3C 3J7, Canada
- Facultad de Ciencias Físico-Matemáticas, Benemérita Universidad Autónoma de Puebla, Av. San Claudio y Av. 18 Sur, Colonia San Manuel Ciudad Universitaria, Puebla Pue 72570, Mexico
| | - Jocelyn Faubert
- Sage-Sentinel Smart Solutions, 1919-1 Tancha, Onna-son, Kunigami-gun, Okinawa 904-0495, Japan;
- Faubert Lab, School of Optometry, Université de Montréal, C.P. 6128, Montreal, QC H3C 3J7, Canada
| |
Collapse
|
24
|
Badrulhisham F, Pogatzki-Zahn E, Segelcke D, Spisak T, Vollert J. Machine learning and artificial intelligence in neuroscience: A primer for researchers. Brain Behav Immun 2024; 115:470-479. [PMID: 37972877 DOI: 10.1016/j.bbi.2023.11.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 10/16/2023] [Accepted: 11/08/2023] [Indexed: 11/19/2023] Open
Abstract
Artificial intelligence (AI) is often used to describe the automation of complex tasks that we would attribute intelligence to. Machine learning (ML) is commonly understood as a set of methods used to develop an AI. Both have seen a recent boom in usage, both in scientific and commercial fields. For the scientific community, ML can solve bottle necks created by complex, multi-dimensional data generated, for example, by functional brain imaging or *omics approaches. ML can here identify patterns that could not have been found using traditional statistic approaches. However, ML comes with serious limitations that need to be kept in mind: their tendency to optimise solutions for the input data means it is of crucial importance to externally validate any findings before considering them more than a hypothesis. Their black-box nature implies that their decisions usually cannot be understood, which renders their use in medical decision making problematic and can lead to ethical issues. Here, we present an introduction for the curious to the field of ML/AI. We explain the principles as commonly used methods as well as recent methodological advancements before we discuss risks and what we see as future directions of the field. Finally, we show practical examples of neuroscience to illustrate the use and limitations of ML.
Collapse
Affiliation(s)
| | - Esther Pogatzki-Zahn
- Department of Anaesthesiology, Intensive Care and Pain Medicine, University Hospital Muenster, Muenster, Germany
| | - Daniel Segelcke
- Department of Anaesthesiology, Intensive Care and Pain Medicine, University Hospital Muenster, Muenster, Germany
| | - Tamas Spisak
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Medicine Essen, Essen, Germany; Center for Translational Neuro- and Behavioral Sciences, Department of Neurology, University Medicine Essen, Essen, Germany
| | - Jan Vollert
- Department of Clinical and Biomedical Sciences, Faculty of Health and Life Sciences, University of Exeter, Exeter, United Kingdom; Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom.
| |
Collapse
|
25
|
Lin CHS, Do TT, Unsworth L, Garrido MI. Are we really Bayesian? Probabilistic inference shows sub-optimal knowledge transfer. PLoS Comput Biol 2024; 20:e1011769. [PMID: 38190413 PMCID: PMC10798629 DOI: 10.1371/journal.pcbi.1011769] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 01/19/2024] [Accepted: 12/18/2023] [Indexed: 01/10/2024] Open
Abstract
Numerous studies have found that the Bayesian framework, which formulates the optimal integration of the knowledge of the world (i.e. prior) and current sensory evidence (i.e. likelihood), captures human behaviours sufficiently well. However, there are debates regarding whether humans use precise but cognitively demanding Bayesian computations for behaviours. Across two studies, we trained participants to estimate hidden locations of a target drawn from priors with different levels of uncertainty. In each trial, scattered dots provided noisy likelihood information about the target location. Participants showed that they learned the priors and combined prior and likelihood information to infer target locations in a Bayes fashion. We then introduced a transfer condition presenting a trained prior and a likelihood that has never been put together during training. How well participants integrate this novel likelihood with their learned prior is an indicator of whether participants perform Bayesian computations. In one study, participants experienced the newly introduced likelihood, which was paired with a different prior, during training. Participants changed likelihood weighting following expected directions although the degrees of change were significantly lower than Bayes-optimal predictions. In another group, the novel likelihoods were never used during training. We found people integrated a new likelihood within (interpolation) better than the one outside (extrapolation) the range of their previous learning experience and they were quantitatively Bayes-suboptimal in both. We replicated the findings of both studies in a validation dataset. Our results showed that Bayesian behaviours may not always be achieved by a full Bayesian computation. Future studies can apply our approach to different tasks to enhance the understanding of decision-making mechanisms.
Collapse
Affiliation(s)
- Chin-Hsuan Sophie Lin
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia
| | - Trang Thuy Do
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia
| | - Lee Unsworth
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia
| | - Marta I. Garrido
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia
- Graeme Clark Institute for Biomedical Engineering, The University of Melbourne, Melbourne, Australia
| |
Collapse
|
26
|
Nozari E, Bertolero MA, Stiso J, Caciagli L, Cornblath EJ, He X, Mahadevan AS, Pappas GJ, Bassett DS. Macroscopic resting-state brain dynamics are best described by linear models. Nat Biomed Eng 2024; 8:68-84. [PMID: 38082179 PMCID: PMC11357987 DOI: 10.1038/s41551-023-01117-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Accepted: 09/26/2023] [Indexed: 12/22/2023]
Abstract
It is typically assumed that large networks of neurons exhibit a large repertoire of nonlinear behaviours. Here we challenge this assumption by leveraging mathematical models derived from measurements of local field potentials via intracranial electroencephalography and of whole-brain blood-oxygen-level-dependent brain activity via functional magnetic resonance imaging. We used state-of-the-art linear and nonlinear families of models to describe spontaneous resting-state activity of 700 participants in the Human Connectome Project and 122 participants in the Restoring Active Memory project. We found that linear autoregressive models provide the best fit across both data types and three performance metrics: predictive power, computational complexity and the extent of the residual dynamics unexplained by the model. To explain this observation, we show that microscopic nonlinear dynamics can be counteracted or masked by four factors associated with macroscopic dynamics: averaging over space and over time, which are inherent to aggregated macroscopic brain activity, and observation noise and limited data samples, which stem from technological limitations. We therefore argue that easier-to-interpret linear models can faithfully describe macroscopic brain dynamics during resting-state conditions.
Collapse
Affiliation(s)
- Erfan Nozari
- Department of Mechanical Engineering, University of California, Riverside, CA, USA
- Department of Electrical and Computer Engineering, University of California, Riverside, CA, USA
- Department of Bioengineering, University of California, Riverside, CA, USA
| | - Maxwell A Bertolero
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA
| | - Jennifer Stiso
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
| | - Lorenzo Caciagli
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA
| | - Eli J Cornblath
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
| | - Xiaosong He
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA
| | - Arun S Mahadevan
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA
| | - George J Pappas
- Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, USA
| | - Dani S Bassett
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Neurology, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA.
- Santa Fe Institute, Santa Fe, NM, USA.
| |
Collapse
|
27
|
Weidler T, Goebel R, Senden M. AngoraPy: A Python toolkit for modeling anthropomorphic goal-driven sensorimotor systems. Front Neuroinform 2023; 17:1223687. [PMID: 38204578 PMCID: PMC10777840 DOI: 10.3389/fninf.2023.1223687] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Accepted: 11/27/2023] [Indexed: 01/12/2024] Open
Abstract
Goal-driven deep learning increasingly supplements classical modeling approaches in computational neuroscience. The strength of deep neural networks as models of the brain lies in their ability to autonomously learn the connectivity required to solve complex and ecologically valid tasks, obviating the need for hand-engineered or hypothesis-driven connectivity patterns. Consequently, goal-driven models can generate hypotheses about the neurocomputations underlying cortical processing that are grounded in macro- and mesoscopic anatomical properties of the network's biological counterpart. Whereas, goal-driven modeling is already becoming prevalent in the neuroscience of perception, its application to the sensorimotor domain is currently hampered by the complexity of the methods required to train models comprising the closed sensation-action loop. This paper describes AngoraPy, a Python library that mitigates this obstacle by providing researchers with the tools necessary to train complex recurrent convolutional neural networks that model the human sensorimotor system. To make the technical details of this toolkit more approachable, an illustrative example that trains a recurrent toy model on in-hand object manipulation accompanies the theoretical remarks. An extensive benchmark on various classical, 3D robotic, and anthropomorphic control tasks demonstrates AngoraPy's general applicability to a wide range of tasks. Together with its ability to adaptively handle custom architectures, the flexibility of this toolkit demonstrates its power for goal-driven sensorimotor modeling.
Collapse
Affiliation(s)
- Tonio Weidler
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Centre, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Rainer Goebel
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Centre, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Mario Senden
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Centre, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
| |
Collapse
|
28
|
Chandran KS, Paul AM, Paul A, Ghosh K. Psychophysics may be the game-changer for deep neural networks (DNNs) to imitate the human vision. Behav Brain Sci 2023; 46:e388. [PMID: 38054301 DOI: 10.1017/s0140525x23001759] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
Psychologically faithful deep neural networks (DNNs) could be constructed by training with psychophysics data. Moreover, conventional DNNs are mostly monocular vision based, whereas the human brain relies mainly on binocular vision. DNNs developed as smaller vision agent networks associated with fundamental and less intelligent visual activities, can be combined to simulate more intelligent visual activities done by the biological brain.
Collapse
Affiliation(s)
- Keerthi S Chandran
- Center for Soft Computing Research, Indian Statistical Institute, Kolkata, India
- Machine Intelligence Unit, Indian Statistical Institute, Kolkata, India
| | - Amrita Mukherjee Paul
- Center for Soft Computing Research, Indian Statistical Institute, Kolkata, India
- Applied Sciences, IIIT Allahabad, Prayagraj, UP, India
| | - Avijit Paul
- Biomedical Engineering, Tufts University, Medford, MA, USA
| | - Kuntal Ghosh
- Machine Intelligence Unit, Indian Statistical Institute, Kolkata, India
| |
Collapse
|
29
|
Schilling A, Sedley W, Gerum R, Metzner C, Tziridis K, Maier A, Schulze H, Zeng FG, Friston KJ, Krauss P. Predictive coding and stochastic resonance as fundamental principles of auditory phantom perception. Brain 2023; 146:4809-4825. [PMID: 37503725 PMCID: PMC10690027 DOI: 10.1093/brain/awad255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 06/27/2023] [Accepted: 07/15/2023] [Indexed: 07/29/2023] Open
Abstract
Mechanistic insight is achieved only when experiments are employed to test formal or computational models. Furthermore, in analogy to lesion studies, phantom perception may serve as a vehicle to understand the fundamental processing principles underlying healthy auditory perception. With a special focus on tinnitus-as the prime example of auditory phantom perception-we review recent work at the intersection of artificial intelligence, psychology and neuroscience. In particular, we discuss why everyone with tinnitus suffers from (at least hidden) hearing loss, but not everyone with hearing loss suffers from tinnitus. We argue that intrinsic neural noise is generated and amplified along the auditory pathway as a compensatory mechanism to restore normal hearing based on adaptive stochastic resonance. The neural noise increase can then be misinterpreted as auditory input and perceived as tinnitus. This mechanism can be formalized in the Bayesian brain framework, where the percept (posterior) assimilates a prior prediction (brain's expectations) and likelihood (bottom-up neural signal). A higher mean and lower variance (i.e. enhanced precision) of the likelihood shifts the posterior, evincing a misinterpretation of sensory evidence, which may be further confounded by plastic changes in the brain that underwrite prior predictions. Hence, two fundamental processing principles provide the most explanatory power for the emergence of auditory phantom perceptions: predictive coding as a top-down and adaptive stochastic resonance as a complementary bottom-up mechanism. We conclude that both principles also play a crucial role in healthy auditory perception. Finally, in the context of neuroscience-inspired artificial intelligence, both processing principles may serve to improve contemporary machine learning techniques.
Collapse
Affiliation(s)
- Achim Schilling
- Neuroscience Lab, University Hospital Erlangen, 91054 Erlangen, Germany
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, 91058 Erlangen, Germany
| | - William Sedley
- Translational and Clinical Research Institute, Newcastle University Medical School, Newcastle upon Tyne NE2 4HH, UK
| | - Richard Gerum
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, 91058 Erlangen, Germany
- Department of Physics and Astronomy and Center for Vision Research, York University, Toronto, ON M3J 1P3, Canada
| | - Claus Metzner
- Neuroscience Lab, University Hospital Erlangen, 91054 Erlangen, Germany
| | | | - Andreas Maier
- Pattern Recognition Lab, University Erlangen-Nürnberg, 91058 Erlangen, Germany
| | - Holger Schulze
- Neuroscience Lab, University Hospital Erlangen, 91054 Erlangen, Germany
| | - Fan-Gang Zeng
- Center for Hearing Research, Departments of Anatomy and Neurobiology, Biomedical Engineering, Cognitive Sciences, Otolaryngology–Head and Neck Surgery, University of California Irvine, Irvine, CA 92697, USA
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Patrick Krauss
- Neuroscience Lab, University Hospital Erlangen, 91054 Erlangen, Germany
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, 91058 Erlangen, Germany
- Pattern Recognition Lab, University Erlangen-Nürnberg, 91058 Erlangen, Germany
| |
Collapse
|
30
|
Yamamori Y, Robinson OJ, Roiser JP. Approach-avoidance reinforcement learning as a translational and computational model of anxiety-related avoidance. eLife 2023; 12:RP87720. [PMID: 37963085 PMCID: PMC10645421 DOI: 10.7554/elife.87720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2023] Open
Abstract
Although avoidance is a prevalent feature of anxiety-related psychopathology, differences in the measurement of avoidance between humans and non-human animals hinder our progress in its theoretical understanding and treatment. To address this, we developed a novel translational measure of anxiety-related avoidance in the form of an approach-avoidance reinforcement learning task, by adapting a paradigm from the non-human animal literature to study the same cognitive processes in human participants. We used computational modelling to probe the putative cognitive mechanisms underlying approach-avoidance behaviour in this task and investigated how they relate to subjective task-induced anxiety. In a large online study (n = 372), participants who experienced greater task-induced anxiety avoided choices associated with punishment, even when this resulted in lower overall reward. Computational modelling revealed that this effect was explained by greater individual sensitivities to punishment relative to rewards. We replicated these findings in an independent sample (n = 627) and we also found fair-to-excellent reliability of measures of task performance in a sub-sample retested 1 week later (n = 57). Our findings demonstrate the potential of approach-avoidance reinforcement learning tasks as translational and computational models of anxiety-related avoidance. Future studies should assess the predictive validity of this approach in clinical samples and experimental manipulations of anxiety.
Collapse
Affiliation(s)
- Yumeya Yamamori
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
| | - Oliver J Robinson
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
- Research Department of Clinical, Educational and Health Psychology, University College LondonLondonUnited Kingdom
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
| |
Collapse
|
31
|
Schilling A, Schaette R, Sedley W, Gerum RC, Maier A, Krauss P. Editorial: Auditory perception and phantom perception in brains, minds and machines. Front Neurosci 2023; 17:1293552. [PMID: 37869508 PMCID: PMC10588478 DOI: 10.3389/fnins.2023.1293552] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 09/25/2023] [Indexed: 10/24/2023] Open
Affiliation(s)
- Achim Schilling
- Neuroscience Lab, University Hospital Erlangen, Erlangen, Germany
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Erlangen, Germany
| | - Roland Schaette
- UCL Ear Institute, University College London, London, United Kingdom
| | - William Sedley
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Richard Carl Gerum
- Department of Physics and Astronomy, York University, Toronto, ON, Canada
| | - Andreas Maier
- Pattern Recognition Lab, University Erlangen-Nürnberg, Erlangen, Germany
| | - Patrick Krauss
- Neuroscience Lab, University Hospital Erlangen, Erlangen, Germany
- Cognitive Computational Neuroscience Group, University Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
32
|
Singh G, Ramanathan M. Repurposing Artificial Intelligence Tools for Disease Modeling: Case Study of Face Recognition Deficits in Neurodegenerative Diseases. Clin Pharmacol Ther 2023; 114:862-873. [PMID: 37394678 DOI: 10.1002/cpt.2987] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 06/20/2023] [Indexed: 07/04/2023]
Abstract
Face recognition deficits occur in diseases such as prosopagnosia, autism, Alzheimer's disease, and dementias. The objective of this study was to evaluate whether degrading the architecture of artificial intelligence (AI) face recognition algorithms can model deficits in diseases. Two established face recognition models, convolutional-classification neural network (C-CNN) and Siamese network (SN), were trained on the FEI faces data set (~ 14 images/person for 200 persons). The trained networks were perturbed by reducing weights (weakening) and node count (lesioning) to emulate brain tissue dysfunction and lesions, respectively. Accuracy assessments were used as surrogates for face recognition deficits. The findings were compared with clinical outcomes from the Alzheimer's Disease Neuroimaging Initiative (ADNI) data set. Face recognition accuracy decreased gradually for weakening factors less than 0.55 for C-CNN, and 0.85 for SN. Rapid accuracy loss occurred at higher values. C-CNN accuracy was similarly affected by weakening any convolutional layer whereas SN accuracy was more sensitive to weakening of the first convolutional layer. SN accuracy declined gradually with a rapid drop when nearly all nodes were lesioned. C-CNN accuracy declined rapidly when as few as 10% of nodes were lesioned. CNN and SN were more sensitive to lesioning of the first convolutional layer. Overall, SN was more robust than C-CNN, and the findings from SN experiments were concordant with ADNI results. As predicted from modeling, brain network failure quotient was related to key clinical outcome measures for cognition and functioning. Perturbation of AI networks is a promising method for modeling disease progression effects on complex cognitive outcomes.
Collapse
Affiliation(s)
- Gargi Singh
- Department of Pharmaceutical Sciences, University at Buffalo, The State University of New York, Buffalo, New York, USA
| | - Murali Ramanathan
- Department of Pharmaceutical Sciences, University at Buffalo, The State University of New York, Buffalo, New York, USA
| |
Collapse
|
33
|
Borst JP, Aubin S, Stewart TC. A whole-task brain model of associative recognition that accounts for human behavior and neuroimaging data. PLoS Comput Biol 2023; 19:e1011427. [PMID: 37682986 PMCID: PMC10511112 DOI: 10.1371/journal.pcbi.1011427] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Revised: 09/20/2023] [Accepted: 08/10/2023] [Indexed: 09/10/2023] Open
Abstract
Brain models typically focus either on low-level biological detail or on qualitative behavioral effects. In contrast, we present a biologically-plausible spiking-neuron model of associative learning and recognition that accounts for both human behavior and low-level brain activity across the whole task. Based on cognitive theories and insights from machine-learning analyses of M/EEG data, the model proceeds through five processing stages: stimulus encoding, familiarity judgement, associative retrieval, decision making, and motor response. The results matched human response times and source-localized MEG data in occipital, temporal, prefrontal, and precentral brain regions; as well as a classic fMRI effect in prefrontal cortex. This required two main conceptual advances: a basal-ganglia-thalamus action-selection system that relies on brief thalamic pulses to change the functional connectivity of the cortex, and a new unsupervised learning rule that causes very strong pattern separation in the hippocampus. The resulting model shows how low-level brain activity can result in goal-directed cognitive behavior in humans.
Collapse
Affiliation(s)
- Jelmer P. Borst
- Bernoulli Institute, University of Groningen; Groningen, The Netherlands
| | - Sean Aubin
- Centre for Theoretical Neuroscience, University of Waterloo; Waterloo, Ontario, Canada
| | - Terrence C. Stewart
- National Research Council Canada, University of Waterloo Collaboration Centre; Waterloo, Ontario, Canada
| |
Collapse
|
34
|
Peelen MV, Downing PE. Testing cognitive theories with multivariate pattern analysis of neuroimaging data. Nat Hum Behav 2023; 7:1430-1441. [PMID: 37591984 PMCID: PMC7616245 DOI: 10.1038/s41562-023-01680-z] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2022] [Accepted: 07/12/2023] [Indexed: 08/19/2023]
Abstract
Multivariate pattern analysis (MVPA) has emerged as a powerful method for the analysis of functional magnetic resonance imaging, electroencephalography and magnetoencephalography data. The new approaches to experimental design and hypothesis testing afforded by MVPA have made it possible to address theories that describe cognition at the functional level. Here we review a selection of studies that have used MVPA to test cognitive theories from a range of domains, including perception, attention, memory, navigation, emotion, social cognition and motor control. This broad view reveals properties of MVPA that make it suitable for understanding the 'how' of human cognition, such as the ability to test predictions expressed at the item or event level. It also reveals limitations and points to future directions.
Collapse
Affiliation(s)
- Marius V Peelen
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands.
| | - Paul E Downing
- Cognitive Neuroscience Institute, Department of Psychology, Bangor University, Bangor, UK.
| |
Collapse
|
35
|
Schütt HH, Kipnis AD, Diedrichsen J, Kriegeskorte N. Statistical inference on representational geometries. eLife 2023; 12:e82566. [PMID: 37610302 PMCID: PMC10446828 DOI: 10.7554/elife.82566] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Accepted: 08/07/2023] [Indexed: 08/24/2023] Open
Abstract
Neuroscience has recently made much progress, expanding the complexity of both neural activity measurements and brain-computational models. However, we lack robust methods for connecting theory and experiment by evaluating our new big models with our new big data. Here, we introduce new inference methods enabling researchers to evaluate and compare models based on the accuracy of their predictions of representational geometries: A good model should accurately predict the distances among the neural population representations (e.g. of a set of stimuli). Our inference methods combine novel 2-factor extensions of crossvalidation (to prevent overfitting to either subjects or conditions from inflating our estimates of model accuracy) and bootstrapping (to enable inferential model comparison with simultaneous generalization to both new subjects and new conditions). We validate the inference methods on data where the ground-truth model is known, by simulating data with deep neural networks and by resampling of calcium-imaging and functional MRI data. Results demonstrate that the methods are valid and conclusions generalize correctly. These data analysis methods are available in an open-source Python toolbox (rsatoolbox.readthedocs.io).
Collapse
Affiliation(s)
- Heiko H Schütt
- Zuckerman Institute, Columbia UniversityNew YorkUnited States
| | | | | | | |
Collapse
|
36
|
Astle DE, Johnson MH, Akarca D. Toward computational neuroconstructivism: a framework for developmental systems neuroscience. Trends Cogn Sci 2023; 27:726-744. [PMID: 37263856 DOI: 10.1016/j.tics.2023.04.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 01/05/2023] [Accepted: 04/19/2023] [Indexed: 06/03/2023]
Abstract
Brain development is underpinned by complex interactions between neural assemblies, driving structural and functional change. This neuroconstructivism (the notion that neural functions are shaped by these interactions) is core to some developmental theories. However, due to their complexity, understanding underlying developmental mechanisms is challenging. Elsewhere in neurobiology, a computational revolution has shown that mathematical models of hidden biological mechanisms can bridge observations with theory building. Can we build a similar computational framework yielding mechanistic insights for brain development? Here, we outline the conceptual and technical challenges of addressing this theory gap, and demonstrate that there is great potential in specifying brain development as mathematically defined processes operating within physical constraints. We provide examples, alongside broader ingredients needed, as the field explores computational explanations of system-wide development.
Collapse
Affiliation(s)
- Duncan E Astle
- Department of Psychiatry, University of Cambridge, Cambridge, CB2 2QQ, UK; MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, CB2 7EF, UK.
| | - Mark H Johnson
- Department of Psychology, University of Cambridge, Cambridge, CB2 3EB, UK; Centre for Brain and Cognitive Development, Birkbeck, University of London, London, WC1E 7JL, UK
| | - Danyal Akarca
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, CB2 7EF, UK
| |
Collapse
|
37
|
Kopsick JD, Tecuatl C, Moradi K, Attili SM, Kashyap HJ, Xing J, Chen K, Krichmar JL, Ascoli GA. Robust Resting-State Dynamics in a Large-Scale Spiking Neural Network Model of Area CA3 in the Mouse Hippocampus. Cognit Comput 2023; 15:1190-1210. [PMID: 37663748 PMCID: PMC10473858 DOI: 10.1007/s12559-021-09954-2] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2021] [Accepted: 10/10/2021] [Indexed: 12/19/2022]
Abstract
Hippocampal area CA3 performs the critical auto-associative function underlying pattern completion in episodic memory. Without external inputs, the electrical activity of this neural circuit reflects the spontaneous spiking interplay among glutamatergic pyramidal neurons and GABAergic interneurons. However, the network mechanisms underlying these resting-state firing patterns are poorly understood. Leveraging the Hippocampome.org knowledge base, we developed a data-driven, large-scale spiking neural network (SNN) model of mouse CA3 with 8 neuron types, 90,000 neurons, 51 neuron-type specific connections, and 250,000,000 synapses. We instantiated the SNN in the CARLsim4 multi-GPU simulation environment using the Izhikevich and Tsodyks-Markram formalisms for neuronal and synaptic dynamics, respectively. We analyzed the resultant population activity upon transient activation. The SNN settled into stable oscillations with a biologically plausible grand-average firing frequency, which was robust relative to a wide range of transient activation. The diverse firing patterns of individual neuron types were consistent with existing knowledge of cell type-specific activity in vivo. Altered network structures that lacked neuron- or connection-type specificity were neither stable nor robust, highlighting the importance of neuron type circuitry. Additionally, external inputs reflecting dentate mossy fibers shifted the observed rhythms to the gamma band. We freely released the CARLsim4-Hippocampome framework on GitHub to test hippocampal hypotheses. Our SNN may be useful to investigate the circuit mechanisms underlying the computational functions of CA3. Moreover, our approach can be scaled to the whole hippocampal formation, which may contribute to elucidating how the unique neuronal architecture of this system subserves its crucial cognitive roles.
Collapse
Affiliation(s)
- Jeffrey D. Kopsick
- Interdepartmental Program in Neuroscience, George Mason University, Fairfax, VA, USA
| | - Carolina Tecuatl
- Bioengineering Department, Volgenau School of Engineering, George Mason University, Fairfax, VA, USA
| | - Keivan Moradi
- Interdepartmental Program in Neuroscience, George Mason University, Fairfax, VA, USA
| | - Sarojini M. Attili
- Interdepartmental Program in Neuroscience, George Mason University, Fairfax, VA, USA
| | - Hirak J. Kashyap
- Department of Computer Science, University of California, Irvine, Irvine, CA, USA
| | - Jinwei Xing
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, USA
| | - Kexin Chen
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, USA
| | - Jeffrey L. Krichmar
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, USA
- Department of Computer Science, University of California, Irvine, Irvine, CA, USA
| | - Giorgio A. Ascoli
- Interdepartmental Program in Neuroscience, George Mason University, Fairfax, VA, USA
- Bioengineering Department, Volgenau School of Engineering, George Mason University, Fairfax, VA, USA
| |
Collapse
|
38
|
Guassi Moreira JF, Méndez Leal AS, Waizman YH, Tashjian SM, Galván A, Silvers JA. Value-based neural representations predict social decision preferences. Cereb Cortex 2023:7161774. [PMID: 37183179 DOI: 10.1093/cercor/bhad144] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 03/30/2023] [Accepted: 03/31/2023] [Indexed: 05/16/2023] Open
Abstract
Social decision-making is omnipresent in everyday life, carrying the potential for both positive and negative consequences for the decision-maker and those closest to them. While evidence suggests that decision-makers use value-based heuristics to guide choice behavior, very little is known about how decision-makers' representations of other agents influence social choice behavior. We used multivariate pattern expression analyses on fMRI data to understand how value-based processes shape neural representations of those affected by one's social decisions and whether value-based encoding is associated with social decision preferences. We found that stronger value-based encoding of a given close other (e.g. parent) relative to a second close other (e.g. friend) was associated with a greater propensity to favor the former during subsequent social decision-making. These results are the first to our knowledge to explicitly show that value-based processes affect decision behavior via representations of close others.
Collapse
Affiliation(s)
| | | | - Yael H Waizman
- Department of Psychology, University of Southern California, Los Angeles, CA 90089, USA
| | - Sarah M Tashjian
- Division of the Humanities & Social Sciences, California Institute of Technology, Pasadena, CA 91125, USA
| | - Adriana Galván
- Department of Psychology, University of California, Los Angeles, CA 90095, USA
| | - Jennifer A Silvers
- Department of Psychology, University of California, Los Angeles, CA 90095, USA
| |
Collapse
|
39
|
Stoewer P, Schilling A, Maier A, Krauss P. Neural network based formation of cognitive maps of semantic spaces and the putative emergence of abstract concepts. Sci Rep 2023; 13:3644. [PMID: 36871003 PMCID: PMC9985610 DOI: 10.1038/s41598-023-30307-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Accepted: 02/21/2023] [Indexed: 03/06/2023] Open
Abstract
How do we make sense of the input from our sensory organs, and put the perceived information into context of our past experiences? The hippocampal-entorhinal complex plays a major role in the organization of memory and thought. The formation of and navigation in cognitive maps of arbitrary mental spaces via place and grid cells can serve as a representation of memories and experiences and their relations to each other. The multi-scale successor representation is proposed to be the mathematical principle underlying place and grid cell computations. Here, we present a neural network, which learns a cognitive map of a semantic space based on 32 different animal species encoded as feature vectors. The neural network successfully learns the similarities between different animal species, and constructs a cognitive map of 'animal space' based on the principle of successor representations with an accuracy of around 30% which is near to the theoretical maximum regarding the fact that all animal species have more than one possible successor, i.e. nearest neighbor in feature space. Furthermore, a hierarchical structure, i.e. different scales of cognitive maps, can be modeled based on multi-scale successor representations. We find that, in fine-grained cognitive maps, the animal vectors are evenly distributed in feature space. In contrast, in coarse-grained maps, animal vectors are highly clustered according to their biological class, i.e. amphibians, mammals and insects. This could be a putative mechanism enabling the emergence of new, abstract semantic concepts. Finally, even completely new or incomplete input can be represented by interpolation of the representations from the cognitive map with remarkable high accuracy of up to 95%. We conclude that the successor representation can serve as a weighted pointer to past memories and experiences, and may therefore be a crucial building block to include prior knowledge, and to derive context knowledge from novel input. Thus, our model provides a new tool to complement contemporary deep learning approaches on the road towards artificial general intelligence.
Collapse
Affiliation(s)
- Paul Stoewer
- Cognitive Computational Neuroscience Group, University Erlangen-Nuremberg, Erlangen, Germany
- Pattern Recognition Lab, University Erlangen-Nuremberg, Erlangen, Germany
| | - Achim Schilling
- Cognitive Computational Neuroscience Group, University Erlangen-Nuremberg, Erlangen, Germany
- Neuroscience Lab, University Hospital Erlangen, Erlangen, Germany
| | - Andreas Maier
- Pattern Recognition Lab, University Erlangen-Nuremberg, Erlangen, Germany
| | - Patrick Krauss
- Cognitive Computational Neuroscience Group, University Erlangen-Nuremberg, Erlangen, Germany.
- Pattern Recognition Lab, University Erlangen-Nuremberg, Erlangen, Germany.
- Neuroscience Lab, University Hospital Erlangen, Erlangen, Germany.
- Linguistics Lab, University Erlangen-Nuremberg, Erlangen, Germany.
| |
Collapse
|
40
|
Pedersen ML, Alnæs D, van der Meer D, Fernandez-Cabello S, Berthet P, Dahl A, Kjelkenes R, Schwarz E, Thompson WK, Barch DM, Andreassen OA, Westlye LT. Computational Modeling of the n-Back Task in the ABCD Study: Associations of Drift Diffusion Model Parameters to Polygenic Scores of Mental Disorders and Cardiometabolic Diseases. BIOLOGICAL PSYCHIATRY. COGNITIVE NEUROSCIENCE AND NEUROIMAGING 2023; 8:290-299. [PMID: 35427796 DOI: 10.1016/j.bpsc.2022.03.012] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Revised: 03/04/2022] [Accepted: 03/31/2022] [Indexed: 10/18/2022]
Abstract
BACKGROUND Cognitive dysfunction is common in mental disorders and represents a potential risk factor in childhood. The nature and extent of associations between childhood cognitive function and polygenic risk for mental disorders is unclear. We applied computational modeling to gain insight into mechanistic processes underlying decision making and working memory in childhood and their associations with polygenic risk scores (PRSs) for mental disorders and comorbid cardiometabolic diseases. METHODS We used the drift diffusion model to infer latent computational processes underlying decision making and working memory during the n-back task in 3707 children ages 9 to 10 years from the Adolescent Brain Cognitive Development (ABCD) Study. Single nucleotide polymorphism-based heritability was estimated for cognitive phenotypes, including computational parameters, aggregated n-back task performance, and neurocognitive assessments. PRSs were calculated for Alzheimer's disease, bipolar disorder, coronary artery disease (CAD), major depressive disorder, obsessive-compulsive disorder, schizophrenia, and type 2 diabetes. RESULTS Heritability estimates of cognitive phenotypes ranged from 12% to 38%. Bayesian mixed models revealed that slower accumulation of evidence was associated with higher PRSs for CAD and schizophrenia. Longer nondecision time was associated with higher PRSs for Alzheimer's disease and lower PRSs for CAD. Narrower decision threshold was associated with higher PRSs for CAD. Load-dependent effects on nondecision time and decision threshold were associated with PRSs for Alzheimer's disease and CAD, respectively. Aggregated neurocognitive test scores were not associated with PRSs for any of the mental or cardiometabolic phenotypes. CONCLUSIONS We identified distinct associations between computational cognitive processes and genetic risk for mental illness and cardiometabolic disease, which could represent childhood cognitive risk factors.
Collapse
Affiliation(s)
- Mads L Pedersen
- Department of Psychology, University of Oslo, Oslo, Norway; NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway.
| | - Dag Alnæs
- NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway; Bjørknes College, Institute of Psychology, Oslo, Norway
| | - Dennis van der Meer
- NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway; School of Mental Health and Neuroscience, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Sara Fernandez-Cabello
- NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Pierre Berthet
- Department of Psychology, University of Oslo, Oslo, Norway; NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Andreas Dahl
- Department of Psychology, University of Oslo, Oslo, Norway; NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Rikka Kjelkenes
- Department of Psychology, University of Oslo, Oslo, Norway; NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Emanuel Schwarz
- Department of Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Wesley K Thompson
- Division of Biostatistics and Department of Radiology, Population Neuroscience and Genetics Laboratory, University of California San Diego, La Jolla, California
| | - Deanna M Barch
- Departments of Psychological & Brain Sciences, Psychiatry, and Radiology, Washington University, St. Louis, Missouri
| | - Ole A Andreassen
- KG Jebsen Centre for Neurodevelopmental Disorders, University of Oslo, Oslo, Norway; NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Lars T Westlye
- Department of Psychology, University of Oslo, Oslo, Norway; KG Jebsen Centre for Neurodevelopmental Disorders, University of Oslo, Oslo, Norway; NORMENT, Division of Mental Health and Addiction, Oslo University Hospital and Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| |
Collapse
|
41
|
Hebart MN, Contier O, Teichmann L, Rockter AH, Zheng CY, Kidder A, Corriveau A, Vaziri-Pashkam M, Baker CI. THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior. eLife 2023; 12:e82580. [PMID: 36847339 PMCID: PMC10038662 DOI: 10.7554/elife.82580] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Accepted: 02/25/2023] [Indexed: 03/01/2023] Open
Abstract
Understanding object representations requires a broad, comprehensive sampling of the objects in our visual world with dense measurements of brain activity and behavior. Here, we present THINGS-data, a multimodal collection of large-scale neuroimaging and behavioral datasets in humans, comprising densely sampled functional MRI and magnetoencephalographic recordings, as well as 4.70 million similarity judgments in response to thousands of photographic images for up to 1,854 object concepts. THINGS-data is unique in its breadth of richly annotated objects, allowing for testing countless hypotheses at scale while assessing the reproducibility of previous findings. Beyond the unique insights promised by each individual dataset, the multimodality of THINGS-data allows combining datasets for a much broader view into object processing than previously possible. Our analyses demonstrate the high quality of the datasets and provide five examples of hypothesis-driven and data-driven applications. THINGS-data constitutes the core public release of the THINGS initiative (https://things-initiative.org) for bridging the gap between disciplines and the advancement of cognitive neuroscience.
Collapse
Affiliation(s)
- Martin N Hebart
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of HealthBethesdaUnited States
- Vision and Computational Cognition Group, Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
- Department of Medicine, Justus Liebig University GiessenGiessenGermany
| | - Oliver Contier
- Vision and Computational Cognition Group, Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
- Max Planck School of Cognition, Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
| | - Lina Teichmann
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of HealthBethesdaUnited States
| | - Adam H Rockter
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of HealthBethesdaUnited States
| | - Charles Y Zheng
- Machine Learning Core, National Institute of Mental Health, National Institutes of HealthBethesdaUnited States
| | - Alexis Kidder
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of HealthBethesdaUnited States
| | - Anna Corriveau
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of HealthBethesdaUnited States
| | - Maryam Vaziri-Pashkam
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of HealthBethesdaUnited States
| | - Chris I Baker
- Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of HealthBethesdaUnited States
| |
Collapse
|
42
|
Pan Y, Wen Y, Wang Y, Schilbach L, Chen J. Interpersonal coordination in schizophrenia: a concise update on paradigms, computations, and neuroimaging findings. PSYCHORADIOLOGY 2023; 3:kkad002. [PMID: 38666124 PMCID: PMC10917372 DOI: 10.1093/psyrad/kkad002] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 01/31/2023] [Accepted: 02/06/2023] [Indexed: 04/28/2024]
Affiliation(s)
- Yafeng Pan
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, Zhejiang 310058, China
| | - Yalan Wen
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, Zhejiang 310058, China
| | - Yajie Wang
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, Zhejiang 310058, China
| | - Leonhard Schilbach
- Department of General Psychiatry 2 and Neuroimaging Section, LVR-Klinikum Düsseldorf, Düsseldorf 40629, Germany
- Medical Faculty, Ludwig-Maximilians University, Munich 80539, Germany
| | - Ji Chen
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, Zhejiang 310058, China
- Department of Psychiatry, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang 322000, China
| |
Collapse
|
43
|
Guerrero LE, Castillo LF, Arango-López J, Moreira F. A systematic review of integrated information theory: a perspective from artificial intelligence and the cognitive sciences. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08328-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/10/2023]
|
44
|
Pirrone A, Tsetsos K. Toward an Atlas of Canonical Cognitive Mechanisms. Cogn Sci 2023; 47:e13243. [PMID: 36744746 DOI: 10.1111/cogs.13243] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 12/19/2022] [Accepted: 01/04/2023] [Indexed: 02/07/2023]
Abstract
A central goal in Cognitive Science is understanding the mechanisms that underlie cognition. Here, we contend that Cognitive Science, despite intense multidisciplinary efforts, has furnished surprisingly few mechanistic insights. We attribute this slow mechanistic progress to the fact that cognitive scientists insist on performing underdetermined exercises, deriving overparametrized mechanistic theories of complex behaviors and seeking validation of these theories to the elusive notions of optimality and biological plausibility. We propose that mechanistic progress in Cognitive Science will accelerate once cognitive scientists start focusing on simpler explananda that will enable them to chart an atlas of elementary cognitive operations. Looking forward, the next challenge for Cognitive Science will be to understand how these elementary cognitive processes are pieced together to explain complex behavior.
Collapse
Affiliation(s)
- Angelo Pirrone
- Centre for Philosophy of Natural and Social Science, London School of Economics
| | - Konstantinos Tsetsos
- School of Psychological Science, University of Bristol
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf
| |
Collapse
|
45
|
Martin VP, Lopez R, Dauvilliers Y, Rouas JL, Philip P, Micoulaud-Franchi JA. Sleepiness in adults: An umbrella review of a complex construct. Sleep Med Rev 2023; 67:101718. [PMID: 36455433 DOI: 10.1016/j.smrv.2022.101718] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Revised: 10/21/2022] [Accepted: 11/10/2022] [Indexed: 11/21/2022]
Abstract
Sleepiness involves many dimensions that require investigation. Since sleepiness is often defined operationally, we exhaustively inventoried all the assessment tools designed to measure it in an umbrella review, without any preconceptions, i.e. a review of reviews. We included all reviews and systematic reviews related to sleepiness assessment tools published up to March 2021. Three investigators independently assessed the eligibility of studies for inclusion and identified 36 relevant reviews. In total, 99 tools were identified and classified into 8 categories. We classified them depending on their category, their publication year and the number of mentions in the 36 included reviews. The 6 most frequently cited were the Epworth sleepiness scale, the multiple sleep latency test, the maintenance of wakefulness test, the Stanford sleepiness scale, the Karolinska sleepiness scale, and the psychomotor vigilance task. Despite the limitation that we may have missed some recently developed tools, this historical perspective on sleepiness measurement is a first step toward a better delineation of the different dimensions underlying the constructs of sleepiness, and will serve as a basis for further discussion in the clinical and research sleep community.
Collapse
Affiliation(s)
- Vincent P Martin
- Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, F-33400, Talence, France; Université de Bordeaux, SANPSY UMR 6033, F-33000, Bordeaux, France.
| | - Régis Lopez
- National Reference Center for Orphan Diseases, Narcolepsy, Idiopathic Hypersomnia and Kleine-Levin Syndrome, Sleep Disorders Center, Department of Neurology, Gui-de-Chauliac Hospital, CHU Montpellier, France; Institute for Neurosciences of Montpellier (INM), Montpellier University, INSERM, Montpellier, France
| | - Yves Dauvilliers
- National Reference Center for Orphan Diseases, Narcolepsy, Idiopathic Hypersomnia and Kleine-Levin Syndrome, Sleep Disorders Center, Department of Neurology, Gui-de-Chauliac Hospital, CHU Montpellier, France; Institute for Neurosciences of Montpellier (INM), Montpellier University, INSERM, Montpellier, France
| | - Jean-Luc Rouas
- Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, F-33400, Talence, France
| | - Pierre Philip
- Service de Médecine Universitaire du sommeil, CHU de Bordeaux, Place Amélie Raba-Leon, Bordeaux, France; Université de Bordeaux, SANPSY UMR 6033, F-33000, Bordeaux, France
| | - Jean-Arthur Micoulaud-Franchi
- Service de Médecine Universitaire du sommeil, CHU de Bordeaux, Place Amélie Raba-Leon, Bordeaux, France; Université de Bordeaux, SANPSY UMR 6033, F-33000, Bordeaux, France.
| |
Collapse
|
46
|
Abstract
A schema refers to a structured body of prior knowledge that captures common patterns across related experiences. Schemas have been studied separately in the realms of episodic memory and spatial navigation across different species and have been grounded in theories of memory consolidation, but there has been little attempt to integrate our understanding across domains, particularly in humans. We propose that experiences during navigation with many similarly structured environments give rise to the formation of spatial schemas (for example, the expected layout of modern cities) that share properties with but are distinct from cognitive maps (for example, the memory of a modern city) and event schemas (such as expected events in a modern city) at both cognitive and neural levels. We describe earlier theoretical frameworks and empirical findings relevant to spatial schemas, along with more targeted investigations of spatial schemas in human and non-human animals. Consideration of architecture and urban analytics, including the influence of scale and regionalization, on different properties of spatial schemas may provide a powerful approach to advance our understanding of spatial schemas.
Collapse
|
47
|
Tresp V, Sharifzadeh S, Li H, Konopatzki D, Ma Y. The Tensor Brain: A Unified Theory of Perception, Memory, and Semantic Decoding. Neural Comput 2023; 35:156-227. [PMID: 36417584 DOI: 10.1162/neco_a_01552] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 08/21/2022] [Indexed: 11/25/2022]
Abstract
We present a unified computational theory of an agent's perception and memory. In our model, both perception and memory are realized by different operational modes of the oscillating interactions between a symbolic index layer and a subsymbolic representation layer. The two layers form a bilayer tensor network (BTN). The index layer encodes indices for concepts, predicates, and episodic instances. The representation layer broadcasts information and reflects the cognitive brain state; it is our model of what authors have called the "mental canvas" or the "global workspace." As a bridge between perceptual input and the index layer, the representation layer enables the grounding of indices by their subsymbolic embeddings, which are implemented as connection weights linking both layers. The propagation of activation to earlier perceptual processing layers in the brain can lead to embodiments of indices. Perception and memories first create subsymbolic representations, which are subsequently decoded semantically to produce sequences of activated indices that form symbolic triple statements. The brain is a sampling engine: only activated indices are communicated to the remaining parts of the brain. Triple statements are dynamically embedded in the representation layer and embodied in earlier processing layers: the brain speaks to itself. Although memory appears to be about the past, its main purpose is to support the agent in the present and the future. Recent episodic memory provides the agent with a sense of the here and now. Remote episodic memory retrieves relevant past experiences to provide information about possible future scenarios. This aids the agent in decision making. "Future" episodic memory, based on expected future events, guides planning and action. Semantic memory retrieves specific information, which is not delivered by current perception, and defines priors for future observations. We argue that it is important for the agent to encode individual entities, not just classes and attributes. Perception is learning: episodic memories are constantly being formed, and we demonstrate that a form of self-supervised learning can acquire new concepts and refine existing ones. We test our model on a standard benchmark data set, which we expanded to contain richer representations for attributes, classes, and individuals. Our key hypothesis is that obtaining a better understanding of perception and memory is a crucial prerequisite to comprehending human-level intelligence.
Collapse
Affiliation(s)
| | | | - Hang Li
- LMU Munich and Siemens Munich, Germany
| | | | - Yunpu Ma
- LMU Munich and Siemens Munich, Germany
| |
Collapse
|
48
|
Makino H. Arithmetic value representation for hierarchical behavior composition. Nat Neurosci 2023; 26:140-149. [PMID: 36550292 PMCID: PMC9829535 DOI: 10.1038/s41593-022-01211-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Accepted: 10/21/2022] [Indexed: 12/24/2022]
Abstract
The ability to compose new skills from a preacquired behavior repertoire is a hallmark of biological intelligence. Although artificial agents extract reusable skills from past experience and recombine them in a hierarchical manner, whether the brain similarly composes a novel behavior is largely unknown. In the present study, I show that deep reinforcement learning agents learn to solve a novel composite task by additively combining representations of prelearned action values of constituent subtasks. Learning efficacy in the composite task was further augmented by the introduction of stochasticity in behavior during pretraining. These theoretical predictions were empirically tested in mice, where subtask pretraining enhanced learning of the composite task. Cortex-wide, two-photon calcium imaging revealed analogous neural representations of combined action values, with improved learning when the behavior variability was amplified. Together, these results suggest that the brain composes a novel behavior with a simple arithmetic operation of preacquired action-value representations with stochastic policies.
Collapse
Affiliation(s)
- Hiroshi Makino
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.
| |
Collapse
|
49
|
Grounding Mental Representations in a Virtual Multi-Level Functional Framework. J Cogn 2023; 6:6. [PMID: 36698786 PMCID: PMC9838229 DOI: 10.5334/joc.249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2022] [Accepted: 11/14/2022] [Indexed: 01/14/2023] Open
Abstract
According to the associative theory of learning, reactive behaviors described by stimulus-response pairs result in the progressive wiring of a plastic brain. In contrast, flexible behaviors are supposedly driven by neurologically grounded mental states that involve computations on informational contents. These theories appear complementary, but are generally opposed to each other. The former is favored by neuro-scientists who explore the low-level biological processes supporting cognition, and the later by cognitive psychologists who look for higher-level structures. This situation can be clarified through an analysis that independently defines abstract neurological and informational functionalities, and then relate them through a virtual interface. This framework is validated through a modeling of the first stage of Piaget's cognitive development theory, whose reported end experiments demonstrate the emergence of mental representations of object displacements. The neural correlates grounding this emergence are given in the isomorphic format of an associative memory. As a child's exploration of the world progresses, his mental models will eventually include representations of space, time and causality. Only then epistemological concepts, such as beliefs, will give rise to higher level mental representations in a possibly richer propositional format. This raises the question of which additional neurological functionalities, if any, would be required in order to include these extensions into a comprehensive grounded model. We relay previously expressed views, which in summary hypothesize that the ability to learn has evolved from associative reflexes and memories, to suggest that the functionality of associative memories could well provide the sufficient means for grounding cognitive capacities.
Collapse
|
50
|
Monaco JD, Hwang GM. Neurodynamical Computing at the Information Boundaries of Intelligent Systems. Cognit Comput 2022; 16:1-13. [PMID: 39129840 PMCID: PMC11306504 DOI: 10.1007/s12559-022-10081-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 11/15/2022] [Indexed: 12/28/2022]
Abstract
Artificial intelligence has not achieved defining features of biological intelligence despite models boasting more parameters than neurons in the human brain. In this perspective article, we synthesize historical approaches to understanding intelligent systems and argue that methodological and epistemic biases in these fields can be resolved by shifting away from cognitivist brain-as-computer theories and recognizing that brains exist within large, interdependent living systems. Integrating the dynamical systems view of cognition with the massive distributed feedback of perceptual control theory highlights a theoretical gap in our understanding of nonreductive neural mechanisms. Cell assemblies-properly conceived as reentrant dynamical flows and not merely as identified groups of neurons-may fill that gap by providing a minimal supraneuronal level of organization that establishes a neurodynamical base layer for computation. By considering information streams from physical embodiment and situational embedding, we discuss this computational base layer in terms of conserved oscillatory and structural properties of cortical-hippocampal networks. Our synthesis of embodied cognition, based in dynamical systems and perceptual control, aims to bypass the neurosymbolic stalemates that have arisen in artificial intelligence, cognitive science, and computational neuroscience.
Collapse
Affiliation(s)
- Joseph D. Monaco
- Dept of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD USA
| | - Grace M. Hwang
- Johns Hopkins University Applied Physics Laboratory, Laurel, MD USA
| |
Collapse
|