1
|
Philippe R, Janet R, Khalvati K, Rao RPN, Lee D, Dreher JC. Neurocomputational mechanisms involved in adaptation to fluctuating intentions of others. Nat Commun 2024; 15:3189. [PMID: 38609372 PMCID: PMC11014977 DOI: 10.1038/s41467-024-47491-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2022] [Accepted: 03/12/2024] [Indexed: 04/14/2024] Open
Abstract
Humans frequently interact with agents whose intentions can fluctuate between competition and cooperation over time. It is unclear how the brain adapts to fluctuating intentions of others when the nature of the interactions (to cooperate or compete) is not explicitly and truthfully signaled. Here, we use model-based fMRI and a task in which participants thought they were playing with another player. In fact, they played with an algorithm that alternated without signaling between cooperative and competitive strategies. We show that a neurocomputational mechanism with arbitration between competitive and cooperative experts outperforms other learning models in predicting choice behavior. At the brain level, the fMRI results show that the ventral striatum and ventromedial prefrontal cortex track the difference of reliability between these experts. When attributing competitive intentions, we find increased coupling between these regions and a network that distinguishes prediction errors related to competition and cooperation. These findings provide a neurocomputational account of how the brain arbitrates dynamically between cooperative and competitive intentions when making adaptive social decisions.
Collapse
Affiliation(s)
- Rémi Philippe
- CNRS-Institut des Sciences Cognitives Marc Jeannerod, UMR5229, Neuroeconomics, reward, and decision making laboratory, Lyon, France
- Université Claude Bernard Lyon 1, Lyon, France
| | - Rémi Janet
- CNRS-Institut des Sciences Cognitives Marc Jeannerod, UMR5229, Neuroeconomics, reward, and decision making laboratory, Lyon, France
- Université Claude Bernard Lyon 1, Lyon, France
| | - Koosha Khalvati
- Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA, USA
| | - Rajesh P N Rao
- Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA, USA
- Center for Neurotechnology, University of Washington, Seattle, WA, USA
| | - Daeyeol Lee
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, USA
- Kavli Discovery Neuroscience Institute, Johns Hopkins University, Baltimore, MD, USA
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, USA
| | - Jean-Claude Dreher
- CNRS-Institut des Sciences Cognitives Marc Jeannerod, UMR5229, Neuroeconomics, reward, and decision making laboratory, Lyon, France.
- Université Claude Bernard Lyon 1, Lyon, France.
| |
Collapse
|
2
|
Fouragnan EF, Hosking B, Cheung Y, Prakash B, Rushworth M, Sel A. Timing along the cardiac cycle modulates neural signals of reward-based learning. Nat Commun 2024; 15:2976. [PMID: 38582905 PMCID: PMC10998831 DOI: 10.1038/s41467-024-46921-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Accepted: 03/14/2024] [Indexed: 04/08/2024] Open
Abstract
Natural fluctuations in cardiac activity modulate brain activity associated with sensory stimuli, as well as perceptual decisions about low magnitude, near-threshold stimuli. However, little is known about the relationship between fluctuations in heart activity and other internal representations. Here we investigate whether the cardiac cycle relates to learning-related internal representations - absolute and signed prediction errors. We combined machine learning techniques with electroencephalography with both simple, direct indices of task performance and computational model-derived indices of learning. Our results demonstrate that just as people are more sensitive to low magnitude, near-threshold sensory stimuli in certain cardiac phases, so are they more sensitive to low magnitude absolute prediction errors in the same cycles. However, this occurs even when the low magnitude prediction errors are associated with clearly suprathreshold sensory events. In addition, participants exhibiting stronger differences in their prediction error representations between cardiac cycles exhibited higher learning rates and greater task accuracy.
Collapse
Affiliation(s)
- Elsa F Fouragnan
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK.
- Brain Research Imaging Centre (BRIC), Faculty of Health, University of Plymouth, Plymouth, PL6 8BU, UK.
- School of Psychology, Faculty of Health, University of Plymouth, Plymouth, PL4 8AA, UK.
| | - Billy Hosking
- Brain Research Imaging Centre (BRIC), Faculty of Health, University of Plymouth, Plymouth, PL6 8BU, UK
- School of Psychology, Faculty of Health, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Yin Cheung
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Brooke Prakash
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Matthew Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Alejandra Sel
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
- Centre for Brain Science, Department of Psychology, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, UK
- Essex ESNEFT Psychological Research Unit for Behaviour, Health and Wellbeing, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, UK
| |
Collapse
|
3
|
Attyé A, Renard F, Anglade V, Krainik A, Kahane P, Mansencal B, Coupé P, Calamante F. Data-driven normative values based on generative manifold learning for quantitative MRI. Sci Rep 2024; 14:7563. [PMID: 38555415 PMCID: PMC10981723 DOI: 10.1038/s41598-024-58141-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Accepted: 03/26/2024] [Indexed: 04/02/2024] Open
Abstract
In medicine, abnormalities in quantitative metrics such as the volume reduction of one brain region of an individual versus a control group are often provided as deviations from so-called normal values. These normative reference values are traditionally calculated based on the quantitative values from a control group, which can be adjusted for relevant clinical co-variables, such as age or sex. However, these average normative values do not take into account the globality of the available quantitative information. For example, quantitative analysis of T1-weighted magnetic resonance images based on anatomical structure segmentation frequently includes over 100 cerebral structures in the quantitative reports, and these tend to be analyzed separately. In this study, we propose a global approach to personalized normative values for each brain structure using an unsupervised Artificial Intelligence technique known as generative manifold learning. We test the potential benefit of these personalized normative values in comparison with the more traditional average normative values on a population of patients with drug-resistant epilepsy operated for focal cortical dysplasia, as well as on a supplementary healthy group and on patients with Alzheimer's disease.
Collapse
Affiliation(s)
| | | | - Vanina Anglade
- Department of Neuroradiology and MRI, SFR RMN Neurosciences, University Grenoble Alpes Hospital, Grenoble, France
| | - Alexandre Krainik
- Department of Neuroradiology and MRI, SFR RMN Neurosciences, University Grenoble Alpes Hospital, Grenoble, France
| | - Philippe Kahane
- Department of Neurology, University Grenoble Alpes Hospital, Grenoble, France
| | - Boris Mansencal
- CNRS, Univ. Bordeaux, Bordeaux INP, LABRI, UMR5800, 33400, Talence, France
| | - Pierrick Coupé
- CNRS, Univ. Bordeaux, Bordeaux INP, LABRI, UMR5800, 33400, Talence, France
| | - Fernando Calamante
- School of Biomedical Engineering, The University of Sydney, Sydney, NSW, 2006, Australia
- Sydney Imaging-The University of Sydney, Sydney, Australia
| |
Collapse
|
4
|
Kanwal S, Arif R, Ahmed S, Kabir M. A novel stacking-based predictor for accurate prediction of antimicrobial peptides. J Biomol Struct Dyn 2024:1-12. [PMID: 38500243 DOI: 10.1080/07391102.2024.2329298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 03/06/2024] [Indexed: 03/20/2024]
Abstract
Antimicrobial peptides (AMPs) are gaining acceptance and support as a chief antibiotic substitute since they boost human immunity. They retain a wide range of actions and have a low risk of developing resistance, which are critical properties to the pharmaceutical industry for drug discovery. Antibiotic sensitivity, however, is an issue that affects people all around the world and has the potential to one day lead to an epidemic. As cutting-edge therapeutic agents, AMPs are also expected to cure microbial infections. In order to produce tolerable drugs, it is crucial to understand the significance of the basic architecture of AMPs. Traditional laboratory methods are expensive and time-consuming for AMPs testing and detection. Currently, bioinformatics techniques are being successfully applied to the detection of AMPs. In this study, we have developed a novel STacking-based ensemble learning framework for AntiMicrobial Peptide (STAMP) prediction. First, we constructed 84 different baseline models by using 12 different feature encoding schemes and 7 popular machine learning algorithms. Second, these baseline models were trained and employed to create a new probabilistic feature vector. Finally, based on the feature selection strategy, we determined the optimal probabilistic feature vector, which was further utilized for the construction of our stacked model. Resultantly, the STAMP predictor achieved excellent performance during cross-validation with an accuracy and Matthew's correlation coefficient of 0.930 and 0.860, respectively. The corresponding metrics during the independent test were 0.710 and 0.464, respectively. Overall, STAMP achieved a more accurate and stable performance than the baseline models and significantly outperformed the existing predictors, demonstrating the effectiveness of our proposed hybrid framework. Furthermore, STAMP is expected to assist community-wide efforts in identifying AMPs and will contribute to the development of novel therapeutic methods and drug-design for immunity.Communicated by Ramaswamy H. Sarma.
Collapse
Affiliation(s)
- Sameera Kanwal
- School of Systems and Technology, University of Management and Technology, Lahore, Pakistan
| | - Roha Arif
- School of Systems and Technology, University of Management and Technology, Lahore, Pakistan
| | - Saeed Ahmed
- School of Systems and Technology, University of Management and Technology, Lahore, Pakistan
| | - Muhammad Kabir
- School of Systems and Technology, University of Management and Technology, Lahore, Pakistan
| |
Collapse
|
5
|
Qu C, Huang Y, Philippe R, Cai S, Derrington E, Moisan F, Shi M, Dreher JC. Transcranial direct current stimulation suggests a causal role of the medial prefrontal cortex in learning social hierarchy. Commun Biol 2024; 7:304. [PMID: 38461216 PMCID: PMC10924847 DOI: 10.1038/s42003-024-05976-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 02/27/2024] [Indexed: 03/11/2024] Open
Abstract
Social hierarchies can be inferred through observational learning of social relationships between individuals. Yet, little is known about the causal role of specific brain regions in learning hierarchies. Here, using transcranial direct current stimulation, we show a causal role of the medial prefrontal cortex (mPFC) in learning social versus non-social hierarchies. In a Training phase, participants acquired knowledge about social and non-social hierarchies by trial and error. During a Test phase, they were presented with two items from hierarchies that were never encountered together, requiring them to make transitive inferences. Anodal stimulation over mPFC impaired social compared with non-social hierarchy learning, and this modulation was influenced by the relative social rank of the members (higher or lower status). Anodal stimulation also impaired transitive inference making, but only during early blocks before learning was established. Together, these findings demonstrate a causal role of the mPFC in learning social ranks by observation.
Collapse
Affiliation(s)
- Chen Qu
- Center for Studies of Psychological Application, Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, China
| | - Yulong Huang
- Center for Studies of Psychological Application, Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, China
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Rémi Philippe
- Laboratory of Neuroeconomics, Institut des Sciences Cognitives Marc Jeannerod, CNRS, Lyon, France
- Université Claude Bernard Lyon 1, Lyon, France
| | - Shenggang Cai
- School of Economics and Management, South China Normal University, Guangzhou, China
- Key Lab for Behavioral Economic Science & Technology, South China Normal University, Guangzhou, China
| | - Edmund Derrington
- Laboratory of Neuroeconomics, Institut des Sciences Cognitives Marc Jeannerod, CNRS, Lyon, France
- Université Claude Bernard Lyon 1, Lyon, France
| | | | - Mengke Shi
- Center for Studies of Psychological Application, Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, China
| | - Jean-Claude Dreher
- Laboratory of Neuroeconomics, Institut des Sciences Cognitives Marc Jeannerod, CNRS, Lyon, France.
- Université Claude Bernard Lyon 1, Lyon, France.
| |
Collapse
|
6
|
Navas-Olive A, Rubio A, Abbaspoor S, Hoffman KL, de la Prida LM. A machine learning toolbox for the analysis of sharp-wave ripples reveals common waveform features across species. Commun Biol 2024; 7:211. [PMID: 38438533 PMCID: PMC10912113 DOI: 10.1038/s42003-024-05871-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Accepted: 01/29/2024] [Indexed: 03/06/2024] Open
Abstract
The study of sharp-wave ripples has advanced our understanding of memory function, and their alteration in neurological conditions such as epilepsy is considered a biomarker of dysfunction. Sharp-wave ripples exhibit diverse waveforms and properties that cannot be fully characterized by spectral methods alone. Here, we describe a toolbox of machine-learning models for automatic detection and analysis of these events. The machine-learning architectures, which resulted from a crowdsourced hackathon, are able to capture a wealth of ripple features recorded in the dorsal hippocampus of mice across awake and sleep conditions. When applied to data from the macaque hippocampus, these models are able to generalize detection and reveal shared properties across species. We hereby provide a user-friendly open-source toolbox for model use and extension, which can help to accelerate and standardize analysis of sharp-wave ripples, lowering the threshold for its adoption in biomedical applications.
Collapse
Affiliation(s)
| | | | - Saman Abbaspoor
- Psychological Sciences, Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
| | - Kari L Hoffman
- Psychological Sciences, Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
- Biomedical Engineering, Vanderbilt University, Nashville, TN, USA
| | | |
Collapse
|
7
|
Liebenow B, Jiang A, DiMarco EK, Sands LP, Moya-Mendez M, Laxton AW, Siddiqui MS, Ul Haq I, Kishida KT. Subjective feelings associated with expectations and rewards during risky decision-making in impulse control disorder. Sci Rep 2024; 14:4627. [PMID: 38438386 PMCID: PMC10912783 DOI: 10.1038/s41598-024-53076-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2022] [Accepted: 01/27/2024] [Indexed: 03/06/2024] Open
Abstract
Impulse Control Disorder (ICD) in Parkinson's disease is a behavioral addiction induced by dopaminergic therapies, but otherwise unclear etiology. The current study investigates the interaction of reward processing variables, dopaminergic therapy, and risky decision-making and subjective feelings in patients with versus without ICD. Patients with (n = 18) and without (n = 12) ICD performed a risky decision-making task both 'on' and 'off' standard-of-care dopaminergic therapies (the task was performed on 2 different days with the order of on and off visits randomized for each patient). During each trial of the task, participants choose between two options, a gamble or a certain reward, and reported how they felt about decision outcomes. Subjective feelings of 'pleasure' are differentially driven by expectations of possible outcomes in patients with, versus without ICD. While off medication, the influence of expectations about risky-decisions on subjective feelings is reduced in patients with ICD versus without ICD. While on medication, the influence of expected outcomes in patients with ICD versus without ICD becomes similar. Computational modeling of behavior supports the idea that latent decision-making factors drive subjective feelings in patients with Parkinson's disease and that ICD status is associated with a change in the relationship between factors associated with risky behavior and subjective feelings about the experienced outcomes. Our results also suggest that dopaminergic medications modulate the impact expectations have on the participants' subjective reports. Altogether our results suggest that expectations about risky decisions may be decoupled from subjective feelings in patients with ICD, and that dopaminergic medications may reengage these circuits and increase emotional reactivity in patients with ICD.
Collapse
Affiliation(s)
- Brittany Liebenow
- Neuroscience Graduate Program, Wake Forest School of Medicine, Winston-Salem, NC, USA
- Department of Translational Neuroscience, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Angela Jiang
- Department of Translational Neuroscience, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Emily K DiMarco
- Neuroscience Graduate Program, Wake Forest School of Medicine, Winston-Salem, NC, USA
- Department of Translational Neuroscience, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - L Paul Sands
- Neuroscience Graduate Program, Wake Forest School of Medicine, Winston-Salem, NC, USA
- Department of Translational Neuroscience, Wake Forest School of Medicine, Winston-Salem, NC, USA
- Fralin Biomedical Research Institute, Virginia Tech, Roanoke, VA, 24016, USA
| | | | - Adrian W Laxton
- Department of Neurosurgery, Wake Forest School of Medicine, Medical Center Boulevard, Winston-Salem, NC, 27157, USA
| | - Mustafa S Siddiqui
- Department of Neurology, Wake Forest School of Medicine, Medical Center Boulevard, Winston-Salem, NC, 27157, USA
| | - Ihtsham Ul Haq
- Department of Neurology, University of Miami Miller School of Medicine, Miami, FL, USA
| | - Kenneth T Kishida
- Neuroscience Graduate Program, Wake Forest School of Medicine, Winston-Salem, NC, USA.
- Department of Translational Neuroscience, Wake Forest School of Medicine, Winston-Salem, NC, USA.
- Department of Neurosurgery, Wake Forest School of Medicine, Medical Center Boulevard, Winston-Salem, NC, 27157, USA.
| |
Collapse
|
8
|
Leong F, Rahmani B, Psaltis D, Moser C, Ghezzi D. An actor-model framework for visual sensory encoding. Nat Commun 2024; 15:808. [PMID: 38280912 PMCID: PMC10821921 DOI: 10.1038/s41467-024-45105-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Accepted: 01/15/2024] [Indexed: 01/29/2024] Open
Abstract
A fundamental challenge in neuroengineering is determining a proper artificial input to a sensory system that yields the desired perception. In neuroprosthetics, this process is known as artificial sensory encoding, and it holds a crucial role in prosthetic devices restoring sensory perception in individuals with disabilities. For example, in visual prostheses, one key aspect of artificial image encoding is to downsample images captured by a camera to a size matching the number of inputs and resolution of the prosthesis. Here, we show that downsampling an image using the inherent computation of the retinal network yields better performance compared to learning-free downsampling methods. We have validated a learning-based approach (actor-model framework) that exploits the signal transformation from photoreceptors to retinal ganglion cells measured in explanted mouse retinas. The actor-model framework generates downsampled images eliciting a neuronal response in-silico and ex-vivo with higher neuronal reliability than the one produced by a learning-free approach. During the learning process, the actor network learns to optimize contrast and the kernel's weights. This methodological approach might guide future artificial image encoding strategies for visual prostheses. Ultimately, this framework could be applicable for encoding strategies in other sensory prostheses such as cochlear or limb.
Collapse
Affiliation(s)
- Franklin Leong
- Medtronic Chair in Neuroengineering, Center for Neuroprosthetics and Institute of Bioengineering, School of Engineering, École Polytechnique Fédérale de Lausanne, Geneva, Switzerland
| | - Babak Rahmani
- Laboratory of Applied Photonics Devices, Institute of Electrical and Micro Engineering, School of Engineering, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- Microsoft Research, Cambridge, UK
| | - Demetri Psaltis
- Optics Laboratory, Institute of Electrical and Micro Engineering, School of Engineering, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Christophe Moser
- Laboratory of Applied Photonics Devices, Institute of Electrical and Micro Engineering, School of Engineering, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Diego Ghezzi
- Medtronic Chair in Neuroengineering, Center for Neuroprosthetics and Institute of Bioengineering, School of Engineering, École Polytechnique Fédérale de Lausanne, Geneva, Switzerland.
- Ophthalmic and Neural Technologies Laboratory, Department of Ophthalmology, University of Lausanne, Hôpital ophtalmique Jules-Gonin, Fondation Asile des Aveugles, Lausanne, Switzerland.
| |
Collapse
|
9
|
Gorgan Mohammadi A, Ganjtabesh M. On computational models of theory of mind and the imitative reinforcement learning in spiking neural networks. Sci Rep 2024; 14:1945. [PMID: 38253595 PMCID: PMC10803361 DOI: 10.1038/s41598-024-52299-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2023] [Accepted: 01/16/2024] [Indexed: 01/24/2024] Open
Abstract
Theory of Mind is referred to the ability of inferring other's mental states, and it plays a crucial role in social cognition and learning. Biological evidences indicate that complex circuits are involved in this ability, including the mirror neuron system. The mirror neuron system influences imitation abilities and action understanding, leading to learn through observing others. To simulate this imitative learning behavior, a Theory-of-Mind-based Imitative Reinforcement Learning (ToM-based ImRL) framework is proposed. Employing the bio-inspired spiking neural networks and the mechanisms of the mirror neuron system, ToM-based ImRL is a bio-inspired computational model which enables an agent to effectively learn how to act in an interactive environment through observing an expert, inferring its goals, and imitating its behaviors. The aim of this paper is to review some computational attempts in modeling ToM and to explain the proposed ToM-based ImRL framework which is tested in the environment of River Raid game from Atari 2600 series.
Collapse
Affiliation(s)
- Ashena Gorgan Mohammadi
- Department of Computer Science, School of Mathematics, Statistics, and Computer Science, College of Science, University of Tehran, Tehran, Iran
| | - Mohammad Ganjtabesh
- Department of Computer Science, School of Mathematics, Statistics, and Computer Science, College of Science, University of Tehran, Tehran, Iran.
| |
Collapse
|
10
|
Pereg M, Hertz U, Ben-Artzi I, Shahar N. Disentangling the contribution of individual and social learning processes in human advice-taking behavior. NPJ Sci Learn 2024; 9:4. [PMID: 38245562 PMCID: PMC10799906 DOI: 10.1038/s41539-024-00214-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Accepted: 01/03/2024] [Indexed: 01/22/2024]
Abstract
The study of social learning examines how individuals learn from others by means of observation, imitation, or compliance with advice. However, it still remains largely unknown whether social learning processes have a distinct contribution to behavior, independent from non-social trial-and-error learning that often occurs simultaneously. 153 participants completed a reinforcement learning task, where they were asked to make choices to gain rewards. Advice from an artificial teacher was presented in 60% of the trials, allowing us to compare choice behavior with and without advice. Results showed a strong and reliable tendency to follow advice (test-retest reliability ~0.73). Computational modeling suggested a unique contribution of three distinct learning strategies: (a) individual learning (i.e., learning the value of actions, independent of advice), (b) informed advice-taking (i.e., learning the value of following advice), and (c) non-informed advice-taking (i.e., a constant bias to follow advice regardless of outcome history). Comparing artificial and empirical data provided specific behavioral regression signatures to both informed and non-informed advice taking processes. We discuss the theoretical implications of integrating internal and external information during the learning process.
Collapse
Affiliation(s)
- Maayan Pereg
- School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel.
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel.
- Minducate Center for the Science of Learning, Sagol School of Neuroscience, Tel Aviv, Israel.
- Department of Psychology, Achva Academic College, Arugot, Israel.
| | - Uri Hertz
- Department of Cognitive Sciences, University of Haifa, Haifa, Israel
- Institute of Information Processing and Decision Making, University of Haifa, Haifa, Israel
| | - Ido Ben-Artzi
- School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
- Minducate Center for the Science of Learning, Sagol School of Neuroscience, Tel Aviv, Israel
| | - Nitzan Shahar
- School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| |
Collapse
|
11
|
Belov V, Erwin-Grabner T, Aghajani M, Aleman A, Amod AR, Basgoze Z, Benedetti F, Besteher B, Bülow R, Ching CRK, Connolly CG, Cullen K, Davey CG, Dima D, Dols A, Evans JW, Fu CHY, Gonul AS, Gotlib IH, Grabe HJ, Groenewold N, Hamilton JP, Harrison BJ, Ho TC, Mwangi B, Jaworska N, Jahanshad N, Klimes-Dougan B, Koopowitz SM, Lancaster T, Li M, Linden DEJ, MacMaster FP, Mehler DMA, Melloni E, Mueller BA, Ojha A, Oudega ML, Penninx BWJH, Poletti S, Pomarol-Clotet E, Portella MJ, Pozzi E, Reneman L, Sacchet MD, Sämann PG, Schrantee A, Sim K, Soares JC, Stein DJ, Thomopoulos SI, Uyar-Demir A, van der Wee NJA, van der Werff SJA, Völzke H, Whittle S, Wittfeld K, Wright MJ, Wu MJ, Yang TT, Zarate C, Veltman DJ, Schmaal L, Thompson PM, Goya-Maldonado R. Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures. Sci Rep 2024; 14:1084. [PMID: 38212349 PMCID: PMC10784593 DOI: 10.1038/s41598-023-47934-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 11/19/2023] [Indexed: 01/13/2024] Open
Abstract
Machine learning (ML) techniques have gained popularity in the neuroimaging field due to their potential for classifying neuropsychiatric disorders. However, the diagnostic predictive power of the existing algorithms has been limited by small sample sizes, lack of representativeness, data leakage, and/or overfitting. Here, we overcome these limitations with the largest multi-site sample size to date (N = 5365) to provide a generalizable ML classification benchmark of major depressive disorder (MDD) using shallow linear and non-linear models. Leveraging brain measures from standardized ENIGMA analysis pipelines in FreeSurfer, we were able to classify MDD versus healthy controls (HC) with a balanced accuracy of around 62%. But after harmonizing the data, e.g., using ComBat, the balanced accuracy dropped to approximately 52%. Accuracy results close to random chance levels were also observed in stratified groups according to age of onset, antidepressant use, number of episodes and sex. Future studies incorporating higher dimensional brain imaging/phenotype features, and/or using more advanced machine and deep learning methods may yield more encouraging prospects.
Collapse
Affiliation(s)
- Vladimir Belov
- Laboratory of Systems Neuroscience and Imaging in Psychiatry (SNIP-Lab), Department of Psychiatry and Psychotherapy, University Medical Center Göttingen (UMG), Georg-August University, Von-Siebold-Str. 5, 37075, Göttingen, Germany
| | - Tracy Erwin-Grabner
- Laboratory of Systems Neuroscience and Imaging in Psychiatry (SNIP-Lab), Department of Psychiatry and Psychotherapy, University Medical Center Göttingen (UMG), Georg-August University, Von-Siebold-Str. 5, 37075, Göttingen, Germany
| | - Moji Aghajani
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- Institute of Education and Child Studies, Section Forensic Family and Youth Care, Leiden University, Leiden, The Netherlands
| | - Andre Aleman
- Department of Biomedical Sciences of Cells and Systems, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| | - Alyssa R Amod
- Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa
| | - Zeynep Basgoze
- Department of Psychiatry and Behavioral Science, University of Minnesota Medical School, Minneapolis, MN, USA
| | - Francesco Benedetti
- Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Bianca Besteher
- Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany
| | - Robin Bülow
- Institute for Radiology and Neuroradiology, University Medicine Greifswald, Greifswald, Germany
| | - Christopher R K Ching
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, USA
| | - Colm G Connolly
- Department of Biomedical Sciences, Florida State University, Tallahassee, FL, USA
| | - Kathryn Cullen
- Department of Psychiatry and Behavioral Science, University of Minnesota Medical School, Minneapolis, MN, USA
| | - Christopher G Davey
- Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne, Parkville, VIC, Australia
| | - Danai Dima
- Department of Psychology, School of Arts and Social Sciences, City, University of London, London, UK
- Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Annemiek Dols
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Jennifer W Evans
- Experimental Therapeutics and Pathophysiology Branch, National Institute for Mental Health, National Institutes of Health, Bethesda, MD, USA
| | - Cynthia H Y Fu
- School of Psychology, University of East London, London, UK
- Centre for Affective Disorders, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Ali Saffet Gonul
- SoCAT Lab, Department of Psychiatry, School of Medicine, Ege University, Izmir, Turkey
| | - Ian H Gotlib
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Hans J Grabe
- Department of Psychiatry and Psychotherapy, University Medicine Greifswald, Greifswald, Germany
| | - Nynke Groenewold
- Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa
| | - J Paul Hamilton
- Center for Social and Affective Neuroscience, Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden
- Center for Medical Imaging and Visualization, Linköping University, Linköping, Sweden
| | - Ben J Harrison
- Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne, Parkville, VIC, Australia
| | - Tiffany C Ho
- Department of Psychiatry and Behavioral Sciences, Division of Child and Adolescent Psychiatry, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, USA
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Benson Mwangi
- Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences, The University of Texas Health Science Center at Houston, Houston, TX, USA
- Center Of Excellence On Mood Disorders, Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences at McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Natalia Jaworska
- Department of Psychiatry, McGill University, Montreal, QC, Canada
| | - Neda Jahanshad
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, USA
| | | | | | - Thomas Lancaster
- Cardiff University Brain Research Imaging Center, Cardiff University, Cardiff, UK
- MRC Center for Neuropsychiatric Genetics and Genomics, Cardiff University, Cardiff, UK
| | - Meng Li
- Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany
| | - David E J Linden
- Cardiff University Brain Research Imaging Center, Cardiff University, Cardiff, UK
- MRC Center for Neuropsychiatric Genetics and Genomics, Cardiff University, Cardiff, UK
- Division of Psychological Medicine and Clinical Neurosciences, Cardiff University, Cardiff, UK
- School of Mental Health and Neuroscience, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Frank P MacMaster
- Departments of Psychiatry and Pediatrics, University of Calgary, Calgary, AB, Canada
| | - David M A Mehler
- Cardiff University Brain Research Imaging Center, Cardiff University, Cardiff, UK
- MRC Center for Neuropsychiatric Genetics and Genomics, Cardiff University, Cardiff, UK
- Department of Psychiatry, Psychotherapy and Psychosomatics, Medical School, RWTH Aachen University, Aachen, Germany
| | - Elisa Melloni
- Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Bryon A Mueller
- Department of Psychiatry and Behavioral Science, University of Minnesota Medical School, Minneapolis, MN, USA
| | - Amar Ojha
- Center for Neuroscience, University of Pittsburgh, Pittsburgh, PA, USA
- Center for Neural Basis of Cognition, University of Pittsburgh, Pittsburgh, PA, USA
| | - Mardien L Oudega
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Brenda W J H Penninx
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Sara Poletti
- Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Edith Pomarol-Clotet
- FIDMAG Germanes Hospitalàries Research Foundation, Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Barcelona, Catalonia, Spain
| | - Maria J Portella
- Sant Pau Mental Health Research Group, Institut de Recerca de L'Hospital de La Santa Creu I Sant Pau, Barcelona, Catalonia, Spain
| | - Elena Pozzi
- Centre for Youth Mental Health, The University of Melbourne, Parkville, VIC, Australia
- Orygen, Parkville, VIC, Australia
| | - Liesbeth Reneman
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | - Matthew D Sacchet
- Meditation Research Program, Department of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | | | - Anouk Schrantee
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | - Kang Sim
- West Region, Institute of Mental Health, Singapore, Singapore
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Jair C Soares
- Center Of Excellence On Mood Disorders, Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences at McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Dan J Stein
- SA MRC Research Unit on Risk and Resilience in Mental Disorders, Department of Psychiatry and Neuroscience Institute, University of Cape Town, Cape Town, South Africa
| | - Sophia I Thomopoulos
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, USA
| | - Aslihan Uyar-Demir
- SoCAT Lab, Department of Psychiatry, School of Medicine, Ege University, Izmir, Turkey
| | - Nic J A van der Wee
- Leiden Institute for Brain and Cognition, Leiden University Medical Center, Leiden, The Netherlands
| | - Steven J A van der Werff
- Leiden Institute for Brain and Cognition, Leiden University Medical Center, Leiden, The Netherlands
- Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands
| | - Henry Völzke
- Institute for Community Medicine, University Medicine Greifswald, Greifswald, Germany
| | - Sarah Whittle
- Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne and Melbourne Health, Melbourne, VIC, Australia
| | - Katharina Wittfeld
- Department of Psychiatry and Psychotherapy, University Medicine Greifswald, Greifswald, Germany
- German Center for Neurodegenerative Diseases (DZNE), Site Rostock/ Greifswald, Greifswald, Germany
| | - Margaret J Wright
- Queensland Brain Institute, The University of Queensland, Brisbane, QLD, Australia
- Centre for Advanced Imaging, The University of Queensland, Brisbane, QLD, Australia
| | - Mon-Ju Wu
- Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences, The University of Texas Health Science Center at Houston, Houston, TX, USA
- Center Of Excellence On Mood Disorders, Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences at McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Tony T Yang
- Department of Psychiatry and Behavioral Sciences, Division of Child and Adolescent Psychiatry, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, USA
| | - Carlos Zarate
- Section on the Neurobiology and Treatment of Mood Disorders, National Institute of Mental Health, Bethesda, MD, USA
| | - Dick J Veltman
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Lianne Schmaal
- Centre for Youth Mental Health, The University of Melbourne, Parkville, VIC, Australia
- Orygen, Parkville, VIC, Australia
| | - Paul M Thompson
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, USA
| | - Roberto Goya-Maldonado
- Laboratory of Systems Neuroscience and Imaging in Psychiatry (SNIP-Lab), Department of Psychiatry and Psychotherapy, University Medical Center Göttingen (UMG), Georg-August University, Von-Siebold-Str. 5, 37075, Göttingen, Germany.
| |
Collapse
|
12
|
Yi Y, Billor N, Ekstrom A, Zheng J. CW_ICA: an efficient dimensionality determination method for independent component analysis. Sci Rep 2024; 14:143. [PMID: 38167428 PMCID: PMC10762178 DOI: 10.1038/s41598-023-49355-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Accepted: 12/07/2023] [Indexed: 01/05/2024] Open
Abstract
Independent component analysis (ICA) is a widely used blind source separation method for signal pre-processing. The determination of the number of independent components (ICs) is crucial for achieving optimal performance, as an incorrect choice can result in either under-decomposition or over-decomposition. In this study, we propose a robust method to automatically determine the optimal number of ICs, named the column-wise independent component analysis (CW_ICA). CW_ICA divides the mixed signals into two blocks and applies ICA separately to each block. A quantitative measure, derived from the rank-based correlation matrix computed from the ICs of the two blocks, is utilized to determine the optimal number of ICs. The proposed method is validated and compared with the existing determination methods using simulation and scalp EEG data. The results demonstrate that CW_ICA is a reliable and robust approach for determining the optimal number of ICs. It offers computational efficiency and can be seamlessly integrated with different ICA methods.
Collapse
Affiliation(s)
- Yuyan Yi
- Department of Mathematics and Statistics, Auburn University, Auburn, AL, 36849, USA
| | - Nedret Billor
- Department of Mathematics and Statistics, Auburn University, Auburn, AL, 36849, USA
| | - Arne Ekstrom
- Department of Psychology and Evelyn McKnight Brain Institute, University of Arizona, Tucson, AZ, 85721, USA
| | - Jingyi Zheng
- Department of Mathematics and Statistics, Auburn University, Auburn, AL, 36849, USA.
| |
Collapse
|
13
|
Algermissen J, Swart JC, Scheeringa R, Cools R, den Ouden HEM. Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases. Nat Commun 2024; 15:19. [PMID: 38168089 PMCID: PMC10762147 DOI: 10.1038/s41467-023-44632-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 12/22/2023] [Indexed: 01/05/2024] Open
Abstract
Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.
Collapse
Affiliation(s)
- Johannes Algermissen
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Jennifer C Swart
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - René Scheeringa
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, Essen, Germany
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Department of Psychiatry, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| |
Collapse
|
14
|
Halvagal MS, Zenke F. The combination of Hebbian and predictive plasticity learns invariant object representations in deep sensory networks. Nat Neurosci 2023; 26:1906-1915. [PMID: 37828226 PMCID: PMC10620089 DOI: 10.1038/s41593-023-01460-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Accepted: 09/08/2023] [Indexed: 10/14/2023]
Abstract
Recognition of objects from sensory stimuli is essential for survival. To that end, sensory networks in the brain must form object representations invariant to stimulus changes, such as size, orientation and context. Although Hebbian plasticity is known to shape sensory networks, it fails to create invariant object representations in computational models, raising the question of how the brain achieves such processing. In the present study, we show that combining Hebbian plasticity with a predictive form of plasticity leads to invariant representations in deep neural network models. We derive a local learning rule that generalizes to spiking neural networks and naturally accounts for several experimentally observed properties of synaptic plasticity, including metaplasticity and spike-timing-dependent plasticity. Finally, our model accurately captures neuronal selectivity changes observed in the primate inferotemporal cortex in response to altered visual experience. Thus, we provide a plausible normative theory emphasizing the importance of predictive plasticity mechanisms for successful representational learning.
Collapse
Affiliation(s)
- Manu Srinath Halvagal
- Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland
- Faculty of Science, University of Basel, Basel, Switzerland
| | - Friedemann Zenke
- Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland.
- Faculty of Science, University of Basel, Basel, Switzerland.
| |
Collapse
|
15
|
Ting CC, Salem-Garcia N, Palminteri S, Engelmann JB, Lebreton M. Neural and computational underpinnings of biased confidence in human reinforcement learning. Nat Commun 2023; 14:6896. [PMID: 37898640 PMCID: PMC10613217 DOI: 10.1038/s41467-023-42589-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 10/16/2023] [Indexed: 10/30/2023] Open
Abstract
While navigating a fundamentally uncertain world, humans and animals constantly evaluate the probability of their decisions, actions or statements being correct. When explicitly elicited, these confidence estimates typically correlates positively with neural activity in a ventromedial-prefrontal (VMPFC) network and negatively in a dorsolateral and dorsomedial prefrontal network. Here, combining fMRI with a reinforcement-learning paradigm, we leverage the fact that humans are more confident in their choices when seeking gains than avoiding losses to reveal a functional dissociation: whereas the dorsal prefrontal network correlates negatively with a condition-specific confidence signal, the VMPFC network positively encodes task-wide confidence signal incorporating the valence-induced bias. Challenging dominant neuro-computational models, we found that decision-related VMPFC activity better correlates with confidence than with option-values inferred from reinforcement-learning models. Altogether, these results identify the VMPFC as a key node in the neuro-computational architecture that builds global feeling-of-confidence signals from latent decision variables and contextual biases during reinforcement-learning.
Collapse
Affiliation(s)
- Chih-Chung Ting
- General Psychology, Universität Hamburg, Von-Melle-Park 11, 20146, Hamburg, Germany.
- CREED, Amsterdam School of Economics (ASE), Universiteit van Amsterdam, Roetersstraat 11, 1018 WB, Amsterdam, the Netherlands.
| | - Nahuel Salem-Garcia
- Swiss Center for Affective Science, Faculty of Psychology and Educational Sciences, University of Geneva, Chem. des Mines 9, 1202, Genève, Switzerland
| | - Stefano Palminteri
- Département d'Études Cognitives, École Normale Supérieure, PSL Research University, 29 rue d'Ulm, 75230, Paris cedex 05, France
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale, 29 rue d'Ulm 75230, Paris cedex 05, France
| | - Jan B Engelmann
- CREED, Amsterdam School of Economics (ASE), Universiteit van Amsterdam, Roetersstraat 11, 1018 WB, Amsterdam, the Netherlands.
- The Tinbergen Institute, Gustav Mahlerplein 117, 1082 MS, Amsterdam, the Netherlands.
| | - Maël Lebreton
- Swiss Center for Affective Science, Faculty of Psychology and Educational Sciences, University of Geneva, Chem. des Mines 9, 1202, Genève, Switzerland.
- Economics of Human Behavior group, Paris-Jourdan Sciences Économiques UMR8545, Paris School of Economics, 48 Boulevard Jourdan, 75014, Paris, France.
| |
Collapse
|
16
|
Wirth C, Toth J, Arvaneh M. Bayesian learning from multi-way EEG feedback for robot navigation and target identification. Sci Rep 2023; 13:16925. [PMID: 37805540 PMCID: PMC10560278 DOI: 10.1038/s41598-023-44077-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 10/03/2023] [Indexed: 10/09/2023] Open
Abstract
Many brain-computer interfaces require a high mental workload. Recent research has shown that this could be greatly alleviated through machine learning, inferring user intentions via reactive brain responses. These signals are generated spontaneously while users merely observe assistive robots performing tasks. Using reactive brain signals, existing studies have addressed robot navigation tasks with a very limited number of potential target locations. Moreover, they use only binary, error-vs-correct classification of robot actions, leaving more detailed information unutilised. In this study a virtual robot had to navigate towards, and identify, target locations in both small and large grids, wherein any location could be the target. For the first time, we apply a system utilising detailed EEG information: 4-way classification of movements is performed, including specific information regarding when the target is reached. Additionally, we classify whether targets are correctly identified. Our proposed Bayesian strategy infers the most likely target location from the brain's responses. The experimental results show that our novel use of detailed information facilitates a more efficient and robust system than the state-of-the-art. Furthermore, unlike state-of-the-art approaches, we show scalability of our proposed approach: By tuning parameters appropriately, our strategy correctly identifies 98% of targets, even in large search spaces.
Collapse
Affiliation(s)
- Christopher Wirth
- Automatic Control and Systems Engineering Department, University of Sheffield, Sheffield, S1 4DT, UK.
- School of Medical Sciences, University of Manchester, Manchester, M13 9NT, UK.
| | - Jake Toth
- Automatic Control and Systems Engineering Department, University of Sheffield, Sheffield, S1 4DT, UK
| | - Mahnaz Arvaneh
- Automatic Control and Systems Engineering Department, University of Sheffield, Sheffield, S1 4DT, UK
| |
Collapse
|
17
|
Pan W, Zhao F, Zeng Y, Han B. Adaptive structure evolution and biologically plausible synaptic plasticity for recurrent spiking neural networks. Sci Rep 2023; 13:16924. [PMID: 37805632 PMCID: PMC10560283 DOI: 10.1038/s41598-023-43488-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 09/25/2023] [Indexed: 10/09/2023] Open
Abstract
The architecture design and multi-scale learning principles of the human brain that evolved over hundreds of millions of years are crucial to realizing human-like intelligence. Spiking neural network based Liquid State Machine (LSM) serves as a suitable architecture to study brain-inspired intelligence because of its brain-inspired structure and the potential for integrating multiple biological principles. Existing researches on LSM focus on different certain perspectives, including high-dimensional encoding or optimization of the liquid layer, network architecture search, and application to hardware devices. There is still a lack of in-depth inspiration from the learning and structural evolution mechanism of the brain. Considering these limitations, this paper presents a novel LSM learning model that integrates adaptive structural evolution and multi-scale biological learning rules. For structural evolution, an adaptive evolvable LSM model is developed to optimize the neural architecture design of liquid layer with separation property. For brain-inspired learning of LSM, we propose a dopamine-modulated Bienenstock-Cooper-Munros (DA-BCM) method that incorporates global long-term dopamine regulation and local trace-based BCM synaptic plasticity. Comparative experimental results on different decision-making tasks show that introducing structural evolution of the liquid layer, and the DA-BCM regulation of the liquid layer and the readout layer could improve the decision-making ability of LSM and flexibly adapt to rule reversal. This work is committed to exploring how evolution can help to design more appropriate network architectures and how multi-scale neuroplasticity principles coordinated to enable the optimization and learning of LSMs for relatively complex decision-making tasks.
Collapse
Affiliation(s)
- Wenxuan Pan
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Feifei Zhao
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Yi Zeng
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China.
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.
- School of Future Technology, University of Chinese Academy of Sciences, Beijing, China.
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China.
| | - Bing Han
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
18
|
Jager F. An open dataset with electrohysterogram records of pregnancies ending in induced and cesarean section delivery. Sci Data 2023; 10:669. [PMID: 37783671 PMCID: PMC10545725 DOI: 10.1038/s41597-023-02581-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Accepted: 09/20/2023] [Indexed: 10/04/2023] Open
Abstract
The existing non-invasive automated preterm birth prediction methods rely on the use of uterine electrohysterogram (EHG) records coming from spontaneous preterm and term deliveries, and are indifferent to term induced and cesarean section deliveries. In order to enhance current publicly available pool of term EHG records, we developed a new EHG dataset, Induced Cesarean EHG DataSet (ICEHG DS), containing 126 30-minute EHG records, recorded early (23rd week), and/or later (31st week) during pregnancy, of those pregnancies that were expected to end in spontaneous term delivery, but ended in induced or cesarean section delivery. The records were collected at the University Medical Center Ljubljana, Ljubljana, Slovenia. The dataset includes 38 and 43, early and later, induced; 11 and 8, early and later, cesarean; and 13 and 13, early and later, induced and cesarean EHG records. This dataset enables better understanding of the underlying physiological mechanisms involved during pregnancies ending in induced and cesarean deliveries, and provides a robust and more realistic assessment of the performance of automated preterm birth prediction methods.
Collapse
Affiliation(s)
- Franc Jager
- Faculty of Computer and Information Science, University of Ljubljana, Večna pot 113, 1000, Ljubljana, Slovenia.
| |
Collapse
|
19
|
Ostmeier S, Axelrod B, Verhaaren BFJ, Christensen S, Mahammedi A, Liu Y, Pulli B, Li LJ, Zaharchuk G, Heit JJ. Non-inferiority of deep learning ischemic stroke segmentation on non-contrast CT within 16-hours compared to expert neuroradiologists. Sci Rep 2023; 13:16153. [PMID: 37752162 PMCID: PMC10522706 DOI: 10.1038/s41598-023-42961-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Accepted: 09/17/2023] [Indexed: 09/28/2023] Open
Abstract
We determined if a convolutional neural network (CNN) deep learning model can accurately segment acute ischemic changes on non-contrast CT compared to neuroradiologists. Non-contrast CT (NCCT) examinations from 232 acute ischemic stroke patients who were enrolled in the DEFUSE 3 trial were included in this study. Three experienced neuroradiologists independently segmented hypodensity that reflected the ischemic core on each scan. The neuroradiologist with the most experience (expert A) served as the ground truth for deep learning model training. Two additional neuroradiologists' (experts B and C) segmentations were used for data testing. The 232 studies were randomly split into training and test sets. The training set was further randomly divided into 5 folds with training and validation sets. A 3-dimensional CNN architecture was trained and optimized to predict the segmentations of expert A from NCCT. The performance of the model was assessed using a set of volume, overlap, and distance metrics using non-inferiority thresholds of 20%, 3 ml, and 3 mm, respectively. The optimized model trained on expert A was compared to test experts B and C. We used a one-sided Wilcoxon signed-rank test to test for the non-inferiority of the model-expert compared to the inter-expert agreement. The final model performance for the ischemic core segmentation task reached a performance of 0.46 ± 0.09 Surface Dice at Tolerance 5mm and 0.47 ± 0.13 Dice when trained on expert A. Compared to the two test neuroradiologists the model-expert agreement was non-inferior to the inter-expert agreement, [Formula: see text]. The before, CNN accurately delineates the hypodense ischemic core on NCCT in acute ischemic stroke patients with an accuracy comparable to neuroradiologists.
Collapse
Affiliation(s)
| | - Brian Axelrod
- Department of Computer Science, Stanford University, Stanford, USA
| | | | | | | | | | | | - Li-Jia Li
- Stanford School of Medicine, Stanford, USA
| | | | | |
Collapse
|
20
|
Kotloski RJ. A machine learning approach to seizure detection in a rat model of post-traumatic epilepsy. Sci Rep 2023; 13:15807. [PMID: 37737238 PMCID: PMC10517002 DOI: 10.1038/s41598-023-40628-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Accepted: 08/14/2023] [Indexed: 09/23/2023] Open
Abstract
Epilepsy is a common neurologic condition frequently investigated using rodent models, with seizures identified by electroencephalography (EEG). Given technological advances, large datasets of EEG are widespread and amenable to machine learning approaches for identification of seizures. While such approaches have been explored for human EEGs, machine learning approaches to identifying seizures in rodent EEG are limited. We utilized a predesigned deep convolutional neural network (DCNN), GoogLeNet, to classify images for seizure identification. Training images were generated through multiplexing spectral content (scalograms), kurtosis, and entropy for two-second EEG segments. Over 2200 h of EEG data were scored for the presence of seizures, with 95.6% of seizures identified by the DCNN and a false positive rate of 34.2% (1.52/h), as compared to visual scoring. Multiplexed images were superior to scalograms alone (scalogram-kurtosis-entropy 0.956 ± 0.010, scalogram 0.890 ± 0.028, t(7) = 3.54, p < 0.01) and a DCNN trained specifically for the individual animal was superior to using DCNNs across animals (intra-animal 0.960 ± 0.0094, inter-animal 0.811 ± 0.015, t(30) = 5.54, p < 0.01). For this dataset the DCNN approach is superior to a previously described algorithm utilizing longer local line lengths, calculated from wavelet-decomposition of EEG, to identify seizures. We demonstrate the novel use of a predesigned DCNN constructed to classify images, utilizing multiplexed images of EEG spectral content, kurtosis, and entropy, to rapidly and objectively identifies seizures in a large dataset of rat EEG with high sensitivity.
Collapse
Affiliation(s)
- Robert J Kotloski
- Department of Neurology, William S Middleton Memorial Veterans Hospital, Madison, WI, 53705, USA.
- Department of Neurology, University of Wisconsin School of Medicine and Public Health, 1685 Highland Avenue, Madison, WI, 53705-2281, USA.
| |
Collapse
|
21
|
Zhang Y, He G, Ma L, Liu X, Hjorth JJJ, Kozlov A, He Y, Zhang S, Kotaleski JH, Tian Y, Grillner S, Du K, Huang T. A GPU-based computational framework that bridges neuron simulation and artificial intelligence. Nat Commun 2023; 14:5798. [PMID: 37723170 PMCID: PMC10507119 DOI: 10.1038/s41467-023-41553-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Accepted: 09/08/2023] [Indexed: 09/20/2023] Open
Abstract
Biophysically detailed multi-compartment models are powerful tools to explore computational principles of the brain and also serve as a theoretical framework to generate algorithms for artificial intelligence (AI) systems. However, the expensive computational cost severely limits the applications in both the neuroscience and AI fields. The major bottleneck during simulating detailed compartment models is the ability of a simulator to solve large systems of linear equations. Here, we present a novel Dendritic Hierarchical Scheduling (DHS) method to markedly accelerate such a process. We theoretically prove that the DHS implementation is computationally optimal and accurate. This GPU-based method performs with 2-3 orders of magnitude higher speed than that of the classic serial Hines method in the conventional CPU platform. We build a DeepDendrite framework, which integrates the DHS method and the GPU computing engine of the NEURON simulator and demonstrate applications of DeepDendrite in neuroscience tasks. We investigate how spatial patterns of spine inputs affect neuronal excitability in a detailed human pyramidal neuron model with 25,000 spines. Furthermore, we provide a brief discussion on the potential of DeepDendrite for AI, specifically highlighting its ability to enable the efficient training of biophysically detailed models in typical image classification tasks.
Collapse
Affiliation(s)
- Yichen Zhang
- National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China
| | - Gan He
- National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China
| | - Lei Ma
- National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China
- Beijing Academy of Artificial Intelligence (BAAI), Beijing, 100084, China
| | - Xiaofei Liu
- National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China
- School of Information Science and Engineering, Yunnan University, Kunming, 650500, China
| | - J J Johannes Hjorth
- Science for Life Laboratory, School of Electrical Engineering and Computer Science, Royal Institute of Technology KTH, Stockholm, SE-10044, Sweden
| | - Alexander Kozlov
- Science for Life Laboratory, School of Electrical Engineering and Computer Science, Royal Institute of Technology KTH, Stockholm, SE-10044, Sweden
- Department of Neuroscience, Karolinska Institute, Stockholm, SE-17165, Sweden
| | - Yutao He
- National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China
| | - Shenjian Zhang
- National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China
| | - Jeanette Hellgren Kotaleski
- Science for Life Laboratory, School of Electrical Engineering and Computer Science, Royal Institute of Technology KTH, Stockholm, SE-10044, Sweden
- Department of Neuroscience, Karolinska Institute, Stockholm, SE-17165, Sweden
| | - Yonghong Tian
- National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China
- School of Electrical and Computer Engineering, Shenzhen Graduate School, Peking University, Shenzhen, 518055, China
| | - Sten Grillner
- Department of Neuroscience, Karolinska Institute, Stockholm, SE-17165, Sweden
| | - Kai Du
- Institute for Artificial Intelligence, Peking University, Beijing, 100871, China.
| | - Tiejun Huang
- National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China
- Beijing Academy of Artificial Intelligence (BAAI), Beijing, 100084, China
- Institute for Artificial Intelligence, Peking University, Beijing, 100871, China
| |
Collapse
|
22
|
Pauli R, Brazil IA, Kohls G, Klein-Flügge MC, Rogers JC, Dikeos D, Dochnal R, Fairchild G, Fernández-Rivas A, Herpertz-Dahlmann B, Hervas A, Konrad K, Popma A, Stadler C, Freitag CM, De Brito SA, Lockwood PL. Action initiation and punishment learning differ from childhood to adolescence while reward learning remains stable. Nat Commun 2023; 14:5689. [PMID: 37709750 PMCID: PMC10502052 DOI: 10.1038/s41467-023-41124-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 08/24/2023] [Indexed: 09/16/2023] Open
Abstract
Theoretical and empirical accounts suggest that adolescence is associated with heightened reward learning and impulsivity. Experimental tasks and computational models that can dissociate reward learning from the tendency to initiate actions impulsively (action initiation bias) are thus critical to characterise the mechanisms that drive developmental differences. However, existing work has rarely quantified both learning ability and action initiation, or it has relied on small samples. Here, using computational modelling of a learning task collected from a large sample (N = 742, 9-18 years, 11 countries), we test differences in reward and punishment learning and action initiation from childhood to adolescence. Computational modelling reveals that whilst punishment learning rates increase with age, reward learning remains stable. In parallel, action initiation biases decrease with age. Results are similar when considering pubertal stage instead of chronological age. We conclude that heightened reward responsivity in adolescence can reflect differences in action initiation rather than enhanced reward learning.
Collapse
Affiliation(s)
- Ruth Pauli
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
| | - Inti A Brazil
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Gregor Kohls
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
- Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU, Dresden, Germany
| | - Miriam C Klein-Flügge
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Jack C Rogers
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Dimitris Dikeos
- Department of Psychiatry, Medical School, National and Kapodistrian University of Athens, Athens, Greece
| | - Roberta Dochnal
- Faculty of Medicine, Child and Adolescent Psychiatry, Department of the Child Health Center, Szeged University, Szeged, Hungary
| | | | | | - Beate Herpertz-Dahlmann
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
| | - Amaia Hervas
- University Hospital Mutua Terrassa, Barcelona, Spain
| | - Kerstin Konrad
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
- JARA-Brain Institute II, Molecular Neuroscience and Neuroimaging, RWTH Aachen and Research Centre Jülich, Jülich, Germany
| | - Arne Popma
- Department of Child and Adolescent Psychiatry, VU University Medical Center, Amsterdam, Netherlands
| | - Christina Stadler
- Department of Child and Adolescent Psychiatry, Psychiatric University Hospital, University of Basel, Basel, Switzerland
| | - Christine M Freitag
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital Frankfurt, Goethe University, Frankfurt am Main, Germany
| | - Stephane A De Brito
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Patricia L Lockwood
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK.
| |
Collapse
|
23
|
Eggl MF, Chater TE, Petkovic J, Goda Y, Tchumatchenko T. Linking spontaneous and stimulated spine dynamics. Commun Biol 2023; 6:930. [PMID: 37696988 PMCID: PMC10495434 DOI: 10.1038/s42003-023-05303-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 08/29/2023] [Indexed: 09/13/2023] Open
Abstract
Our brains continuously acquire and store memories through synaptic plasticity. However, spontaneous synaptic changes can also occur and pose a challenge for maintaining stable memories. Despite fluctuations in synapse size, recent studies have shown that key population-level synaptic properties remain stable over time. This raises the question of how local synaptic plasticity affects the global population-level synaptic size distribution and whether individual synapses undergoing plasticity escape the stable distribution to encode specific memories. To address this question, we (i) studied spontaneously evolving spines and (ii) induced synaptic potentiation at selected sites while observing the spine distribution pre- and post-stimulation. We designed a stochastic model to describe how the current size of a synapse affects its future size under baseline and stimulation conditions and how these local effects give rise to population-level synaptic shifts. Our study offers insights into how seemingly spontaneous synaptic fluctuations and local plasticity both contribute to population-level synaptic dynamics.
Collapse
Affiliation(s)
- Maximilian F Eggl
- University of Mainz Medical Center, Anselm-Franz-von-Bentzel-Weg 3, 55128, Mainz, Germany
| | - Thomas E Chater
- Laboratory for Synaptic Plasticity and Connectivity, RIKEN Center for Brain Science, Wako-shi, Saitama, Japan
- Department of Physiology, Keio University School of Medicine, Tokyo, Japan
| | - Janko Petkovic
- University of Mainz Medical Center, Anselm-Franz-von-Bentzel-Weg 3, 55128, Mainz, Germany
| | - Yukiko Goda
- Laboratory for Synaptic Plasticity and Connectivity, RIKEN Center for Brain Science, Wako-shi, Saitama, Japan
- Synapse Biology Unit, Okinawa Institute of Science and Technology Graduate University, Onna-son, Kunigami-gun, Okinawa, Japan
| | - Tatjana Tchumatchenko
- University of Mainz Medical Center, Anselm-Franz-von-Bentzel-Weg 3, 55128, Mainz, Germany.
- Institute of Experimental Epileptology and Cognition Research, University of Bonn Medical Center, Venusberg-Campus 1, 53127, Bonn, Germany.
| |
Collapse
|
24
|
Constant M, Pereira M, Faivre N, Filevich E. Prior information differentially affects discrimination decisions and subjective confidence reports. Nat Commun 2023; 14:5473. [PMID: 37673881 PMCID: PMC10482953 DOI: 10.1038/s41467-023-41112-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Accepted: 08/22/2023] [Indexed: 09/08/2023] Open
Abstract
According to Bayesian models, both decisions and confidence are based on the same precision-weighted integration of prior expectations ("priors") and incoming information ("likelihoods"). This assumes that priors are integrated optimally and equally in decisions and confidence, which has not been tested. In three experiments, we quantify how priors inform decisions and confidence. With a dual-decision task we create pairs of conditions that are matched in posterior information, but differ on whether the prior or likelihood is more informative. We find that priors are underweighted in discrimination decisions, but are less underweighted in confidence about those decisions, and this is not due to differences in processing time. The same patterns remain with exogenous probabilistic cues as priors. With a Bayesian model we quantify the weighting parameters for the prior at both levels, and find converging evidence that priors are more optimally used in explicit confidence, even when underused in decisions.
Collapse
Affiliation(s)
- Marika Constant
- Humboldt-Universität zu Berlin, Faculty of Life Sciences, Department of Psychology, Unter den Linden 6, 10099, Berlin, Germany.
- Bernstein Center for Computational Neuroscience Berlin, Philippstraße 13 Haus 6, 10115, Berlin, Germany.
- Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Luisenstraße 56, 10115, Berlin, Germany.
| | - Michael Pereira
- , Université Grenoble Alpes, Université Savoie Mont Blanc, CNRS, LPNC, 38000, Grenoble, France
| | - Nathan Faivre
- , Université Grenoble Alpes, Université Savoie Mont Blanc, CNRS, LPNC, 38000, Grenoble, France
| | - Elisa Filevich
- Humboldt-Universität zu Berlin, Faculty of Life Sciences, Department of Psychology, Unter den Linden 6, 10099, Berlin, Germany
- Bernstein Center for Computational Neuroscience Berlin, Philippstraße 13 Haus 6, 10115, Berlin, Germany
- Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Luisenstraße 56, 10115, Berlin, Germany
- Hector Institute for Education Sciences & Psychology, University of Tübingen, Europastraße 6, 72072, Tübingen, Germany
| |
Collapse
|
25
|
Taylor J, Kriegeskorte N. Extracting and visualizing hidden activations and computational graphs of PyTorch models with TorchLens. Sci Rep 2023; 13:14375. [PMID: 37658079 PMCID: PMC10474256 DOI: 10.1038/s41598-023-40807-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Accepted: 08/16/2023] [Indexed: 09/03/2023] Open
Abstract
Deep neural network models (DNNs) are essential to modern AI and provide powerful models of information processing in biological neural networks. Researchers in both neuroscience and engineering are pursuing a better understanding of the internal representations and operations that undergird the successes and failures of DNNs. Neuroscientists additionally evaluate DNNs as models of brain computation by comparing their internal representations to those found in brains. It is therefore essential to have a method to easily and exhaustively extract and characterize the results of the internal operations of any DNN. Many models are implemented in PyTorch, the leading framework for building DNN models. Here we introduce TorchLens, a new open-source Python package for extracting and characterizing hidden-layer activations in PyTorch models. Uniquely among existing approaches to this problem, TorchLens has the following features: (1) it exhaustively extracts the results of all intermediate operations, not just those associated with PyTorch module objects, yielding a full record of every step in the model's computational graph, (2) it provides an intuitive visualization of the model's complete computational graph along with metadata about each computational step in a model's forward pass for further analysis, (3) it contains a built-in validation procedure to algorithmically verify the accuracy of all saved hidden-layer activations, and (4) the approach it uses can be automatically applied to any PyTorch model with no modifications, including models with conditional (if-then) logic in their forward pass, recurrent models, branching models where layer outputs are fed into multiple subsequent layers in parallel, and models with internally generated tensors (e.g., injections of noise). Furthermore, using TorchLens requires minimal additional code, making it easy to incorporate into existing pipelines for model development and analysis, and useful as a pedagogical aid when teaching deep learning concepts. We hope this contribution will help researchers in AI and neuroscience understand the internal representations of DNNs.
Collapse
Affiliation(s)
- JohnMark Taylor
- Zuckerman Mind Brain Behavior Institute, Columbia University, 3227 Broadway, New York, NY, 10027, USA.
| | - Nikolaus Kriegeskorte
- Zuckerman Mind Brain Behavior Institute, Columbia University, 3227 Broadway, New York, NY, 10027, USA
| |
Collapse
|
26
|
Saponati M, Vinck M. Sequence anticipation and spike-timing-dependent plasticity emerge from a predictive learning rule. Nat Commun 2023; 14:4985. [PMID: 37604825 PMCID: PMC10442404 DOI: 10.1038/s41467-023-40651-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2021] [Accepted: 08/03/2023] [Indexed: 08/23/2023] Open
Abstract
Intelligent behavior depends on the brain's ability to anticipate future events. However, the learning rules that enable neurons to predict and fire ahead of sensory inputs remain largely unknown. We propose a plasticity rule based on predictive processing, where the neuron learns a low-rank model of the synaptic input dynamics in its membrane potential. Neurons thereby amplify those synapses that maximally predict other synaptic inputs based on their temporal relations, which provide a solution to an optimization problem that can be implemented at the single-neuron level using only local information. Consequently, neurons learn sequences over long timescales and shift their spikes towards the first inputs in a sequence. We show that this mechanism can explain the development of anticipatory signalling and recall in a recurrent network. Furthermore, we demonstrate that the learning rule gives rise to several experimentally observed STDP (spike-timing-dependent plasticity) mechanisms. These findings suggest prediction as a guiding principle to orchestrate learning and synaptic plasticity in single neurons.
Collapse
Affiliation(s)
- Matteo Saponati
- Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck Society, 60528, Frankfurt Am Main, Germany.
- IMPRS for Neural Circuits, Max-Planck Institute for Brain Research, 60438, Frankfurt Am Main, Germany.
- Donders Centre for Neuroscience, Department of Neuroinformatics, Radboud University, 6525, Nijmegen, The Netherlands.
| | - Martin Vinck
- Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck Society, 60528, Frankfurt Am Main, Germany.
- Donders Centre for Neuroscience, Department of Neuroinformatics, Radboud University, 6525, Nijmegen, The Netherlands.
| |
Collapse
|
27
|
Maes A, Barahona M, Clopath C. Long- and short-term history effects in a spiking network model of statistical learning. Sci Rep 2023; 13:12939. [PMID: 37558704 PMCID: PMC10412617 DOI: 10.1038/s41598-023-39108-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Accepted: 07/20/2023] [Indexed: 08/11/2023] Open
Abstract
The statistical structure of the environment is often important when making decisions. There are multiple theories of how the brain represents statistical structure. One such theory states that neural activity spontaneously samples from probability distributions. In other words, the network spends more time in states which encode high-probability stimuli. Starting from the neural assembly, increasingly thought of to be the building block for computation in the brain, we focus on how arbitrary prior knowledge about the external world can both be learned and spontaneously recollected. We present a model based upon learning the inverse of the cumulative distribution function. Learning is entirely unsupervised using biophysical neurons and biologically plausible learning rules. We show how this prior knowledge can then be accessed to compute expectations and signal surprise in downstream networks. Sensory history effects emerge from the model as a consequence of ongoing learning.
Collapse
Affiliation(s)
- Amadeus Maes
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, USA.
- Department of Bioengineering, Imperial College London, London, UK.
| | | | - Claudia Clopath
- Department of Bioengineering, Imperial College London, London, UK
| |
Collapse
|
28
|
Hanssen R, Rigoux L, Kuzmanovic B, Iglesias S, Kretschmer AC, Schlamann M, Albus K, Edwin Thanarajah S, Sitnikow T, Melzer C, Cornely OA, Brüning JC, Tittgemeyer M. Liraglutide restores impaired associative learning in individuals with obesity. Nat Metab 2023; 5:1352-1363. [PMID: 37592007 PMCID: PMC10447249 DOI: 10.1038/s42255-023-00859-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 07/07/2023] [Indexed: 08/19/2023]
Abstract
Survival under selective pressure is driven by the ability of our brain to use sensory information to our advantage to control physiological needs. To that end, neural circuits receive and integrate external environmental cues and internal metabolic signals to form learned sensory associations, consequently motivating and adapting our behaviour. The dopaminergic midbrain plays a crucial role in learning adaptive behaviour and is particularly sensitive to peripheral metabolic signals, including intestinal peptides, such as glucagon-like peptide 1 (GLP-1). In a single-blinded, randomized, controlled, crossover basic human functional magnetic resonance imaging study relying on a computational model of the adaptive learning process underlying behavioural responses, we show that adaptive learning is reduced when metabolic sensing is impaired in obesity, as indexed by reduced insulin sensitivity (participants: N = 30 with normal insulin sensitivity; N = 24 with impaired insulin sensitivity). Treatment with the GLP-1 receptor agonist liraglutide normalizes impaired learning of sensory associations in men and women with obesity. Collectively, our findings reveal that GLP-1 receptor activation modulates associative learning in people with obesity via its central effects within the mesoaccumbens pathway. These findings provide evidence for how metabolic signals can act as neuromodulators to adapt our behaviour to our body's internal state and how GLP-1 receptor agonists work in clinics.
Collapse
Affiliation(s)
- Ruth Hanssen
- Max Planck Institute for Metabolism Research, Cologne, Germany
- Faculty of Medicine and University Hospital Cologne, Policlinic for Endocrinology, Diabetology and Preventive Medicine (PEPD), University of Cologne, Cologne, Germany
| | - Lionel Rigoux
- Max Planck Institute for Metabolism Research, Cologne, Germany
| | | | - Sandra Iglesias
- Translational Neuromodeling Unit, Institute for Biomedical Engineering, University of Zurich and Swiss Federal Institute of Technology, Zurich, Switzerland
| | - Alina C Kretschmer
- Faculty of Medicine and University Hospital Cologne, Department I of Internal Medicine, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf (CIO ABCD) and Excellence Center for Medical Mycology (ECMM), University of Cologne, Cologne, Germany
| | - Marc Schlamann
- Faculty of Medicine and University Hospital Cologne, Institute for Diagnostic and Interventional Radiology, University of Cologne, Cologne, Germany
| | - Kerstin Albus
- Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, Germany
| | - Sharmili Edwin Thanarajah
- Max Planck Institute for Metabolism Research, Cologne, Germany
- Department of Psychiatry, Psychosomatic Medicine and Psychotherapy, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Tamara Sitnikow
- Faculty of Medicine and University Hospital Cologne, Policlinic for Endocrinology, Diabetology and Preventive Medicine (PEPD), University of Cologne, Cologne, Germany
| | - Corina Melzer
- Max Planck Institute for Metabolism Research, Cologne, Germany
| | - Oliver A Cornely
- Faculty of Medicine and University Hospital Cologne, Department I of Internal Medicine, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf (CIO ABCD) and Excellence Center for Medical Mycology (ECMM), University of Cologne, Cologne, Germany
- Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, Germany
- German Centre for Infection Research (DZIF), Partner Site Bonn-Cologne, Cologne, Germany
- Faculty of Medicine and University Hospital Cologne, Clinical Trials Centre Cologne (ZKS Köln), University of Cologne, Cologne, Germany
| | - Jens C Brüning
- Max Planck Institute for Metabolism Research, Cologne, Germany
- Faculty of Medicine and University Hospital Cologne, Policlinic for Endocrinology, Diabetology and Preventive Medicine (PEPD), University of Cologne, Cologne, Germany
- Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, Germany
| | - Marc Tittgemeyer
- Max Planck Institute for Metabolism Research, Cologne, Germany.
- Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, Germany.
| |
Collapse
|
29
|
Zika O, Wiech K, Reinecke A, Browning M, Schuck NW. Trait anxiety is associated with hidden state inference during aversive reversal learning. Nat Commun 2023; 14:4203. [PMID: 37452030 PMCID: PMC10349120 DOI: 10.1038/s41467-023-39825-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Accepted: 06/29/2023] [Indexed: 07/18/2023] Open
Abstract
Updating beliefs in changing environments can be driven by gradually adapting expectations or by relying on inferred hidden states (i.e. contexts), and changes therein. Previous work suggests that increased reliance on context could underly fear relapse phenomena that hinder clinical treatment of anxiety disorders. We test whether trait anxiety variations in a healthy population influence how much individuals rely on hidden-state inference. In a Pavlovian learning task, participants observed cues that predicted an upcoming electrical shock with repeatedly changing probability, and were asked to provide expectancy ratings on every trial. We show that trait anxiety is associated with steeper expectation switches after contingency reversals and reduced oddball learning. Furthermore, trait anxiety is related to better fit of a state inference, compared to a gradual learning, model when contingency changes are large. Our findings support previous work suggesting hidden-state inference as a mechanism behind anxiety-related to fear relapse phenomena.
Collapse
Affiliation(s)
- Ondrej Zika
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany.
- Max Planck UCL Centre for Computational Psychiatry and Aging Research, Berlin, Germany.
| | - Katja Wiech
- Wellcome Centre for Integrative Neuroimaging (WIN), Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Andrea Reinecke
- Department of Psychiatry, University of Oxford, Oxford, UK
- Oxford Health NHS Trust, Warneford Hospital, Oxford, UK
| | - Michael Browning
- Department of Psychiatry, University of Oxford, Oxford, UK
- Oxford Health NHS Trust, Warneford Hospital, Oxford, UK
| | - Nicolas W Schuck
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany.
- Max Planck UCL Centre for Computational Psychiatry and Aging Research, Berlin, Germany.
- Institute of Psychology, Universität Hamburg, Hamburg, Germany.
| |
Collapse
|
30
|
Sugiyama T, Schweighofer N, Izawa J. Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance. Nat Commun 2023; 14:3988. [PMID: 37422476 PMCID: PMC10329706 DOI: 10.1038/s41467-023-39536-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Accepted: 06/16/2023] [Indexed: 07/10/2023] Open
Abstract
Humans and animals develop learning-to-learn strategies throughout their lives to accelerate learning. One theory suggests that this is achieved by a metacognitive process of controlling and monitoring learning. Although such learning-to-learn is also observed in motor learning, the metacognitive aspect of learning regulation has not been considered in classical theories of motor learning. Here, we formulated a minimal mechanism of this process as reinforcement learning of motor learning properties, which regulates a policy for memory update in response to sensory prediction error while monitoring its performance. This theory was confirmed in human motor learning experiments, in which the subjective sense of learning-outcome association determined the direction of up- and down-regulation of both learning speed and memory retention. Thus, it provides a simple, unifying account for variations in learning speeds, where the reinforcement learning mechanism monitors and controls the motor learning process.
Collapse
Affiliation(s)
- Taisei Sugiyama
- Empowerment Informatics, University of Tsukuba, Tsukuba, Ibaraki, 305-8573, Japan
| | - Nicolas Schweighofer
- Biokinesiology and Physical Therapy, University of Southern California, Los Angeles, CA, 90089-9006, USA
| | - Jun Izawa
- Institute of Systems and Information Engineering, University of Tsukuba, Tsukuba, Ibaraki, 305-8573, Japan.
| |
Collapse
|
31
|
Mikus N, Eisenegger C, Mathys C, Clark L, Müller U, Robbins TW, Lamm C, Naef M. Blocking D2/D3 dopamine receptors in male participants increases volatility of beliefs when learning to trust others. Nat Commun 2023; 14:4049. [PMID: 37422466 PMCID: PMC10329681 DOI: 10.1038/s41467-023-39823-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 06/29/2023] [Indexed: 07/10/2023] Open
Abstract
The ability to learn about other people is crucial for human social functioning. Dopamine has been proposed to regulate the precision of beliefs, but direct behavioural evidence of this is lacking. In this study, we investigate how a high dose of the D2/D3 dopamine receptor antagonist sulpiride impacts learning about other people's prosocial attitudes in a repeated Trust game. Using a Bayesian model of belief updating, we show that in a sample of 76 male participants sulpiride increases the volatility of beliefs, which leads to higher precision weights on prediction errors. This effect is driven by participants with genetically conferred higher dopamine availability (Taq1a polymorphism) and remains even after controlling for working memory performance. Higher precision weights are reflected in higher reciprocal behaviour in the repeated Trust game but not in single-round Trust games. Our data provide evidence that the D2 receptors are pivotal in regulating prediction error-driven belief updating in a social context.
Collapse
Affiliation(s)
- Nace Mikus
- Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria.
- Interacting Minds Centre, Aarhus University, Aarhus, Denmark.
| | - Christoph Eisenegger
- Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
- Behavioural and Clinical Neuroscience Institute and Department of Psychology, University of Cambridge, Cambridge, UK
| | - Christoph Mathys
- Interacting Minds Centre, Aarhus University, Aarhus, Denmark
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich and ETH Zurich, Zurich, Switzerland
- Scuola Internazionale Superiore di Studi Avanzati (SISSA), Trieste, Italy
| | - Luke Clark
- Centre for Gambling Research at UBC, Department of Psychology, University of British, Columbia, Vancouver, BC, Canada
- Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, BC, Canada
| | - Ulrich Müller
- Behavioural and Clinical Neuroscience Institute and Department of Psychology, University of Cambridge, Cambridge, UK
- Adult Neurodevelopmental Services, Health & Community Services, Government of Jersey, St Helier, Jersey
| | - Trevor W Robbins
- Behavioural and Clinical Neuroscience Institute and Department of Psychology, University of Cambridge, Cambridge, UK
| | - Claus Lamm
- Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria.
| | - Michael Naef
- Department of Economics, University of Durham, Durham, UK.
| |
Collapse
|
32
|
Konaka Y, Naoki H. Decoding reward-curiosity conflict in decision-making from irrational behaviors. Nat Comput Sci 2023; 3:418-432. [PMID: 38177842 PMCID: PMC10768639 DOI: 10.1038/s43588-023-00439-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 03/29/2023] [Indexed: 01/06/2024]
Abstract
Humans and animals are not always rational. They not only rationally exploit rewards but also explore an environment owing to their curiosity. However, the mechanism of such curiosity-driven irrational behavior is largely unknown. Here, we developed a decision-making model for a two-choice task based on the free energy principle, which is a theory integrating recognition and action selection. The model describes irrational behaviors depending on the curiosity level. We also proposed a machine learning method to decode temporal curiosity from behavioral data. By applying it to rat behavioral data, we found that the rat had negative curiosity, reflecting conservative selection sticking to more certain options and that the level of curiosity was upregulated by the expected future information obtained from an uncertain environment. Our decoding approach can be a fundamental tool for identifying the neural basis for reward-curiosity conflicts. Furthermore, it could be effective in diagnosing mental disorders.
Collapse
Affiliation(s)
- Yuki Konaka
- Laboratory of Data-Driven Biology, Graduate School of Integrated Sciences for Life, Hiroshima University, Hiroshima, Japan
| | - Honda Naoki
- Laboratory of Data-Driven Biology, Graduate School of Integrated Sciences for Life, Hiroshima University, Hiroshima, Japan.
- Kansei-Brain Informatics Group, Center for Brain, Mind and Kansei Sciences Research, Hiroshima University, Hiroshima, Japan.
- Theoretical Biology Research Group, Exploratory Research Center on Life and Living Systems, National Institutes of Natural Sciences, Okazaki, Japan.
- Laboratory of Theoretical Biology, Graduate School of Biostudies, Kyoto University, Kyoto, Japan.
| |
Collapse
|
33
|
Shervani-Tabar N, Rosenbaum R. Meta-learning biologically plausible plasticity rules with random feedback pathways. Nat Commun 2023; 14:1805. [PMID: 37002222 PMCID: PMC10066328 DOI: 10.1038/s41467-023-37562-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Accepted: 03/21/2023] [Indexed: 04/04/2023] Open
Abstract
Backpropagation is widely used to train artificial neural networks, but its relationship to synaptic plasticity in the brain is unknown. Some biological models of backpropagation rely on feedback projections that are symmetric with feedforward connections, but experiments do not corroborate the existence of such symmetric backward connectivity. Random feedback alignment offers an alternative model in which errors are propagated backward through fixed, random backward connections. This approach successfully trains shallow models, but learns slowly and does not perform well with deeper models or online learning. In this study, we develop a meta-learning approach to discover interpretable, biologically plausible plasticity rules that improve online learning performance with fixed random feedback connections. The resulting plasticity rules show improved online training of deep models in the low data regime. Our results highlight the potential of meta-learning to discover effective, interpretable learning rules satisfying biological constraints.
Collapse
Affiliation(s)
- Navid Shervani-Tabar
- Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN, 46556, USA.
| | - Robert Rosenbaum
- Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN, 46556, USA
| |
Collapse
|
34
|
Galinsky VL, Frank LR. Critically synchronized brain waves form an effective, robust and flexible basis for human memory and learning. Sci Rep 2023; 13:4343. [PMID: 36928606 PMCID: PMC10020450 DOI: 10.1038/s41598-023-31365-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Accepted: 03/10/2023] [Indexed: 03/18/2023] Open
Abstract
The effectiveness, robustness, and flexibility of memory and learning constitute the very essence of human natural intelligence, cognition, and consciousness. However, currently accepted views on these subjects have, to date, been put forth without any basis on a true physical theory of how the brain communicates internally via its electrical signals. This lack of a solid theoretical framework has implications not only for our understanding of how the brain works, but also for wide range of computational models developed from the standard orthodox view of brain neuronal organization and brain network derived functioning based on the Hodgkin-Huxley ad-hoc circuit analogies that have produced a multitude of Artificial, Recurrent, Convolution, Spiking, etc., Neural Networks (ARCSe NNs) that have in turn led to the standard algorithms that form the basis of artificial intelligence (AI) and machine learning (ML) methods. Our hypothesis, based upon our recently developed physical model of weakly evanescent brain wave propagation (WETCOW) is that, contrary to the current orthodox model that brain neurons just integrate and fire under accompaniment of slow leaking, they can instead perform much more sophisticated tasks of efficient coherent synchronization/desynchronization guided by the collective influence of propagating nonlinear near critical brain waves, the waves that currently assumed to be nothing but inconsequential subthreshold noise. In this paper we highlight the learning and memory capabilities of our WETCOW framework and then apply it to the specific application of AI/ML and Neural Networks. We demonstrate that the learning inspired by these critically synchronized brain waves is shallow, yet its timing and accuracy outperforms deep ARCSe counterparts on standard test datasets. These results have implications for both our understanding of brain function and for the wide range of AI/ML applications.
Collapse
Affiliation(s)
- Vitaly L Galinsky
- Center for Scientific Computation in Imaging, University of California at San Diego, La Jolla, CA, 92037-0854, USA.
| | - Lawrence R Frank
- Center for Scientific Computation in Imaging, University of California at San Diego, La Jolla, CA, 92037-0854, USA
- Center for Functional MRI, University of California at San Diego, La Jolla, CA, 92037-0677, USA
| |
Collapse
|
35
|
Pietrzak P, Szczęsny S, Huderek D, Przyborowski Ł. Overview of Spiking Neural Network Learning Approaches and Their Computational Complexities. Sensors (Basel) 2023; 23:3037. [PMID: 36991750 PMCID: PMC10053242 DOI: 10.3390/s23063037] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 03/08/2023] [Accepted: 03/09/2023] [Indexed: 06/19/2023]
Abstract
Spiking neural networks (SNNs) are subjects of a topic that is gaining more and more interest nowadays. They more closely resemble actual neural networks in the brain than their second-generation counterparts, artificial neural networks (ANNs). SNNs have the potential to be more energy efficient than ANNs on event-driven neuromorphic hardware. This can yield drastic maintenance cost reduction for neural network models, as the energy consumption would be much lower in comparison to regular deep learning models hosted in the cloud today. However, such hardware is still not yet widely available. On standard computer architectures consisting mainly of central processing units (CPUs) and graphics processing units (GPUs) ANNs, due to simpler models of neurons and simpler models of connections between neurons, have the upper hand in terms of execution speed. In general, they also win in terms of learning algorithms, as SNNs do not reach the same levels of performance as their second-generation counterparts in typical machine learning benchmark tasks, such as classification. In this paper, we review existing learning algorithms for spiking neural networks, divide them into categories by type, and assess their computational complexity.
Collapse
|
36
|
Fornari L, Ioumpa K, Nostro AD, Evans NJ, De Angelis L, Speer SPH, Paracampo R, Gallo S, Spezio M, Keysers C, Gazzola V. Neuro-computational mechanisms and individual biases in action-outcome learning under moral conflict. Nat Commun 2023; 14:1218. [PMID: 36878911 PMCID: PMC9988878 DOI: 10.1038/s41467-023-36807-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Accepted: 02/17/2023] [Indexed: 03/08/2023] Open
Abstract
Learning to predict action outcomes in morally conflicting situations is essential for social decision-making but poorly understood. Here we tested which forms of Reinforcement Learning Theory capture how participants learn to choose between self-money and other-shocks, and how they adapt to changes in contingencies. We find choices were better described by a reinforcement learning model based on the current value of separately expected outcomes than by one based on the combined historical values of past outcomes. Participants track expected values of self-money and other-shocks separately, with the substantial individual difference in preference reflected in a valuation parameter balancing their relative weight. This valuation parameter also predicted choices in an independent costly helping task. The expectations of self-money and other-shocks were biased toward the favored outcome but fMRI revealed this bias to be reflected in the ventromedial prefrontal cortex while the pain-observation network represented pain prediction errors independently of individual preferences.
Collapse
Affiliation(s)
- Laura Fornari
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Kalliopi Ioumpa
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Alessandra D Nostro
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Nathan J Evans
- School of Psychology, University of Queensland, Brisbane, QLD, Australia
| | - Lorenzo De Angelis
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Sebastian P H Speer
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Riccardo Paracampo
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Selene Gallo
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands
| | - Michael Spezio
- Psychology, Neuroscience, & Data Science, Scripps College, 1030 Columbia Ave, CA 91711, Claremont, CA, USA
| | - Christian Keysers
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands.,Department of Psychology, University of Amsterdam, Nieuwe Achtergracht 129-B, 1018 WT, Amsterdam, The Netherlands
| | - Valeria Gazzola
- Netherlands Institute for Neuroscience, KNAW, Meibergdreef 47, 1105BA, Amsterdam, The Netherlands. .,Department of Psychology, University of Amsterdam, Nieuwe Achtergracht 129-B, 1018 WT, Amsterdam, The Netherlands.
| |
Collapse
|
37
|
Topalovic U, Barclay S, Ling C, Alzuhair A, Yu W, Hokhikyan V, Chandrakumar H, Rozgic D, Jiang W, Basir-Kazeruni S, Maoz SL, Inman CS, Stangl M, Gill J, Bari A, Fallah A, Eliashiv D, Pouratian N, Fried I, Suthana N, Markovic D. A wearable platform for closed-loop stimulation and recording of single-neuron and local field potential activity in freely moving humans. Nat Neurosci 2023; 26:517-27. [PMID: 36804647 DOI: 10.1038/s41593-023-01260-4] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 01/17/2023] [Indexed: 02/22/2023]
Abstract
Advances in technologies that can record and stimulate deep brain activity in humans have led to impactful discoveries within the field of neuroscience and contributed to the development of novel therapies for neurological and psychiatric disorders. Further progress, however, has been hindered by device limitations in that recording of single-neuron activity during freely moving behaviors in humans has not been possible. Additionally, implantable neurostimulation devices, currently approved for human use, have limited stimulation programmability and restricted full-duplex bidirectional capability. In this study, we developed a wearable bidirectional closed-loop neuromodulation system (Neuro-stack) and used it to record single-neuron and local field potential activity during stationary and ambulatory behavior in humans. Together with a highly flexible and customizable stimulation capability, the Neuro-stack provides an opportunity to investigate the neurophysiological basis of disease, develop improved responsive neuromodulation therapies, explore brain function during naturalistic behaviors in humans and, consequently, bridge decades of neuroscientific findings across species.
Collapse
|
38
|
Spisak T, Bingel U, Wager TD. Multivariate BWAS can be replicable with moderate sample sizes. Nature 2023; 615:E4-7. [PMID: 36890392 DOI: 10.1038/s41586-023-05745-x] [Citation(s) in RCA: 26] [Impact Index Per Article: 26.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2022] [Accepted: 01/19/2023] [Indexed: 03/10/2023]
|
39
|
Johnston WJ, Fusi S. Abstract representations emerge naturally in neural networks trained to perform multiple tasks. Nat Commun 2023; 14:1040. [PMID: 36823136 PMCID: PMC9950464 DOI: 10.1038/s41467-023-36583-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Accepted: 02/07/2023] [Indexed: 02/25/2023] Open
Abstract
Humans and other animals demonstrate a remarkable ability to generalize knowledge across distinct contexts and objects during natural behavior. We posit that this ability to generalize arises from a specific representational geometry, that we call abstract and that is referred to as disentangled in machine learning. These abstract representations have been observed in recent neurophysiological studies. However, it is unknown how they emerge. Here, using feedforward neural networks, we demonstrate that the learning of multiple tasks causes abstract representations to emerge, using both supervised and reinforcement learning. We show that these abstract representations enable few-sample learning and reliable generalization on novel tasks. We conclude that abstract representations of sensory and cognitive variables may emerge from the multiple behaviors that animals exhibit in the natural world, and, as a consequence, could be pervasive in high-level brain regions. We also make several specific predictions about which variables will be represented abstractly.
Collapse
Affiliation(s)
- W Jeffrey Johnston
- Center for Theoretical Neuroscience, Columbia University, New York, NY, USA. .,Mortimer B. Zuckerman Mind, Brain and Behavior Institute, Columbia University, New York, NY, USA.
| | - Stefano Fusi
- Center for Theoretical Neuroscience, Columbia University, New York, NY, USA. .,Mortimer B. Zuckerman Mind, Brain and Behavior Institute, Columbia University, New York, NY, USA.
| |
Collapse
|
40
|
Makino H. Arithmetic value representation for hierarchical behavior composition. Nat Neurosci 2023; 26:140-149. [PMID: 36550292 PMCID: PMC9829535 DOI: 10.1038/s41593-022-01211-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Accepted: 10/21/2022] [Indexed: 12/24/2022]
Abstract
The ability to compose new skills from a preacquired behavior repertoire is a hallmark of biological intelligence. Although artificial agents extract reusable skills from past experience and recombine them in a hierarchical manner, whether the brain similarly composes a novel behavior is largely unknown. In the present study, I show that deep reinforcement learning agents learn to solve a novel composite task by additively combining representations of prelearned action values of constituent subtasks. Learning efficacy in the composite task was further augmented by the introduction of stochasticity in behavior during pretraining. These theoretical predictions were empirically tested in mice, where subtask pretraining enhanced learning of the composite task. Cortex-wide, two-photon calcium imaging revealed analogous neural representations of combined action values, with improved learning when the behavior variability was amplified. Together, these results suggest that the brain composes a novel behavior with a simple arithmetic operation of preacquired action-value representations with stochastic policies.
Collapse
Affiliation(s)
- Hiroshi Makino
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.
| |
Collapse
|
41
|
Abstract
Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions1-3. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction4; however, so far there has been little consideration of how direct policy learning might inform our understanding5. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning6.
Collapse
|
42
|
Harris DJ, Arthur T, Vine SJ, Liu J, Abd Rahman HR, Han F, Wilson MR. Task-evoked pupillary responses track precision-weighted prediction errors and learning rate during interceptive visuomotor actions. Sci Rep 2022; 12:22098. [PMID: 36543845 DOI: 10.1038/s41598-022-26544-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Accepted: 12/15/2022] [Indexed: 12/24/2022] Open
Abstract
In this study, we examined the relationship between physiological encoding of surprise and the learning of anticipatory eye movements. Active inference portrays perception and action as interconnected inference processes, driven by the imperative to minimise the surprise of sensory observations. To examine this characterisation of oculomotor learning during a hand-eye coordination task, we tested whether anticipatory eye movements were updated in accordance with Bayesian principles and whether trial-by-trial learning rates tracked pupil dilation as a marker of 'surprise'. Forty-four participants completed an interception task in immersive virtual reality that required them to hit bouncing balls that had either expected or unexpected bounce profiles. We recorded anticipatory eye movements known to index participants' beliefs about likely ball bounce trajectories. By fitting a hierarchical Bayesian inference model to the trial-wise trajectories of these predictive eye movements, we were able to estimate each individual's expectations about bounce trajectories, rates of belief updating, and precision-weighted prediction errors. We found that the task-evoked pupil response tracked prediction errors and learning rates but not beliefs about ball bounciness or environmental volatility. These findings are partially consistent with active inference accounts and shed light on how encoding of surprise may shape the control of action.
Collapse
|
43
|
Saini F, Ponzo S, Silvestrin F, Fotopoulou A, David AS. Depersonalization disorder as a systematic downregulation of interoceptive signals. Sci Rep 2022; 12:22123. [PMID: 36543824 PMCID: PMC9772393 DOI: 10.1038/s41598-022-22277-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 10/12/2022] [Indexed: 12/24/2022] Open
Abstract
Depersonalisation disorder (DPD) is a psychopathological condition characterised by a feeling of detachment from one's own body and surrounding, and it is understood as emerging from the downregulation of interoceptive afferents. However, the precise mechanisms that drive this 'interoceptive silencing' are yet to be clarified. Here we present a computational and neurobiologically plausible model of DPD within the active inference framework. Specifically, we describe DPD as arising from disrupted interoceptive processing at higher levels of the cortical hierarchy where the interoceptive and exteroceptive streams are integrated. We simulated the behaviour of an agent subjected to a situation of high interoceptive activation despite the absence of a perceivable threat in the external environment. The simulation showed how a similar condition, if perceived as inescapable, would result in a downregulation of interoceptive signals, whilst leaving the exteroceptive ones unaffected. Such interoceptive silencing would force the agent to over-rely on exteroceptive information and would ultimately lead to the DPD phenomenology. Finally, our simulation shows that repeated exposure to similar situations over time will lead the agent to increasingly disengage from bodily responses even in the face of a less triggering situation, explaining how a single episode of depersonalization can lead to chronic DPD.
Collapse
Affiliation(s)
- Fedal Saini
- grid.499389.60000 0004 0375 2443Institute of Psychiatry, Psychology and Neuroscience, King’s London College, London, SE5 8AF UK
| | - Sonia Ponzo
- Flo Health, London, UK ,grid.83440.3b0000000121901201Institute of Health Informatics, University College London, London, UK
| | - Francesco Silvestrin
- Thrive Therapeutic Software Ltd., London, UK ,grid.8273.e0000 0001 1092 7967University of East Anglia, Norwich Research Park, Norwich, Norfolk NR4 7TJ UK
| | - Aikaterini Fotopoulou
- grid.83440.3b0000000121901201Division of Psychology & Language Sciences, Clinical, Educational & Health Psychology Research Department, University College London, London, UK
| | - Anthony S. David
- grid.83440.3b0000000121901201Institute of Mental Health, Faculty of Brain Sciences, University College London, London, UK
| |
Collapse
|
44
|
Gu Z, Jamison K, Sabuncu M, Kuceyeski A. Personalized visual encoding model construction with small data. Commun Biol 2022; 5:1382. [PMID: 36528715 PMCID: PMC9759560 DOI: 10.1038/s42003-022-04347-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2022] [Accepted: 12/05/2022] [Indexed: 12/23/2022] Open
Abstract
Quantifying population heterogeneity in brain stimuli-response mapping may allow insight into variability in bottom-up neural systems that can in turn be related to individual's behavior or pathological state. Encoding models that predict brain responses to stimuli are one way to capture this relationship. However, they generally need a large amount of fMRI data to achieve optimal accuracy. Here, we propose an ensemble approach to create encoding models for novel individuals with relatively little data by modeling each subject's predicted response vector as a linear combination of the other subjects' predicted response vectors. We show that these ensemble encoding models trained with hundreds of image-response pairs, achieve accuracy not different from models trained on 20,000 image-response pairs. Importantly, the ensemble encoding models preserve patterns of inter-individual differences in the image-response relationship. We also show the proposed approach is robust against domain shift by validating on data with a different scanner and experimental setup. Additionally, we show that the ensemble encoding models are able to discover the inter-individual differences in various face areas' responses to images of animal vs human faces using a recently developed NeuroGen framework. Our approach shows the potential to use existing densely-sampled data, i.e. large amounts of data collected from a single individual, to efficiently create accurate, personalized encoding models and, subsequently, personalized optimal synthetic images for new individuals scanned under different experimental conditions.
Collapse
Affiliation(s)
- Zijin Gu
- grid.5386.8000000041936877XSchool of Electrical and Computer Engineering, Cornell University, Ithaca, NY USA
| | - Keith Jamison
- grid.5386.8000000041936877XDepartment of Radiology, Weill Cornell Medicine, New York, NY USA
| | - Mert Sabuncu
- grid.5386.8000000041936877XSchool of Electrical and Computer Engineering, Cornell University, Ithaca, NY USA ,grid.5386.8000000041936877XDepartment of Radiology, Weill Cornell Medicine, New York, NY USA
| | - Amy Kuceyeski
- grid.5386.8000000041936877XDepartment of Radiology, Weill Cornell Medicine, New York, NY USA
| |
Collapse
|
45
|
Hernandez Petzsche MR, de la Rosa E, Hanning U, Wiest R, Valenzuela W, Reyes M, Meyer M, Liew SL, Kofler F, Ezhov I, Robben D, Hutton A, Friedrich T, Zarth T, Bürkle J, Baran TA, Menze B, Broocks G, Meyer L, Zimmer C, Boeckh-Behrens T, Berndt M, Ikenberg B, Wiestler B, Kirschke JS. ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset. Sci Data 2022; 9:762. [PMID: 36496501 PMCID: PMC9741583 DOI: 10.1038/s41597-022-01875-5] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Accepted: 11/28/2022] [Indexed: 12/13/2022] Open
Abstract
Magnetic resonance imaging (MRI) is an important imaging modality in stroke. Computer based automated medical image processing is increasingly finding its way into clinical routine. The Ischemic Stroke Lesion Segmentation (ISLES) challenge is a continuous effort to develop and identify benchmark methods for acute and sub-acute ischemic stroke lesion segmentation. Here we introduce an expert-annotated, multicenter MRI dataset for segmentation of acute to subacute stroke lesions ( https://doi.org/10.5281/zenodo.7153326 ). This dataset comprises 400 multi-vendor MRI cases with high variability in stroke lesion size, quantity and location. It is split into a training dataset of n = 250 and a test dataset of n = 150. All training data is publicly available. The test dataset will be used for model validation only and will not be released to the public. This dataset serves as the foundation of the ISLES 2022 challenge ( https://www.isles-challenge.org/ ) with the goal of finding algorithmic methods to enable the development and benchmarking of automatic, robust and accurate segmentation methods for ischemic stroke.
Collapse
Affiliation(s)
- Moritz R. Hernandez Petzsche
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Ezequiel de la Rosa
- grid.435381.8icometrix, Leuven, Belgium ,grid.6936.a0000000123222966Department of Informatics, Technical University of Munich, Munich, Germany
| | - Uta Hanning
- grid.13648.380000 0001 2180 3484Department of Diagnostic and Interventional Neuroradiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Roland Wiest
- grid.5734.50000 0001 0726 5157Institute of Diagnostic and Interventional Neuroradiology, University of Bern, Bern, Switzerland
| | - Waldo Valenzuela
- grid.5734.50000 0001 0726 5157Institute of Diagnostic and Interventional Neuroradiology, University of Bern, Bern, Switzerland
| | - Mauricio Reyes
- grid.5734.50000 0001 0726 5157ARTORG Center for Biomedical Engineering Research, Univ. of Bern, Bern, Switzerland
| | | | - Sook-Lei Liew
- grid.42505.360000 0001 2156 6853Chan Division of Occupational Science and Occupational Therapy, University of Southern California, Los Angeles, CA USA
| | - Florian Kofler
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany ,grid.6936.a0000000123222966Department of Informatics, Technical University of Munich, Munich, Germany ,grid.6936.a0000000123222966TranslaTUM – Central Institute for Translational Cancer Research, Technical University of Munich, Munich, Germany ,Helmholtz AI, Helmholtz Zentrum Munich, Munich, Germany
| | - Ivan Ezhov
- grid.6936.a0000000123222966Department of Informatics, Technical University of Munich, Munich, Germany ,grid.6936.a0000000123222966TranslaTUM – Central Institute for Translational Cancer Research, Technical University of Munich, Munich, Germany
| | | | - Alexandre Hutton
- grid.42505.360000 0001 2156 6853Chan Division of Occupational Science and Occupational Therapy, University of Southern California, Los Angeles, CA USA
| | - Tassilo Friedrich
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Teresa Zarth
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Johannes Bürkle
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - The Anh Baran
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Björn Menze
- grid.6936.a0000000123222966Department of Informatics, Technical University of Munich, Munich, Germany ,grid.7400.30000 0004 1937 0650Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | - Gabriel Broocks
- grid.13648.380000 0001 2180 3484Department of Diagnostic and Interventional Neuroradiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Lukas Meyer
- grid.13648.380000 0001 2180 3484Department of Diagnostic and Interventional Neuroradiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Claus Zimmer
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Tobias Boeckh-Behrens
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Maria Berndt
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Benno Ikenberg
- grid.6936.a0000000123222966Department of Neurology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Benedikt Wiestler
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany ,grid.6936.a0000000123222966TranslaTUM – Central Institute for Translational Cancer Research, Technical University of Munich, Munich, Germany
| | - Jan S. Kirschke
- grid.6936.a0000000123222966Department of Diagnostic and Interventional Neuroradiology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany ,grid.6936.a0000000123222966TranslaTUM – Central Institute for Translational Cancer Research, Technical University of Munich, Munich, Germany
| |
Collapse
|
46
|
Zhang Z, Wang L, Cheng S. Composed query image retrieval based on triangle area triple loss function and combining CNN with transformer. Sci Rep 2022; 12:20800. [PMID: 36460827 PMCID: PMC9718755 DOI: 10.1038/s41598-022-25340-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 11/29/2022] [Indexed: 12/03/2022] Open
Abstract
The existing typical combined query image retrieval methods adopt Euclidean distance as sample distance measurement method, and the model trained by triple loss function blindly pursues absolute distance between samples, resulting in unsatisfactory image retrieval performance. Meanwhile, these methods singularly adopt Convolutional Neural Network (CNN) to extract reference image features. However, receptive field of convolution operation has the characteristics of locality, which is easy to cause the loss of edge feature information of reference images. In view of shortcomings of these methods, the following improvements are proposed in this paper: (1) We propose Triangle Area Triple Loss Function (TATLF), which adopts Triangle Area (TA) as measurement of sample distance. TA comprehensively considers the absolute distance and included angle between samples, so that the trained model has better retrieval performance; (2) We combine CNN with Transformer to simultaneously extract local and edge features of reference images, which can effectively reduce the loss of reference images information. Specifically, CNN is adopted to extract local feature information of reference images. Transformer is used to pay attention to the edge feature information of reference images. Extensive experiments on two public datasets, Fashion200k and MIT-States, confirm the excellent performance of our proposed method.
Collapse
Affiliation(s)
- Zhiwei Zhang
- grid.413254.50000 0000 9544 7024College of Information Science and Engineering, Xinjiang University, Urumqi, 830046 China
| | - Liejun Wang
- grid.413254.50000 0000 9544 7024College of Information Science and Engineering, Xinjiang University, Urumqi, 830046 China
| | - Shuli Cheng
- grid.413254.50000 0000 9544 7024College of Information Science and Engineering, Xinjiang University, Urumqi, 830046 China
| |
Collapse
|
47
|
Zarkeshian P, Kergan T, Ghobadi R, Nicola W, Simon C. Photons guided by axons may enable backpropagation-based learning in the brain. Sci Rep 2022; 12:20720. [PMID: 36456619 PMCID: PMC9715721 DOI: 10.1038/s41598-022-24871-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Accepted: 11/22/2022] [Indexed: 12/03/2022] Open
Abstract
Despite great advances in explaining synaptic plasticity and neuron function, a complete understanding of the brain's learning algorithms is still missing. Artificial neural networks provide a powerful learning paradigm through the backpropagation algorithm which modifies synaptic weights by using feedback connections. Backpropagation requires extensive communication of information back through the layers of a network. This has been argued to be biologically implausible and it is not clear whether backpropagation can be realized in the brain. Here we suggest that biophotons guided by axons provide a potential channel for backward transmission of information in the brain. Biophotons have been experimentally shown to be produced in the brain, yet their purpose is not understood. We propose that biophotons can propagate from each post-synaptic neuron to its pre-synaptic one to carry the required information backward. To reflect the stochastic character of biophoton emissions, our model includes the stochastic backward transmission of teaching signals. We demonstrate that a three-layered network of neurons can learn the MNIST handwritten digit classification task using our proposed backpropagation-like algorithm with stochastic photonic feedback. We model realistic restrictions and show that our system still learns the task for low rates of biophoton emission, information-limited (one bit per photon) backward transmission, and in the presence of noise photons. Our results suggest a new functionality for biophotons and provide an alternate mechanism for backward transmission in the brain.
Collapse
Affiliation(s)
- Parisa Zarkeshian
- grid.22072.350000 0004 1936 7697Department of Physics & Astronomy, University of Calgary, 2500 University Drive NW, Calgary, AB T2N 1N4 Canada ,grid.22072.350000 0004 1936 7697Institute for Quantum Science and Technology, University of Calgary, 2500 University Drive NW, Calgary, AB T2N 1N4 Canada ,grid.22072.350000 0004 1936 7697Hotchkiss Brain Institute, University of Calgary, 3330 Hospital Drive NW, Calgary, AB T2N 4N1 Canada ,1QB Information Technologies (1QBit), Vancouver, BC Canada
| | - Taylor Kergan
- grid.22072.350000 0004 1936 7697Department of Physics & Astronomy, University of Calgary, 2500 University Drive NW, Calgary, AB T2N 1N4 Canada
| | - Roohollah Ghobadi
- grid.22072.350000 0004 1936 7697Department of Physics & Astronomy, University of Calgary, 2500 University Drive NW, Calgary, AB T2N 1N4 Canada ,grid.22072.350000 0004 1936 7697Institute for Quantum Science and Technology, University of Calgary, 2500 University Drive NW, Calgary, AB T2N 1N4 Canada ,grid.22072.350000 0004 1936 7697Hotchkiss Brain Institute, University of Calgary, 3330 Hospital Drive NW, Calgary, AB T2N 4N1 Canada
| | - Wilten Nicola
- grid.22072.350000 0004 1936 7697Department of Physics & Astronomy, University of Calgary, 2500 University Drive NW, Calgary, AB T2N 1N4 Canada ,grid.22072.350000 0004 1936 7697Hotchkiss Brain Institute, University of Calgary, 3330 Hospital Drive NW, Calgary, AB T2N 4N1 Canada ,grid.22072.350000 0004 1936 7697Department of Cell Biology and Anatomy, University of Calgary, Cumming School of Medicine, 3330 Hospital Drive NW, Calgary, AB Canada
| | - Christoph Simon
- grid.22072.350000 0004 1936 7697Department of Physics & Astronomy, University of Calgary, 2500 University Drive NW, Calgary, AB T2N 1N4 Canada ,grid.22072.350000 0004 1936 7697Institute for Quantum Science and Technology, University of Calgary, 2500 University Drive NW, Calgary, AB T2N 1N4 Canada ,grid.22072.350000 0004 1936 7697Hotchkiss Brain Institute, University of Calgary, 3330 Hospital Drive NW, Calgary, AB T2N 4N1 Canada
| |
Collapse
|
48
|
Ou W, Xiao S, Zhu C, Han W, Zhang Q. An overview of brain-like computing: Architecture, applications, and future trends. Front Neurorobot 2022; 16:1041108. [PMID: 36506817 PMCID: PMC9730831 DOI: 10.3389/fnbot.2022.1041108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2022] [Accepted: 10/31/2022] [Indexed: 11/25/2022] Open
Abstract
With the development of technology, Moore's law will come to an end, and scientists are trying to find a new way out in brain-like computing. But we still know very little about how the brain works. At the present stage of research, brain-like models are all structured to mimic the brain in order to achieve some of the brain's functions, and then continue to improve the theories and models. This article summarizes the important progress and status of brain-like computing, summarizes the generally accepted and feasible brain-like computing models, introduces, analyzes, and compares the more mature brain-like computing chips, outlines the attempts and challenges of brain-like computing applications at this stage, and looks forward to the future development of brain-like computing. It is hoped that the summarized results will help relevant researchers and practitioners to quickly grasp the research progress in the field of brain-like computing and acquire the application methods and related knowledge in this field.
Collapse
Affiliation(s)
- Wei Ou
- The School of Cyberspace Security, Hainan University, Hainan, China
- Henan Key Laboratory of Network Cryptography Technology, Zhengzhou, China
| | - Shitao Xiao
- The School of Computer Science and Technology, Hainan, China
| | - Chengyu Zhu
- The School of Cyberspace Security, Hainan University, Hainan, China
| | - Wenbao Han
- The School of Cyberspace Security, Hainan University, Hainan, China
| | - Qionglu Zhang
- State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
49
|
Mancini F, Zhang S, Seymour B. Computational and neural mechanisms of statistical pain learning. Nat Commun 2022; 13:6613. [PMID: 36329014 PMCID: PMC9633765 DOI: 10.1038/s41467-022-34283-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Accepted: 10/11/2022] [Indexed: 11/06/2022] Open
Abstract
Pain invariably changes over time. These fluctuations contain statistical regularities which, in theory, could be learned by the brain to generate expectations and control responses. We demonstrate that humans learn to extract these regularities and explicitly predict the likelihood of forthcoming pain intensities in a manner consistent with optimal Bayesian inference with dynamic update of beliefs. Healthy participants received probabilistic, volatile sequences of low and high-intensity electrical stimuli to the hand during brain fMRI. The inferred frequency of pain correlated with activity in sensorimotor cortical regions and dorsal striatum, whereas the uncertainty of these inferences was encoded in the right superior parietal cortex. Unexpected changes in stimulus frequencies drove the update of internal models by engaging premotor, prefrontal and posterior parietal regions. This study extends our understanding of sensory processing of pain to include the generation of Bayesian internal models of the temporal statistics of pain.
Collapse
Affiliation(s)
- Flavia Mancini
- Department of Engineering, University of Cambridge, Trumpington Street, Cambridge, CB2 1PZ, UK.
| | - Suyi Zhang
- Wellcome Centre for Integrative Neuroimaging, John Radcliffe Hospital, Headington, Oxford, OX3 9DU, UK
| | - Ben Seymour
- Wellcome Centre for Integrative Neuroimaging, John Radcliffe Hospital, Headington, Oxford, OX3 9DU, UK
- Center for Information and Neural Networks (CiNet), 1-4 Yamadaoka, Suita City, Osaka, 565-0871, Japan
| |
Collapse
|
50
|
Li Y, Zheng H, Huang X, Chang J, Hou D, Lu H. Research on lung nodule recognition algorithm based on deep feature fusion and MKL-SVM-IPSO. Sci Rep 2022; 12:17403. [PMID: 36257988 DOI: 10.1038/s41598-022-22442-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Accepted: 10/14/2022] [Indexed: 01/10/2023] Open
Abstract
Lung CAD system can provide auxiliary third-party opinions for doctors, improve the accuracy of lung nodule recognition. The selection and fusion of nodule features and the advancement of recognition algorithms are crucial improving lung CAD systems. Based on the HDL model, this paper mainly focuses on the three key algorithms of feature extraction, feature fusion and nodule recognition of lung CAD system. First, CBAM is embedded into VGG16 and VGG19, and feature extraction models AE-VGG16 and AE-VGG19 are constructed, so that the network can pay more attention to the key feature information in nodule description. Then, feature dimensionality reduction based on PCA and feature fusion based on CCA are sequentially performed on the extracted depth features to obtain low-dimensional fusion features. Finally, the fusion features are input into the proposed MKL-SVM-IPSO model based on the improved Particle Swarm Optimization algorithm to speed up the training speed, get the global optimal parameter group. The public dataset LUNA16 was selected for the experiment. The results show that the accuracy of lung nodule recognition of the proposed lung CAD system can reach 99.56%, and the sensitivity and F1-score can reach 99.3% and 0.9965, respectively, which can reduce the possibility of false detection and missed detection of nodules.
Collapse
|