1
|
Tong K, Fu X, Hoo NP, Kean Mun L, Vassiliu C, Langley C, Sahakian BJ, Leong V. The development of cognitive flexibility and its implications for mental health disorders. Psychol Med 2024:1-7. [PMID: 39247963 DOI: 10.1017/s0033291724001508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 09/10/2024]
Affiliation(s)
- Ke Tong
- Cambridge-NTU Centre for Lifelong Learning and Individualised Cognition (CLIC), Nanyang Technological University, Singapore
| | - Xinchen Fu
- Cambridge-NTU Centre for Lifelong Learning and Individualised Cognition (CLIC), Nanyang Technological University, Singapore
| | - Natalie P Hoo
- Cambridge-NTU Centre for Lifelong Learning and Individualised Cognition (CLIC), Nanyang Technological University, Singapore
| | - Lee Kean Mun
- Cambridge-NTU Centre for Lifelong Learning and Individualised Cognition (CLIC), Nanyang Technological University, Singapore
- Early Mental Potential and Wellbeing Research (EMPOWER) Centre, Nanyang Technological University, Singapore
| | | | | | | | - Victoria Leong
- Cambridge-NTU Centre for Lifelong Learning and Individualised Cognition (CLIC), Nanyang Technological University, Singapore
- Early Mental Potential and Wellbeing Research (EMPOWER) Centre, Nanyang Technological University, Singapore
| |
Collapse
|
2
|
Parr AC, Sydnor VJ, Calabro FJ, Luna B. Adolescent-to-adult gains in cognitive flexibility are adaptively supported by reward sensitivity, exploration, and neural variability. Curr Opin Behav Sci 2024; 58:101399. [PMID: 38826569 PMCID: PMC11138371 DOI: 10.1016/j.cobeha.2024.101399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/04/2024]
Abstract
Cognitive flexibility exhibits dynamic changes throughout development, with different forms of flexibility showing dissociable developmental trajectories. In this review, we propose that an adolescent-specific mode of flexibility in the face of changing environmental contingencies supports the emergence of adolescent-to-adult gains in cognitive shifting efficiency. We first describe how cognitive shifting abilities monotonically improve from childhood to adulthood, accompanied by increases in brain state flexibility, neural variability, and excitatory/inhibitory balance. We next summarize evidence supporting the existence of a dopamine-driven, adolescent peak in flexible behavior that results in reward seeking, undirected exploration, and environmental sampling. We propose a neurodevelopmental framework that relates these adolescent behaviors to the refinement of neural phenotypes relevant to mature cognitive flexibility, and thus highlight the importance of the adolescent period in fostering healthy neurocognitive trajectories.
Collapse
Affiliation(s)
- Ashley C. Parr
- Department of Psychiatry, University of Pittsburgh, Pittsburgh PA, 14213, USA
| | - Valerie J. Sydnor
- Department of Psychiatry, University of Pittsburgh, Pittsburgh PA, 14213, USA
| | - Finnegan J. Calabro
- Department of Psychiatry, University of Pittsburgh, Pittsburgh PA, 14213, USA
| | - Beatriz Luna
- Department of Psychiatry, University of Pittsburgh, Pittsburgh PA, 14213, USA
- Department of Psychology, University of Pittsburgh, Pittsburgh PA, 14213, USA
| |
Collapse
|
3
|
Sullivan ED, Dannenhoffer CA, Sutherland EB, Vidrascu EM, Gómez-A A, Boettiger CA, Robinson DL. Effects of adolescent intermittent ethanol exposure on cortical perineuronal net and parvalbumin expression in adulthood mediate behavioral inflexibility. ALCOHOL, CLINICAL & EXPERIMENTAL RESEARCH 2024; 48:1507-1518. [PMID: 39073296 PMCID: PMC11305908 DOI: 10.1111/acer.15395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 05/03/2024] [Accepted: 06/03/2024] [Indexed: 07/30/2024]
Abstract
BACKGROUND Alcohol is commonly consumed by adolescents in a binge-like pattern, which can lead to long-lasting cognitive deficits, including reduced behavioral flexibility. We and others have determined that adolescent intermittent ethanol (AIE) exposure leads to increased number of perineuronal net (PNN) numbers in brain regions that are important for behavioral flexibility. However, whether altered neurochemistry stemming from AIE exposure plays a significant role in reduced behavioral flexibility is unknown. METHODS We measured the number and size of parvalbumin expressing (PV+) interneurons and associated PNNs within the orbitofrontal cortex (OFC), prelimbic cortex (PrL), infralimbic cortex (IL), and anterior insular cortex (AIC) of female and male rats following AIE or control exposure and subsequent training on an attentional set-shift task (ASST). We then ran analyses to determine whether AIE-induced changes in PV and PNN measures statistically mediated the AIE-induced behavioral deficit in reversal learning. RESULTS We demonstrate that AIE exposure impaired behavioral flexibility on reversal two of the ASST (i.e., recalling the initial learned associations), and led to smaller PV+ cells and increased PNN numbers in the AIC. Interestingly, PNN size and number were not altered in the PrL or IL following AIE exposure, in contrast to prior reports. Mediation analyses suggest that AIE alters behavioral flexibility, at least in part through changes in PV and PNN fluorescent measures in the AIC. CONCLUSIONS This study reveals a significant link between AIE exposure, neural alterations, and diminished behavioral flexibility in rats, and highlights a potential novel mechanism comprising changes in PV and PNN measures within the AIC. Future studies should explore the impact of PNN degradation within the AIC on behavioral flexibility.
Collapse
Affiliation(s)
- Emily D.K. Sullivan
- Bowles Center for Alcohol Studies at University of North Carolina at Chapel Hill, Dept. of Psychiatry, Chapel Hill, NC, 27278, USA
| | - Carol A. Dannenhoffer
- Bowles Center for Alcohol Studies at University of North Carolina at Chapel Hill, Dept. of Psychiatry, Chapel Hill, NC, 27278, USA
| | - Elizabeth B. Sutherland
- Bowles Center for Alcohol Studies at University of North Carolina at Chapel Hill, Dept. of Psychology & Neuroscience, Chapel Hill, NC, 27278, USA
| | - Elena M. Vidrascu
- Bowles Center for Alcohol Studies at University of North Carolina at Chapel Hill, Dept. of Psychology & Neuroscience, Chapel Hill, NC, 27278, USA
| | - Alexander Gómez-A
- Bowles Center for Alcohol Studies at University of North Carolina at Chapel Hill, Dept. of Psychiatry, Chapel Hill, NC, 27278, USA
| | - Charlotte A. Boettiger
- Bowles Center for Alcohol Studies at University of North Carolina at Chapel Hill, Dept. of Psychology & Neuroscience, Chapel Hill, NC, 27278, USA
| | - Donita L. Robinson
- Bowles Center for Alcohol Studies at University of North Carolina at Chapel Hill, Dept. of Psychiatry, Chapel Hill, NC, 27278, USA
| |
Collapse
|
4
|
Beck DW, Heaton CN, Davila LD, Rakocevic LI, Drammis SM, Tyulmankov D, Vara P, Giri A, Umashankar Beck S, Zhang Q, Pokojovy M, Negishi K, Batson SA, Salcido AA, Reyes NF, Macias AY, Ibanez-Alcala RJ, Hossain SB, Waller GL, O'Dell LE, Moschak TM, Goosens KA, Friedman A. Model of a striatal circuit exploring biological mechanisms underlying decision-making during normal and disordered states. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.29.605535. [PMID: 39211231 PMCID: PMC11361035 DOI: 10.1101/2024.07.29.605535] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/04/2024]
Abstract
Decision-making requires continuous adaptation to internal and external contexts. Changes in decision-making are reliable transdiagnostic symptoms of neuropsychiatric disorders. We created a computational model demonstrating how the striosome compartment of the striatum constructs a mathematical space for decision-making computations depending on context, and how the matrix compartment defines action value depending on the space. The model explains multiple experimental results and unifies other theories like reward prediction error, roles of the direct versus indirect pathways, and roles of the striosome versus matrix, under one framework. We also found, through new analyses, that striosome and matrix neurons increase their synchrony during difficult tasks, caused by a necessary increase in dimensionality of the space. The model makes testable predictions about individual differences in disorder susceptibility, decision-making symptoms shared among neuropsychiatric disorders, and differences in neuropsychiatric disorder symptom presentation. The model reframes the role of the striosomal circuit in neuroeconomic and disorder-affected decision-making. Highlights Striosomes prioritize decision-related data used by matrix to set action values. Striosomes and matrix have different roles in the direct and indirect pathways. Abnormal information organization/valuation alters disorder presentation. Variance in data prioritization may explain individual differences in disorders. eTOC Beck et al. developed a computational model of how a striatal circuit functions during decision-making. The model unifies and extends theories about the direct versus indirect pathways. It further suggests how aberrant circuit function underlies decision-making phenomena observed in neuropsychiatric disorders.
Collapse
|
5
|
Marzuki AA, Banca P, Garofalo S, Degni LAE, Dalbagno D, Badioli M, Sule A, Kaser M, Conway-Morris A, Sahakian BJ, Robbins TW. Compulsive avoidance in youths and adults with OCD: an aversive pavlovian-to-instrumental transfer study. Transl Psychiatry 2024; 14:308. [PMID: 39060253 PMCID: PMC11282188 DOI: 10.1038/s41398-024-03028-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/06/2024] [Revised: 07/12/2024] [Accepted: 07/17/2024] [Indexed: 07/28/2024] Open
Abstract
Compulsive behaviour may often be triggered by Pavlovian cues. Assessing how Pavlovian cues drive instrumental behaviour in obsessive-compulsive disorder (OCD) is therefore crucial to understand how compulsions develop and are maintained. An aversive Pavlovian-to-Instrumental transfer (PIT) paradigm, particularly one involving avoidance/cancellation of negative outcomes, can enable such investigation and has not previously been studied in clinical-OCD. Forty-one participants diagnosed with OCD (21 adults; 20 youths) and 44 controls (21 adults; 23 youths) completed an aversive PIT task. Participants had to prevent the delivery of unpleasant noises by moving a joystick in the correct direction. They could infer these correct responses by learning appropriate response-outcome (instrumental) and stimulus-outcome (Pavlovian) associations. We then assessed whether Pavlovian cues elicited specific instrumental avoidance responses (specific PIT) and induced general instrumental avoidance (general PIT). We investigated whether task learning and confidence indices influenced PIT strength differentially between groups. There was no overall group difference in PIT performance, although youths with OCD showed weaker specific PIT than youth controls. However, urge to avoid unpleasant noises and preference for safe over unsafe stimuli influenced specific and general PIT respectively in OCD, while PIT in controls was more influenced by confidence in instrumental and Pavlovian learning. Thus, in OCD, implicit motivational factors, but not learnt knowledge, may contribute to the successful integration of aversive Pavlovian and instrumental cues. This implies that compulsive avoidance may be driven by these automatic processes. Youths with OCD show deficits in specific PIT, suggesting cue integration impairments are only apparent in adolescence. These findings may be clinically relevant as they emphasise the importance of targeting such implicit motivational processes when treating OCD.
Collapse
Affiliation(s)
- Aleya A Marzuki
- Behavioural and Clinical Neuroscience Institute, Department of Psychology, University of Cambridge, Cambridge, UK.
- Department of Psychology, School of Medical and Life Sciences, Sunway University, Petaling Jaya, Selangor, Malaysia.
| | - Paula Banca
- Behavioural and Clinical Neuroscience Institute, Department of Psychology, University of Cambridge, Cambridge, UK
| | - Sara Garofalo
- Department of Psychology, University of Bologna, Bologna, Italy
| | - Luigi A E Degni
- Department of Psychology, University of Bologna, Bologna, Italy
| | | | - Marco Badioli
- Department of Psychology, University of Bologna, Bologna, Italy
| | - Akeem Sule
- Department of Psychiatry, School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Muzaffer Kaser
- Department of Psychiatry, School of Clinical Medicine, University of Cambridge, Cambridge, UK
- Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, UK
| | | | - Barbara J Sahakian
- Behavioural and Clinical Neuroscience Institute, Department of Psychology, University of Cambridge, Cambridge, UK
- Department of Psychiatry, School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Trevor W Robbins
- Behavioural and Clinical Neuroscience Institute, Department of Psychology, University of Cambridge, Cambridge, UK.
| |
Collapse
|
6
|
Shi R, Xiang S, Jia T, Robbins TW, Kang J, Banaschewski T, Barker GJ, Bokde ALW, Desrivières S, Flor H, Grigis A, Garavan H, Gowland P, Heinz A, Brühl R, Martinot JL, Martinot MLP, Artiges E, Nees F, Orfanos DP, Paus T, Poustka L, Hohmann S, Millenet S, Fröhner JH, Smolka MN, Vaidya N, Walter H, Whelan R, Schumann G, Lin X, Sahakian BJ, Feng J. Investigating grey matter volumetric trajectories through the lifespan at the individual level. Nat Commun 2024; 15:5954. [PMID: 39009591 PMCID: PMC11251262 DOI: 10.1038/s41467-024-50305-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 07/04/2024] [Indexed: 07/17/2024] Open
Abstract
Adolescents exhibit remarkable heterogeneity in the structural architecture of brain development. However, due to limited large-scale longitudinal neuroimaging studies, existing research has largely focused on population averages, and the neurobiological basis underlying individual heterogeneity remains poorly understood. Here we identify, using the IMAGEN adolescent cohort followed up over 9 years (14-23 y), three groups of adolescents characterized by distinct developmental patterns of whole-brain gray matter volume (GMV). Group 1 show continuously decreasing GMV associated with higher neurocognitive performances than the other two groups during adolescence. Group 2 exhibit a slower rate of GMV decrease and lower neurocognitive performances compared with Group 1, which was associated with epigenetic differences and greater environmental burden. Group 3 show increasing GMV and lower baseline neurocognitive performances due to a genetic variation. Using the UK Biobank, we show these differences may be attenuated in mid-to-late adulthood. Our study reveals clusters of adolescent neurodevelopment based on GMV and the potential long-term impact.
Collapse
Grants
- R01 DA049238 NIDA NIH HHS
- R01 MH085772 NIMH NIH HHS
- R56 AG058854 NIA NIH HHS
- U54 EB020403 NIBIB NIH HHS
- National Key R&D Program of China (No.2023YFE0199700 [to X.L.])
- the Medical Research Foundation and Medical Research Council (grants MR/R00465X/1 and MR/S020306/1 [to S.D.]), the National Institutes of Health (NIH) funded ENIGMA (grants 5U54EB020403-05 and 1R56AG058854-01 [to S.D.])
- NSFC grant 82150710554 and environMENTAL grant. Further support was provided by grants from: - the ANR (ANR-12-SAMA-0004, AAPG2019 - GeBra [to J.-L.M.]), the Eranet Neuron (AF12-NEUR0008-01 - WM2NA; and ANR-18-NEUR00002-01 - ADORe [to J.-L.M.]), the Fondation de France (00081242 [to J.-L.M.]), the Fondation pour la Recherche Médicale (DPA20140629802 [to J.-L.M.]), the Mission Interministérielle de Lutte-contre-les-Drogues-et-les-Conduites-Addictives (MILDECA [to J.-L.M.]), Paris Sud University IDEX 2012 [to J.-L.M.]
- the Assistance-Publique-Hôpitaux-de-Paris and INSERM (interface grant [to M.-L.P.M.]), the Fondation de l’Avenir (grant AP-RM-17-013 [to M.-L.P.M.])
- the Fédération pour la Recherche sur le Cerveau; the National Institutes of Health, Science Foundation Ireland (16/ERCD/3797 [to R.W.])
- the European Union-funded FP6 Integrated Project IMAGEN (Reinforcement-related behaviour in normal brain function and psychopathology) (LSHM-CT- 2007-037286 [to G.S.]), the Horizon 2020 funded ERC Advanced Grant ‘STRATIFY’ (Brain network based stratification of reinforcement-related disorders) (695313 [to G.S.]), Human Brain Project (HBP SGA 2, 785907, and HBP SGA 3, 945539 [to G.S.]), the Medical Research Council Grant 'c-VEDA’ (Consortium on Vulnerability to Externalizing Disorders and Addictions) (MR/N000390/1 [to G.S.]), the National Institute of Health (NIH) (R01DA049238 [to G.S.], A decentralized macro and micro gene-by-environment interaction analysis of substance use behavior and its brain biomarkers), the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London, the Bundesministeriumfür Bildung und Forschung (BMBF grants 01GS08152; 01EV0711 [to G.S.]; Forschungsnetz AERIAL 01EE1406A, 01EE1406B; Forschungsnetz IMAC-Mind 01GL1745B [to G.S.]), the Deutsche Forschungsgemeinschaft (DFG grants SM 80/7-2, SFB 940, TRR 265, NE 1383/14-1 [to G.S.])
- National Key R&D Program of China (No.2019YFA0709502 [to J.F.], No.2018YFC1312904 [to J.F.]),No.2019YFA0709502 [to J.F.], No.2018YFC1312904 [to J.F.]), Shanghai Municipal Science and Technology Major Project (No.2018SHZDZX01 [to J.F.], ZJ Lab [to J.F.], and Shanghai Center for Brain Science and Brain-Inspired Technology [to J.F.]), the 111 Project (No.B18015 [to J.F.])
Collapse
Affiliation(s)
- Runye Shi
- School of Data Science, Fudan University, Shanghai, China
| | - Shitong Xiang
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
- Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, Shanghai, China
| | - Tianye Jia
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
- Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, Shanghai, China
- Social Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
- Centre for Population Neuroscience and Precision Medicine (PONS), Institute of Science and Technology for Brain-Inspired Intelligence (ISTBI), Fudan University, Shanghai, China
- School of Psychology, University of Southampton, Southampton, UK
| | - Trevor W Robbins
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
- Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, Shanghai, China
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, UK
| | - Jujiao Kang
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
- Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, Shanghai, China
| | - Tobias Banaschewski
- Department of Child and Adolescent Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, Mannheim, Germany
| | - Gareth J Barker
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Arun L W Bokde
- Discipline of Psychiatry, School of Medicine and Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin, Ireland
| | - Sylvane Desrivières
- Social Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Herta Flor
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, Mannheim, Germany
- Department of Psychology, School of Social Sciences, University of Mannheim, Mannheim, Germany
| | - Antoine Grigis
- NeuroSpin, CEA, Université Paris-Saclay, F-91191, Gif-sur-Yvette, France
| | - Hugh Garavan
- Departments of Psychiatry and Psychology, University of Vermont, Burlington, VT, USA
| | - Penny Gowland
- Sir Peter Mansfield Imaging Centre School of Physics and Astronomy, University of Nottingham, University Park, Nottingham, UK
| | - Andreas Heinz
- Department of Psychiatry and Psychotherapy CCM, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany
| | - Rüdiger Brühl
- Physikalisch-Technische Bundesanstalt (PTB), Braunschweig and Berlin, Germany
| | - Jean-Luc Martinot
- Institut National de la Santé et de la Recherche Médicale, INSERM U A10 "Trajectoires développementales en psychiatrie", Université Paris-Saclay, Ecole Normale supérieure Paris-Saclay, CNRS, Centre Borelli, Gif-sur-Yvette, France
| | - Marie-Laure Paillère Martinot
- Institut National de la Santé et de la Recherche Médicale, INSERM U A10 "Trajectoires développementales en psychiatrie", Université Paris-Saclay, Ecole Normale supérieure Paris-Saclay, CNRS, Centre Borelli, Gif-sur-Yvette, France
- Department of Child and Adolescent Psychiatry, AP-HP, Sorbonne Université, Pitié-Salpêtrière Hospital, Paris, France
| | - Eric Artiges
- Institut National de la Santé et de la Recherche Médicale, INSERM U A10 "Trajectoires développementales en psychiatrie", Université Paris-Saclay, Ecole Normale supérieure Paris-Saclay, CNRS, Centre Borelli, Gif-sur-Yvette, France
- Psychiatry Department, EPS Barthélémy Durand, Etampes, France
| | - Frauke Nees
- Department of Child and Adolescent Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, Mannheim, Germany
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, Mannheim, Germany
- Institute of Medical Psychology and Medical Sociology, University Medical Center Schleswig-Holstein Kiel University, Kiel, Germany
| | | | - Tomáš Paus
- Department of Psychiatry, Faculty of Medicine and Centre Hospitalier Universitaire Sainte-Justine, University of Montreal, Montreal, QC, Canada
- Departments of Psychiatry and Psychology, University of Toronto, Toronto, ON, Canada
| | - Luise Poustka
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Medical Centre Göttingen, von-Siebold-Str. 5, Göttingen, Germany
| | - Sarah Hohmann
- Department of Child and Adolescent Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, Mannheim, Germany
| | - Sabina Millenet
- Department of Child and Adolescent Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, Mannheim, Germany
| | - Juliane H Fröhner
- Department of Psychiatry and Neuroimaging Center, Technische Universität Dresden, Dresden, Germany
| | - Michael N Smolka
- Department of Psychiatry and Neuroimaging Center, Technische Universität Dresden, Dresden, Germany
| | - Nilakshi Vaidya
- Centre for Population Neuroscience and Stratified Medicine (PONS), Department of Psychiatry and Psychotherapy, Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Henrik Walter
- Department of Psychiatry and Psychotherapy CCM, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany
| | - Robert Whelan
- School of Psychology and Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
| | - Gunter Schumann
- Centre for Population Neuroscience and Precision Medicine (PONS), Institute of Science and Technology for Brain-Inspired Intelligence (ISTBI), Fudan University, Shanghai, China
- Centre for Population Neuroscience and Stratified Medicine (PONS), Department of Psychiatry and Psychotherapy, Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Xiaolei Lin
- School of Data Science, Fudan University, Shanghai, China.
- Huashan Institute of Medicine, Huashan Hospital affiliated to Fudan University, Shanghai, China.
| | - Barbara J Sahakian
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China.
- Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, Shanghai, China.
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, UK.
| | - Jianfeng Feng
- School of Data Science, Fudan University, Shanghai, China.
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China.
- Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, Shanghai, China.
- MOE Frontiers Center for Brain Science, Fudan University, Shanghai, China.
- Zhangjiang Fudan International Innovation Center, Shanghai, China.
- Department of Computer Science, University of Warwick, Coventry, UK.
| |
Collapse
|
7
|
Falck J, Zhang L, Raffington L, Mohn JJ, Triesch J, Heim C, Shing YL. Hippocampus and striatum show distinct contributions to longitudinal changes in value-based learning in middle childhood. eLife 2024; 12:RP89483. [PMID: 38953517 PMCID: PMC11219037 DOI: 10.7554/elife.89483] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/04/2024] Open
Abstract
The hippocampal-dependent memory system and striatal-dependent memory system modulate reinforcement learning depending on feedback timing in adults, but their contributions during development remain unclear. In a 2-year longitudinal study, 6-to-7-year-old children performed a reinforcement learning task in which they received feedback immediately or with a short delay following their response. Children's learning was found to be sensitive to feedback timing modulations in their reaction time and inverse temperature parameter, which quantifies value-guided decision-making. They showed longitudinal improvements towards more optimal value-based learning, and their hippocampal volume showed protracted maturation. Better delayed model-derived learning covaried with larger hippocampal volume longitudinally, in line with the adult literature. In contrast, a larger striatal volume in children was associated with both better immediate and delayed model-derived learning longitudinally. These findings show, for the first time, an early hippocampal contribution to the dynamic development of reinforcement learning in middle childhood, with neurally less differentiated and more cooperative memory systems than in adults.
Collapse
Affiliation(s)
- Johannes Falck
- Department of Psychology, Goethe University FrankfurtFrankfurtGermany
| | - Lei Zhang
- Centre for Human Brain Health, School of Psychology, University of BirminghamBirminghamUnited Kingdom
- Institute for Mental Health, School of Psychology, University of BirminghamBirminghamUnited Kingdom
- Centre for Developmental Science, School of Psychology, University of BirminghamBirminghamUnited Kingdom
- Social, Cognitive and Affective Neuroscience Unit, Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of ViennaViennaAustria
| | - Laurel Raffington
- Max Planck Research Group Biosocial, Max Planck Institute for Human DevelopmentBerlinGermany
| | - Johannes Julius Mohn
- Charité – Universitätsmedizin Berlin, Institute of Medical PsychologyBerlinGermany
- Max Planck School of Cognition, Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
| | - Jochen Triesch
- Frankfurt Institute for Advanced Studies (FIAS)Frankfurt am MainGermany
| | - Christine Heim
- Charité – Universitätsmedizin Berlin, Institute of Medical PsychologyBerlinGermany
- Center for Safe & Healthy Children, The Pennsylvania State UniversityUniversity ParkUnited States
| | - Yee Lee Shing
- Department of Psychology, Goethe University FrankfurtFrankfurtGermany
| |
Collapse
|
8
|
Chung YS, van den Berg B, Roberts KC, Woldorff MG, Gaffrey MS. Electrical brain activations in preadolescents during a probabilistic reward-learning task reflect cognitive processes and behavioral strategy. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.10.16.562326. [PMID: 37905129 PMCID: PMC10614771 DOI: 10.1101/2023.10.16.562326] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/02/2023]
Abstract
Both adults and children learn through feedback which environmental events and choices are associated with higher probability of reward, an ability thought to be supported by the development of fronto-striatal reward circuits. Recent developmental studies have applied computational models of reward learning to investigate such learning in children. However, tasks and measures effective for assaying the cascade of reward-learning neural processes in children have been limited. Using a child-version of a probabilistic reward-learning task while recording event-related-potential (ERP) measures of electrical brain activity, this study examined key processes of reward learning in preadolescents (8-12 years old; n=30), namely: (1) reward-feedback sensitivity, as measured by the early-latency, reward-related, frontal ERP positivity, (2) rapid attentional shifting of processing toward favored visual stimuli, as measured by the N2pc component, and (3) longer-latency attention-related responses to reward feedback as a function of behavioral strategies (i.e., Win-Stay-Lose-Shift), as measured by the central-parietal P300. Consistent with our prior work in adults, the behavioral findings indicate preadolescents can learn stimulus-reward outcome associations, but at varying levels of performance. Neurally, poor preadolescent learners (those with slower learning rates) showed greater reward-related positivity amplitudes relative to good learners, suggesting greater reward-feedback sensitivity. We also found attention shifting towards to-be-chosen stimuli, as evidenced by the N2pc, but not to more highly rewarded stimuli as we have observed in adults. Lastly, we found the behavioral learning strategy (i.e., Win-Stay-Lose-Shift) reflected by the feedback-elicited parietal P300. These findings provide novel insights into the key neural processes underlying reinforcement learning in preadolescents.
Collapse
Affiliation(s)
- Yu Sun Chung
- Department of Psychology and Neuroscience, Duke University, Reuben-Cooke Building, 417 Chapel Drive, Durham, NC 27708, USA
| | | | - Kenneth C. Roberts
- Center for Cognitive Neuroscience, Department of Psychiatry, Psychology & Neuroscience and Neurobiology, Duke University, Durham, NC, 27708 USA
| | - Marty G. Woldorff
- Department of Psychology and Neuroscience, Duke University, Reuben-Cooke Building, 417 Chapel Drive, Durham, NC 27708, USA
- Center for Cognitive Neuroscience, Department of Psychiatry, Psychology & Neuroscience and Neurobiology, Duke University, Durham, NC, 27708 USA
| | - Michael S. Gaffrey
- Department of Psychology and Neuroscience, Duke University, Reuben-Cooke Building, 417 Chapel Drive, Durham, NC 27708, USA
- Children’s Wisconsin, 9000 W. Wisconsin Avenue, Milwaukee, WI, 53226
- Medical College of Wisconsin, Division of Pediatric Psychology and Developmental Medicine, Department of Pediatrics, 8701 Watertown Plank Road, Milwaukee, WI, 53226
| |
Collapse
|
9
|
Zhu J, Katahira K, Hirakawa M, Nakao T. Internally Formed Preferences for Options only Influence Initial Decisions in Gambling Tasks, while the Gambling Outcomes do not Alter these Preferences. J Gambl Stud 2024:10.1007/s10899-024-10326-2. [PMID: 38922495 DOI: 10.1007/s10899-024-10326-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/13/2024] [Indexed: 06/27/2024]
Abstract
All humans must engage in decision-making. Decision-making processes can be broadly classified into internally guided decision-making (IDM), which is determined by individuals' internal value criteria, such as preference, or externally guided decision-making (EDM), which is determined by environmental external value criteria, such as monetary rewards. However, real-life decisions are never made simply using one kind of decision-making, and the relationship between IDM and EDM remains unclear. This study had individuals perform gambling tasks requiring the EDM using stimuli that formed preferences through the preference judgment task as the IDM. Computational model analysis revealed that strong preferences in the IDM affected initial choice behavior in the EDM. Moreover, through the analysis of the subjective preference evaluation after the gambling tasks, we found that even when stimuli that were preferred in the IDM were perceived as less valuable in the EDM, the preference for IDM was maintained after EDM. These results indicate that although internal criteria, such as preferences, influence EDM, the results show that internal and external criteria differ.
Collapse
Affiliation(s)
- Jianhong Zhu
- Graduate School of Humanities and Social Sciences, Hiroshima University, 1-1-1 Kagamiyama, Higashi-hiroshima, Hiroshima, 739-8524, Japan.
| | - Kentaro Katahira
- Department of Cognitive and Psychological Sciences, Graduate School of Informatics, Nagoya University, Furo, Chikusa, Nagoya, Aichi, 464-8601, Japan
| | - Makoto Hirakawa
- Graduate School of Humanities and Social Sciences, Hiroshima University, 1-1-1 Kagamiyama, Higashi-hiroshima, Hiroshima, 739-8524, Japan
| | - Takashi Nakao
- Graduate School of Humanities and Social Sciences, Hiroshima University, 1-1-1 Kagamiyama, Higashi-hiroshima, Hiroshima, 739-8524, Japan
| |
Collapse
|
10
|
Rmus M, Pan TF, Xia L, Collins AGE. Artificial neural networks for model identification and parameter estimation in computational cognitive models. PLoS Comput Biol 2024; 20:e1012119. [PMID: 38748770 PMCID: PMC11132492 DOI: 10.1371/journal.pcbi.1012119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Revised: 05/28/2024] [Accepted: 04/27/2024] [Indexed: 05/28/2024] Open
Abstract
Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.
Collapse
Affiliation(s)
- Milena Rmus
- Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America
| | - Ti-Fen Pan
- Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America
| | - Liyu Xia
- Department of Mathematics, University of California, Berkeley, Berkeley, California, United States of America
| | - Anne G. E. Collins
- Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
| |
Collapse
|
11
|
Rmus M, Pan TF, Xia L, Collins AGE. Artificial neural networks for model identification and parameter estimation in computational cognitive models. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.09.14.557793. [PMID: 37767088 PMCID: PMC10521012 DOI: 10.1101/2023.09.14.557793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 09/29/2023]
Abstract
Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.
Collapse
|
12
|
Gregorová K, Eldar E, Deserno L, Reiter AMF. A cognitive-computational account of mood swings in adolescence. Trends Cogn Sci 2024; 28:290-303. [PMID: 38503636 DOI: 10.1016/j.tics.2024.02.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 02/06/2024] [Accepted: 02/12/2024] [Indexed: 03/21/2024]
Abstract
Teenagers have a reputation for being fickle, in both their choices and their moods. This variability may help adolescents as they begin to independently navigate novel environments. Recently, however, adolescent moodiness has also been linked to psychopathology. Here, we consider adolescents' mood swings from a novel computational perspective, grounded in reinforcement learning (RL). This model proposes that mood is determined by surprises about outcomes in the environment, and how much we learn from these surprises. It additionally suggests that mood biases learning and choice in a bidirectional manner. Integrating independent lines of research, we sketch a cognitive-computational account of how adolescents' mood, learning, and choice dynamics influence each other, with implications for normative and psychopathological development.
Collapse
Affiliation(s)
- Klára Gregorová
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital, Würzburg 97080, Germany; Department of Psychology, Julius-Maximilians-Universität, Würzburg 97070, Germany; German Center of Prevention Research on Mental Health, Würzburg 97080, Germany
| | - Eran Eldar
- Department of Psychology, Hebrew University of Jerusalem, Jerusalem 9190501, Israel; Department of Cognitive & Brain Sciences, Hebrew University of Jerusalem, Jerusalem 9190501, Israel
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital, Würzburg 97080, Germany; Department of Psychology, Julius-Maximilians-Universität, Würzburg 97070, Germany; Department of Cognitive & Brain Sciences, Hebrew University of Jerusalem, Jerusalem 9190501, Israel; Department of Psychiatry and Psychotherapy, Technical University of Dresden, Dresden 01069, Germany
| | - Andrea M F Reiter
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital, Würzburg 97080, Germany; Department of Psychology, Julius-Maximilians-Universität, Würzburg 97070, Germany; German Center of Prevention Research on Mental Health, Würzburg 97080, Germany; Collaborative Research Centre 940 Volition and Cognitive Control, Technical University of Dresden, Dresden 01069, Germany.
| |
Collapse
|
13
|
Heffer T, Flournoy JC, Baum GL, Somerville LH. Examining the Association between Punishment and Reward Sensitivity and Response Inhibition to Previously-Incentivized Cues across Development. J Youth Adolesc 2024:10.1007/s10964-024-01966-z. [PMID: 38499821 DOI: 10.1007/s10964-024-01966-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 02/23/2024] [Indexed: 03/20/2024]
Abstract
Processing and learning from affective cues to guide goal-directed behavior may be particularly important during adolescence; yet the factors that promote and/or disrupt the ability to integrate value in order to guide decision making across development remain unclear. The present study (N = 1046) assessed individual difference factors (self-reported punishment and reward sensitivity) related to whether previously-rewarded and previously-punished cues differentially impact goal-directed behavior (response inhibition) in a large developmental sample. Participants were between the ages of 8-21 years (Mage = 14.29, SD = 3.97, 50.38% female). Previously-rewarded cues improved response inhibition among participants age 14 and older. Further, punishment sensitivity predicted overall improved response inhibition among participants aged 10 to 18. The results highlight two main factors that are associated with improvements in the ability to integrate value to guide goal-directed behaviour - cues in the environment (e.g., reward-laden cues) and individual differences in punishment sensitivity. These findings have implications for both educational and social policies aimed at characterizing the ways in which youth integrate value to guide decision making.
Collapse
Affiliation(s)
- Taylor Heffer
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA, 02138, USA.
| | - John C Flournoy
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA, 02138, USA
| | - Graham L Baum
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA, 02138, USA
| | - Leah H Somerville
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA, 02138, USA
| |
Collapse
|
14
|
Wilbrecht L, Davidow JY. Goal-directed learning in adolescence: neurocognitive development and contextual influences. Nat Rev Neurosci 2024; 25:176-194. [PMID: 38263216 DOI: 10.1038/s41583-023-00783-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/12/2023] [Indexed: 01/25/2024]
Abstract
Adolescence is a time during which we transition to independence, explore new activities and begin pursuit of major life goals. Goal-directed learning, in which we learn to perform actions that enable us to obtain desired outcomes, is central to many of these processes. Currently, our understanding of goal-directed learning in adolescence is itself in a state of transition, with the scientific community grappling with inconsistent results. When we examine metrics of goal-directed learning through the second decade of life, we find that many studies agree there are steady gains in performance in the teenage years, but others report that adolescent goal-directed learning is already adult-like, and some find adolescents can outperform adults. To explain the current variability in results, sophisticated experimental designs are being applied to test learning in different contexts. There is also increasing recognition that individuals of different ages and in different states will draw on different neurocognitive systems to support goal-directed learning. Through adoption of more nuanced approaches, we can be better prepared to recognize and harness adolescent strengths and to decipher the purpose (or goals) of adolescence itself.
Collapse
Affiliation(s)
- Linda Wilbrecht
- Department of Psychology, University of California, Berkeley, CA, USA.
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA.
| | - Juliet Y Davidow
- Department of Psychology, Northeastern University, Boston, MA, USA.
| |
Collapse
|
15
|
Qü AJ, Tai LH, Hall CD, Tu EM, Eckstein MK, Mishchanchuk K, Lin WC, Chase JB, MacAskill AF, Collins AG, Gershman SJ, Wilbrecht L. Nucleus accumbens dopamine release reflects Bayesian inference during instrumental learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.10.566306. [PMID: 38014354 PMCID: PMC10680647 DOI: 10.1101/2023.11.10.566306] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/29/2023]
Abstract
Dopamine release in the nucleus accumbens has been hypothesized to signal reward prediction error, the difference between observed and predicted reward, suggesting a biological implementation for reinforcement learning. Rigorous tests of this hypothesis require assumptions about how the brain maps sensory signals to reward predictions, yet this mapping is still poorly understood. In particular, the mapping is non-trivial when sensory signals provide ambiguous information about the hidden state of the environment. Previous work using classical conditioning tasks has suggested that reward predictions are generated conditional on probabilistic beliefs about the hidden state, such that dopamine implicitly reflects these beliefs. Here we test this hypothesis in the context of an instrumental task (a two-armed bandit), where the hidden state switches repeatedly. We measured choice behavior and recorded dLight signals reflecting dopamine release in the nucleus accumbens core. Model comparison based on the behavioral data favored models that used Bayesian updating of probabilistic beliefs. These same models also quantitatively matched the dopamine measurements better than non-Bayesian alternatives. We conclude that probabilistic belief computation plays a fundamental role in instrumental performance and associated mesolimbic dopamine signaling.
Collapse
Affiliation(s)
- Albert J. Qü
- Department of Psychology, University of California, Berkeley, CA, 94720, USA
- Center for Computational Biology, University of California, Berkeley, CA, 94720, USA
| | - Lung-Hao Tai
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, 94720, USA
| | - Christopher D. Hall
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, London, W1T 4JG, UK
| | - Emilie M. Tu
- Department of Psychology, University of California, Berkeley, CA, 94720, USA
| | | | - Karyna Mishchanchuk
- Department of Neuroscience, Physiology and Pharmacology, University College London, UK
| | - Wan Chen Lin
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, 94720, USA
| | - Juliana B. Chase
- Department of Psychology, University of California, Berkeley, CA, 94720, USA
| | - Andrew F. MacAskill
- Department of Neuroscience, Physiology and Pharmacology, University College London, UK
| | - Anne G.E. Collins
- Department of Psychology, University of California, Berkeley, CA, 94720, USA
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, 94720, USA
| | - Samuel J. Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA, 02138, USA
- Center for Brains, Minds, and Machines, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Linda Wilbrecht
- Department of Psychology, University of California, Berkeley, CA, 94720, USA
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, 94720, USA
| |
Collapse
|
16
|
Hollocks MJ, McQuaid GA, Yerys BE, Strang JF, Anthony LG, Kenworthy L, Lee NR, Wallace GL. Measuring flexibility in autistic adults: Exploring the factor structure of the flexibility scale self report. Autism Res 2023; 16:2208-2219. [PMID: 37698531 DOI: 10.1002/aur.3025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Accepted: 08/21/2023] [Indexed: 09/13/2023]
Abstract
Cognitive flexibility differences are common for autistic individuals and have an impact on a range of clinical outcomes. However, there is currently a lack of well validated measurement tools to assess flexibility in adulthood. The Flexibility Scale was originally designed as a parent-report measure of real-world flexibility challenges in youth. The original Flexibility Scale provides a total score and five subscales: Routines and Rituals, Transitions and Change, Special Interests, Social Flexibility, and Generativity. In this study, we evaluate the factorial validity of the Flexibility Scale as a self-report (Flexibility Scale Self Report) measure of cognitive flexibility, adapted from the original Flexibility Scale, for use by autistic adults. This study includes both a primary sample (n = 813; mean age = 40.3; 59% female) and an independently recruited replication sample (n = 120; mean age = 32.8; 74% female) of individuals who completed the Flexibility Scale Self Report. The analysis consisted of an initial confirmatory factor analysis (CFA) of the original Flexibility Scale structure, followed by exploratory factor analysis (EFA) and factor optimization within a structural equation modeling framework to identify the optimal structure for the questionnaire in adults. The identified structure was then replicated through CFA in the replication sample. Our results indicate an alternative optimal scale structure from the original Flexibility Scale, which includes fewer items, and only three (Routines/Rituals, Transitions and Change, Special Interests) of the five subscales contributing to the flexibility total score. Comparisons revealed no structural differences within the scale based on sex assigned at birth. Here the Generativity and Social Flexibility scales are treated as independent but related scales. The implications for measurement of cognitive flexibility in clinical and research settings, as well as theoretical underpinnings are discussed.
Collapse
Affiliation(s)
- Matthew J Hollocks
- Department of Child & Adolescent Psychiatry, King's College London, London, UK
| | - Goldie A McQuaid
- Department of Psychology, George Mason University, Fairfax, Virginia, USA
| | - Benjamin E Yerys
- Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - John F Strang
- Gender and Autism Program, Children's National Hospital, Rockville, Maryland, USA
| | - Laura G Anthony
- Department of Psychiatry and Behavioral Sciences, University of Colorado School of Medicine, Aurora, Colorado, USA
| | - Lauren Kenworthy
- Departments of Pediatrics, Psychiatry, and Behavioral Sciences, George Washington University School of Medicine, Washington, District of Columbia, USA
- Division of Neuropsychology, Children's National Hospital, Washington, District of Columbia, USA
| | - Nancy R Lee
- Department of Psychological and Brain Sciences, Drexel University, Philadelphia, Pennsylvania, USA
| | - Gregory L Wallace
- Department of Speech, Language, and Hearing Sciences, The George Washington University, Washington, District of Columbia, USA
| |
Collapse
|
17
|
Shi R, Xiang S, Jia T, Robbins TW, Kang J, Banaschewski T, Barker GJ, Bokde ALW, Desrivières S, Flor H, Grigis A, Garavan H, Gowland P, Heinz A, Brühl R, Martinot JL, Martinot MLP, Artiges E, Nees F, Orfanos DP, Paus T, Poustka L, Hohmann S, Millenet S, Fröhner JH, Smolka MN, Vaidya N, Walter H, Whelan R, Schumann G, Lin X, Sahakian BJ, Feng J. Structural neurodevelopment at the individual level - a life-course investigation using ABCD, IMAGEN and UK Biobank data. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.09.20.23295841. [PMID: 37790416 PMCID: PMC10543061 DOI: 10.1101/2023.09.20.23295841] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
Adolescents exhibit remarkable heterogeneity in the structural architecture of brain development. However, due to the lack of large-scale longitudinal neuroimaging studies, existing research has largely focused on population averages and the neurobiological basis underlying individual heterogeneity remains poorly understood. Using structural magnetic resonance imaging from the IMAGEN cohort (n=1,543), we show that adolescents can be clustered into three groups defined by distinct developmental patterns of whole-brain gray matter volume (GMV). Genetic and epigenetic determinants of group clustering and long-term impacts of neurodevelopment in mid-to-late adulthood were investigated using data from the ABCD, IMAGEN and UK Biobank cohorts. Group 1, characterized by continuously decreasing GMV, showed generally the best neurocognitive performances during adolescence. Compared to Group 1, Group 2 exhibited a slower rate of GMV decrease and worsened neurocognitive development, which was associated with epigenetic changes and greater environmental burden. Further, Group 3 showed increasing GMV and delayed neurocognitive development during adolescence due to a genetic variation, while these disadvantages were attenuated in mid-to-late adulthood. In summary, our study revealed novel clusters of adolescent structural neurodevelopment and suggested that genetically-predicted delayed neurodevelopment has limited long-term effects on mental well-being and socio-economic outcomes later in life. Our results could inform future research on policy interventions aimed at reducing the financial and emotional burden of mental illness.
Collapse
|
18
|
Pauli R, Brazil IA, Kohls G, Klein-Flügge MC, Rogers JC, Dikeos D, Dochnal R, Fairchild G, Fernández-Rivas A, Herpertz-Dahlmann B, Hervas A, Konrad K, Popma A, Stadler C, Freitag CM, De Brito SA, Lockwood PL. Action initiation and punishment learning differ from childhood to adolescence while reward learning remains stable. Nat Commun 2023; 14:5689. [PMID: 37709750 PMCID: PMC10502052 DOI: 10.1038/s41467-023-41124-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 08/24/2023] [Indexed: 09/16/2023] Open
Abstract
Theoretical and empirical accounts suggest that adolescence is associated with heightened reward learning and impulsivity. Experimental tasks and computational models that can dissociate reward learning from the tendency to initiate actions impulsively (action initiation bias) are thus critical to characterise the mechanisms that drive developmental differences. However, existing work has rarely quantified both learning ability and action initiation, or it has relied on small samples. Here, using computational modelling of a learning task collected from a large sample (N = 742, 9-18 years, 11 countries), we test differences in reward and punishment learning and action initiation from childhood to adolescence. Computational modelling reveals that whilst punishment learning rates increase with age, reward learning remains stable. In parallel, action initiation biases decrease with age. Results are similar when considering pubertal stage instead of chronological age. We conclude that heightened reward responsivity in adolescence can reflect differences in action initiation rather than enhanced reward learning.
Collapse
Affiliation(s)
- Ruth Pauli
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
| | - Inti A Brazil
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Gregor Kohls
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
- Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU, Dresden, Germany
| | - Miriam C Klein-Flügge
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Jack C Rogers
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Dimitris Dikeos
- Department of Psychiatry, Medical School, National and Kapodistrian University of Athens, Athens, Greece
| | - Roberta Dochnal
- Faculty of Medicine, Child and Adolescent Psychiatry, Department of the Child Health Center, Szeged University, Szeged, Hungary
| | | | | | - Beate Herpertz-Dahlmann
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
| | - Amaia Hervas
- University Hospital Mutua Terrassa, Barcelona, Spain
| | - Kerstin Konrad
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
- JARA-Brain Institute II, Molecular Neuroscience and Neuroimaging, RWTH Aachen and Research Centre Jülich, Jülich, Germany
| | - Arne Popma
- Department of Child and Adolescent Psychiatry, VU University Medical Center, Amsterdam, Netherlands
| | - Christina Stadler
- Department of Child and Adolescent Psychiatry, Psychiatric University Hospital, University of Basel, Basel, Switzerland
| | - Christine M Freitag
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital Frankfurt, Goethe University, Frankfurt am Main, Germany
| | - Stephane A De Brito
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Patricia L Lockwood
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK.
| |
Collapse
|
19
|
Tsai CG, Fu YF, Li CW. Prediction errors arising from switches between major and minor modes in music: An fMRI study. Brain Cogn 2023; 169:105987. [PMID: 37126951 DOI: 10.1016/j.bandc.2023.105987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2023] [Revised: 04/15/2023] [Accepted: 04/17/2023] [Indexed: 05/03/2023]
Abstract
The major and minor modes in Western music have positive and negative connotations, respectively. The present fMRI study examined listeners' neural responses to switches between major and minor modes. We manipulated the final chords of J. S. Bach's keyboard pieces so that each major-mode passage ended with either the major (Major-Major) or minor (Major-Minor) tonic chord, and each minor-mode passage ended with either the minor (Minor-Minor) or major (Minor-Major) tonic chord. If the final major and minor chords have positive and negative reward values respectively, the Major-Minor and Minor-Major stimuli would cause negative and positive reward prediction errors (RPEs) respectively in a listener's brain. We found that activity in a frontoparietal network was significantly higher for Major-Minor than for Major-Major. Based on previous research, these results support the idea that a major-to-minor switch causes negative RPE. The contrast of Minor-Major minus Minor-Minor yielded activation in the ventral insula and visual cortex, speaking against the idea that a minor-to-major switch causes positive RPE. We discuss our results in relation to executive functions and the emotional connotations of major versus minor modes.
Collapse
Affiliation(s)
- Chen-Gia Tsai
- Graduate Institute of Musicology, National Taiwan University, Taipei, Taiwan; Neurobiology and Cognitive Science Center, National Taiwan University, Taipei, Taiwan
| | - Yi-Fan Fu
- Department of Bio-Industry Communication and Development, National Taiwan University, Taipei, Taiwan
| | - Chia-Wei Li
- Department of Radiology, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan.
| |
Collapse
|
20
|
Topel S, Ma I, Sleutels J, van Steenbergen H, de Bruijn ERA, van Duijvenvoorde ACK. Expecting the unexpected: a review of learning under uncertainty across development. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023:10.3758/s13415-023-01098-0. [PMID: 37237092 PMCID: PMC10390612 DOI: 10.3758/s13415-023-01098-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/28/2023] [Indexed: 05/28/2023]
Abstract
Many of our decisions take place under uncertainty. To successfully navigate the environment, individuals need to estimate the degree of uncertainty and adapt their behaviors accordingly by learning from experiences. However, uncertainty is a broad construct and distinct types of uncertainty may differentially influence our learning. We provide a semi-systematic review to illustrate cognitive and neurobiological processes involved in learning under two types of uncertainty: learning in environments with stochastic outcomes, and with volatile outcomes. We specifically reviewed studies (N = 26 studies) that included an adolescent population, because adolescence is a period in life characterized by heightened exploration and learning, as well as heightened uncertainty due to experiencing many new, often social, environments. Until now, reviews have not comprehensively compared learning under distinct types of uncertainties in this age range. Our main findings show that although the overall developmental patterns were mixed, most studies indicate that learning from stochastic outcomes, as indicated by increased accuracy in performance, improved with age. We also found that adolescents tended to have an advantage compared with adults and children when learning from volatile outcomes. We discuss potential mechanisms explaining these age-related differences and conclude by outlining future research directions.
Collapse
Affiliation(s)
- Selin Topel
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands.
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands.
| | - Ili Ma
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands
| | - Jan Sleutels
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden University, Institute for Philosophy, Leiden, The Netherlands
| | - Henk van Steenbergen
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands
| | - Ellen R A de Bruijn
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands
| | - Anna C K van Duijvenvoorde
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands
| |
Collapse
|
21
|
Chierchia G, Soukupová M, Kilford EJ, Griffin C, Leung J, Palminteri S, Blakemore SJ. Confirmatory reinforcement learning changes with age during adolescence. Dev Sci 2023; 26:e13330. [PMID: 36194156 PMCID: PMC7615280 DOI: 10.1111/desc.13330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Revised: 07/26/2022] [Accepted: 09/20/2022] [Indexed: 11/26/2022]
Abstract
Understanding how learning changes during human development has been one of the long-standing objectives of developmental science. Recently, advances in computational biology have demonstrated that humans display a bias when learning to navigate novel environments through rewards and punishments: they learn more from outcomes that confirm their expectations than from outcomes that disconfirm them. Here, we ask whether confirmatory learning is stable across development, or whether it might be attenuated in developmental stages in which exploration is beneficial, such as in adolescence. In a reinforcement learning (RL) task, 77 participants aged 11-32 years (four men, mean age = 16.26) attempted to maximize monetary rewards by repeatedly sampling different pairs of novel options, which varied in their reward/punishment probabilities. Mixed-effect models showed an age-related increase in accuracy as long as learning contingencies remained stable across trials, but less so when they reversed halfway through the trials. Age was also associated with a greater tendency to stay with an option that had just delivered a reward, more than to switch away from an option that had just delivered a punishment. At the computational level, a confirmation model provided increasingly better fit with age. This model showed that age differences are captured by decreases in noise or exploration, rather than in the magnitude of the confirmation bias. These findings provide new insights into how learning changes during development and could help better tailor learning environments to people of different ages. RESEARCH HIGHLIGHTS: Reinforcement learning shows age-related improvement during adolescence, but more in stable learning environments compared with volatile learning environments. People tend to stay with an option after a win more than they shift from an option after a loss, and this asymmetry increases with age during adolescence. Computationally, these changes are captured by a developing confirmatory learning style, in which people learn more from outcomes that confirm rather than disconfirm their choices. Age-related differences in confirmatory learning are explained by decreases in stochasticity, rather than changes in the magnitude of the confirmation bias.
Collapse
Affiliation(s)
- Gabriele Chierchia
- Department of Psychology, University of Cambridge, UK
- Institute of Cognitive Neuroscience, University College London, UK
| | | | - Emma J. Kilford
- Institute of Cognitive Neuroscience, University College London, UK
- Department of Clinical, Educational and Health Psychology, University College London, UK
| | - Cait Griffin
- Institute of Cognitive Neuroscience, University College London, UK
| | - Jovita Leung
- Institute of Cognitive Neuroscience, University College London, UK
| | - Stefano Palminteri
- Institute of Cognitive Neuroscience, University College London, UK
- Department of Cognitive Science, École Normale Supérieure, FR
- Institute of Cognitive Neuroscience, HSE, Moscow, Federation of Russia
| | - Sarah-Jayne Blakemore
- Department of Psychology, University of Cambridge, UK
- Institute of Cognitive Neuroscience, University College London, UK
| |
Collapse
|
22
|
Cisler JM, Tamman AJF, Fonzo GA. Diminished prospective mental representations of reward mediate reward learning strategies among youth with internalizing symptoms. Psychol Med 2023; 53:1-11. [PMID: 36878892 PMCID: PMC10600826 DOI: 10.1017/s0033291723000478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 01/09/2023] [Accepted: 02/08/2023] [Indexed: 03/08/2023]
Abstract
BACKGROUND Adolescent internalizing symptoms and trauma exposure have been linked with altered reward learning processes and decreased ventral striatal responses to rewarding cues. Recent computational work on decision-making highlights an important role for prospective representations of the imagined outcomes of different choices. This study tested whether internalizing symptoms and trauma exposure among youth impact the generation of prospective reward representations during decision-making and potentially mediate altered behavioral strategies during reward learning. METHODS Sixty-one adolescent females with varying exposure to interpersonal violence exposure (n = 31 with histories of physical or sexual assault) and severity of internalizing symptoms completed a social reward learning task during fMRI. Multivariate pattern analyses (MVPA) were used to decode neural reward representations at the time of choice. RESULTS MVPA demonstrated that rewarding outcomes could accurately be decoded within several large-scale distributed networks (e.g. frontoparietal and striatum networks), that these reward representations were reactivated prospectively at the time of choice in proportion to the expected probability of receiving reward, and that youth with behavioral strategies that favored exploiting high reward options demonstrated greater prospective generation of reward representations. Youth internalizing symptoms, but not trauma exposure characteristics, were negatively associated with both the behavioral strategy of exploiting high reward options as well as the prospective generation of reward representations in the striatum. CONCLUSIONS These data suggest diminished prospective mental simulation of reward as a mechanism of altered reward learning strategies among youth with internalizing symptoms.
Collapse
Affiliation(s)
- Josh M. Cisler
- Department of Psychiatry and Behavioral Sciences, Dell Medical School, University of Texas at Austin, USA
- Institute for Early Life Adversity Research, Dell Medical School, University of Texas at Austin, USA
| | - Amanda J. F. Tamman
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - Greg A. Fonzo
- Department of Psychiatry and Behavioral Sciences, Dell Medical School, University of Texas at Austin, USA
- Institute for Early Life Adversity Research, Dell Medical School, University of Texas at Austin, USA
- Center for Psychedelic Research and Therapy, Dell Medical School, University of Texas at Austin, USA
| |
Collapse
|
23
|
Waltmann M, Herzog N, Reiter AMF, Villringer A, Horstmann A, Deserno L. Diminished reinforcement sensitivity in adolescence is associated with enhanced response switching and reduced coding of choice probability in the medial frontal pole. Dev Cogn Neurosci 2023; 60:101226. [PMID: 36905874 PMCID: PMC10005907 DOI: 10.1016/j.dcn.2023.101226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 03/06/2023] [Accepted: 03/06/2023] [Indexed: 03/09/2023] Open
Abstract
Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.
Collapse
Affiliation(s)
- Maria Waltmann
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
| | - Nadine Herzog
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Andrea M F Reiter
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; CRC-940 Volition and Cognitive Control, Faculty of Psychology, Technical University of Dresden, Dresden, Germany
| | - Arno Villringer
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; MindBrainBody Institute, Berlin School of Mind and Brain, Charité-Universitätsmedizin Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Annette Horstmann
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Neuroimaging Center, Technical University of Dresden, Dresden, Germany
| |
Collapse
|
24
|
Wittmann MK, Scheuplein M, Gibbons SG, Noonan MP. Local and global reward learning in the lateral frontal cortex show differential development during human adolescence. PLoS Biol 2023; 21:e3002010. [PMID: 36862726 PMCID: PMC10013901 DOI: 10.1371/journal.pbio.3002010] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 03/14/2023] [Accepted: 01/20/2023] [Indexed: 03/03/2023] Open
Abstract
Reward-guided choice is fundamental for adaptive behaviour and depends on several component processes supported by prefrontal cortex. Here, across three studies, we show that two such component processes, linking reward to specific choices and estimating the global reward state, develop during human adolescence and are linked to the lateral portions of the prefrontal cortex. These processes reflect the assignment of rewards contingently to local choices, or noncontingently, to choices that make up the global reward history. Using matched experimental tasks and analysis platforms, we show the influence of both mechanisms increase during adolescence (study 1) and that lesions to lateral frontal cortex (that included and/or disconnected both orbitofrontal and insula cortex) in human adult patients (study 2) and macaque monkeys (study 3) impair both local and global reward learning. Developmental effects were distinguishable from the influence of a decision bias on choice behaviour, known to depend on medial prefrontal cortex. Differences in local and global assignments of reward to choices across adolescence, in the context of delayed grey matter maturation of the lateral orbitofrontal and anterior insula cortex, may underlie changes in adaptive behaviour.
Collapse
Affiliation(s)
- Marco K. Wittmann
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Headington, Oxford, United Kingdom
- Department of Experimental Psychology, University College London, London, United Kingdom
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, United Kingdom
| | - Maximilian Scheuplein
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Institute of Education and Child Studies, Leiden University, Leiden, the Netherlands
| | - Sophie G. Gibbons
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
| | - MaryAnn P. Noonan
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Department of Psychology, University of York, York, United Kingdom
- * E-mail:
| |
Collapse
|
25
|
Colas JT, Dundon NM, Gerraty RT, Saragosa‐Harris NM, Szymula KP, Tanwisuth K, Tyszka JM, van Geen C, Ju H, Toga AW, Gold JI, Bassett DS, Hartley CA, Shohamy D, Grafton ST, O'Doherty JP. Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T. Hum Brain Mapp 2022; 43:4750-4790. [PMID: 35860954 PMCID: PMC9491297 DOI: 10.1002/hbm.25988] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Revised: 05/20/2022] [Accepted: 06/10/2022] [Indexed: 11/12/2022] Open
Abstract
The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.
Collapse
Affiliation(s)
- Jaron T. Colas
- Department of Psychological and Brain SciencesUniversity of CaliforniaSanta BarbaraCaliforniaUSA
- Division of the Humanities and Social SciencesCalifornia Institute of TechnologyPasadenaCaliforniaUSA
- Computation and Neural Systems Program, California Institute of TechnologyPasadenaCaliforniaUSA
| | - Neil M. Dundon
- Department of Psychological and Brain SciencesUniversity of CaliforniaSanta BarbaraCaliforniaUSA
- Department of Child and Adolescent Psychiatry, Psychotherapy, and PsychosomaticsUniversity of FreiburgFreiburg im BreisgauGermany
| | - Raphael T. Gerraty
- Department of PsychologyColumbia UniversityNew YorkNew YorkUSA
- Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkNew YorkUSA
- Center for Science and SocietyColumbia UniversityNew YorkNew YorkUSA
| | - Natalie M. Saragosa‐Harris
- Department of PsychologyNew York UniversityNew YorkNew YorkUSA
- Department of PsychologyUniversity of CaliforniaLos AngelesCaliforniaUSA
| | - Karol P. Szymula
- Department of BioengineeringUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Koranis Tanwisuth
- Division of the Humanities and Social SciencesCalifornia Institute of TechnologyPasadenaCaliforniaUSA
- Department of PsychologyUniversity of CaliforniaBerkeleyCaliforniaUSA
| | - J. Michael Tyszka
- Division of the Humanities and Social SciencesCalifornia Institute of TechnologyPasadenaCaliforniaUSA
| | - Camilla van Geen
- Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkNew YorkUSA
- Department of PsychologyUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Harang Ju
- Neuroscience Graduate GroupUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Arthur W. Toga
- Laboratory of Neuro ImagingUSC Stevens Neuroimaging and Informatics Institute, Keck School of Medicine of USC, University of Southern CaliforniaLos AngelesCaliforniaUSA
| | - Joshua I. Gold
- Department of NeuroscienceUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Dani S. Bassett
- Department of BioengineeringUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of Electrical and Systems EngineeringUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of NeurologyUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of PsychiatryUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of Physics and AstronomyUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Santa Fe InstituteSanta FeNew MexicoUSA
| | - Catherine A. Hartley
- Department of PsychologyNew York UniversityNew YorkNew YorkUSA
- Center for Neural ScienceNew York UniversityNew YorkNew YorkUSA
| | - Daphna Shohamy
- Department of PsychologyColumbia UniversityNew YorkNew YorkUSA
- Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkNew YorkUSA
- Kavli Institute for Brain ScienceColumbia UniversityNew YorkNew YorkUSA
| | - Scott T. Grafton
- Department of Psychological and Brain SciencesUniversity of CaliforniaSanta BarbaraCaliforniaUSA
| | - John P. O'Doherty
- Division of the Humanities and Social SciencesCalifornia Institute of TechnologyPasadenaCaliforniaUSA
- Computation and Neural Systems Program, California Institute of TechnologyPasadenaCaliforniaUSA
| |
Collapse
|
26
|
Young people need experiences that boost their mental health. Nature 2022; 610:253-256. [PMID: 36216911 DOI: 10.1038/d41586-022-03172-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
27
|
Nussenbaum K, Velez JA, Washington BT, Hamling HE, Hartley CA. Flexibility in valenced reinforcement learning computations across development. Child Dev 2022; 93:1601-1615. [PMID: 35596654 PMCID: PMC9831067 DOI: 10.1111/cdev.13791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Abstract
Optimal integration of positive and negative outcomes during learning varies depending on an environment's reward statistics. The present study investigated the extent to which children, adolescents, and adults (N = 142 8-25 year-olds, 55% female, 42% White, 31% Asian, 17% mixed race, and 8% Black; data collected in 2021) adapt their weighting of better-than-expected and worse-than-expected outcomes when learning from reinforcement. Participants made choices across two contexts: one in which weighting positive outcomes more heavily than negative outcomes led to better performance, and one in which the reverse was true. Reinforcement learning modeling revealed that across age, participants shifted their valence biases in accordance with environmental structure. Exploratory analyses revealed strengthening of context-dependent flexibility with increasing age.
Collapse
Affiliation(s)
| | | | | | | | - Catherine A. Hartley
- Corresponding Author: Catherine A. Hartley, Department of Psychology, New York University, 6 Washington Place, Room 871A, New York, NY, 10003.
| |
Collapse
|
28
|
The development of creative search strategies. Cognition 2022; 225:105102. [DOI: 10.1016/j.cognition.2022.105102] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2021] [Revised: 02/11/2022] [Accepted: 03/18/2022] [Indexed: 11/20/2022]
|
29
|
García-Hernández JP, Iribe-Burgos FA, Cortes PM, Sotelo-Tapia C, Guevara MA, Hernández-González M. Cortical functionality during reversal learning on a decision-making task in young men. Brain Res 2022; 1791:147998. [PMID: 35780864 DOI: 10.1016/j.brainres.2022.147998] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Revised: 06/13/2022] [Accepted: 06/28/2022] [Indexed: 11/19/2022]
Abstract
The evaluation of external and internal stimuli permits the ongoing actualization of choice-related information and, thus, the association between stimuli and outcomes. This process is essential to decision-making as it allows constant adaptation to environmental changes in order to maximize gains and minimize losses. Reversal learning paradigms are used to study this process, which has been associated with prefrontal cortex activity (frontopolar, dorsolateral) in conjunction with posterior areas (parietal, temporal), due to their participation in integrating and processing the stimuli-reward relation. The aim of this study was to assess the cortical functionality associated with reversal learning during the decision-making process. The EEG activity of 22 young men was recorded while performing a decision-making task in a reversal learning condition compared to an initial learning condition. EEG data were analyzed during evaluation of the stimuli, before motor execution (formation of preferences), and during task feedback (outcome evaluation). The formation of preferences stage was characterized by a higher correlation of the alpha2 band between the parietal cortices. In the feedback stage of the reversal learning condition, a higher absolute power of the theta band in the left dorsolateral (F3), and a lower correlation of the alpha1 band between the right frontopolar and dorsolateral (Fp2-F4), as well as between the right frontopolar and temporal (Fp2-T4), were observed. The data obtained show that the EEG activity of the areas recorded changed in the evaluation of the stimuli information in the reversal learning condition during a decision-making task.
Collapse
Affiliation(s)
| | | | - Pedro Manuel Cortes
- Instituto de Neurociencias, Universidad de Guadalajara, Guadalajara, Jalisco, Mexico
| | - Carolina Sotelo-Tapia
- Instituto de Neurociencias, Universidad de Guadalajara, Guadalajara, Jalisco, Mexico
| | - Miguel Angel Guevara
- Instituto de Neurociencias, Universidad de Guadalajara, Guadalajara, Jalisco, Mexico
| | | |
Collapse
|
30
|
Jepma M, Roy M, Ramlakhan K, van Velzen M, Dahan A. Different brain systems support learning from received and avoided pain during human pain-avoidance learning. eLife 2022; 11:74149. [PMID: 35731646 PMCID: PMC9217130 DOI: 10.7554/elife.74149] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Accepted: 06/07/2022] [Indexed: 12/14/2022] Open
Abstract
Both unexpected pain and unexpected pain absence can drive avoidance learning, but whether they do so via shared or separate neural and neurochemical systems is largely unknown. To address this issue, we combined an instrumental pain-avoidance learning task with computational modeling, functional magnetic resonance imaging (fMRI), and pharmacological manipulations of the dopaminergic (100 mg levodopa) and opioidergic (50 mg naltrexone) systems (N = 83). Computational modeling provided evidence that untreated participants learned more from received than avoided pain. Our dopamine and opioid manipulations negated this learning asymmetry by selectively increasing learning rates for avoided pain. Furthermore, our fMRI analyses revealed that pain prediction errors were encoded in subcortical and limbic brain regions, whereas no-pain prediction errors were encoded in frontal and parietal cortical regions. However, we found no effects of our pharmacological manipulations on the neural encoding of prediction errors. Together, our results suggest that human pain-avoidance learning is supported by separate threat- and safety-learning systems, and that dopamine and endogenous opioids specifically regulate learning from successfully avoided pain.
Collapse
Affiliation(s)
- Marieke Jepma
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands.,Department of Psychology, Leiden University, Leiden, Netherlands.,Leiden Institute for Brain and Cognition, Leiden, Netherlands
| | - Mathieu Roy
- Department of Psychology, McGill University, Montreal, Canada.,Alan Edwards Centre for Research on Pain, McGill University, Montreal, Canada
| | - Kiran Ramlakhan
- Department of Psychology, Leiden University, Leiden, Netherlands.,Department of Research and Statistics, Municipality of Amsterdam, Amsterdam, Netherlands
| | - Monique van Velzen
- Department of Anesthesiology, Leiden University Medical Center, Leiden, Netherlands
| | - Albert Dahan
- Department of Anesthesiology, Leiden University Medical Center, Leiden, Netherlands
| |
Collapse
|
31
|
Soares MAA, Alves AMDJT, Santos AMR, Costa GEA, Farias PKS, Fonseca VS, Martins AMEDBL. Prevalência e Caracterização do Consumo de Tabaco entre Adolescentes de Montes Claros, Minas Gerais, 2019-2020. REVISTA BRASILEIRA DE CANCEROLOGIA 2022. [DOI: 10.32635/2176-9745.rbc.2022v68n2.1919] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022] Open
Abstract
Introdução: As doenças tabaco-relacionadas são responsáveis por cinco milhões de óbitos a cada ano. Em 2020, estimou-se que o número de óbitos anuais aumentaria para dez milhões, dos quais 70% ocorreriam nos países em desenvolvimento. Objetivo: Estimar a prevalência do tabagismo e caracterizar o consumo de tabaco entre adolescentes de 12 e de 15 anos na cidade de Montes Claros, Minas Gerais, em 2019-2020. Método: Estudo epidemiológico quantitativo transversal, parte do projeto “Levantamento epidemiológico sobre condições de saúde bucal entre escolares de Montes Claros, Minas Gerais, Brasil”, realizado em 2019-2020. A amostra estimada foi de 354 escolares de 12 anos e 355 de 15 anos, calculada entre 4.458 escolares de 12 anos e 4.524 de 15 anos, respectivamente. Os dados foram analisados descritivamente por meio de medidas de tendência central, variância e frequência. Resultados: Houve prevalência do uso de tabaco entre adolescentes alguma vez na vida de 8,6 %, sendo 4,6% relativos ao consumo do cigarro Paiol. Conclusão: A prevalência do consumo encontrada entre os adolescentes e preocupante por causa dos efeitos deletérios a saúde. Conhecer o perfil dos desses usuários pode contribuir para as políticas de saúde coletiva com o objetivo de reduzir o seu uso.
Collapse
|
32
|
Chen J, Wu S, Li F. Cognitive Neural Mechanism of Backward Inhibition and Deinhibition: A Review. Front Behav Neurosci 2022; 16:846369. [PMID: 35668866 PMCID: PMC9165717 DOI: 10.3389/fnbeh.2022.846369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Accepted: 04/19/2022] [Indexed: 11/18/2022] Open
Abstract
Task switching is one of the typical paradigms to study cognitive control. When switching back to a recently inhibited task (e.g., “A” in an ABA sequence), the performance is often worse compared to a task without N-2 task repetitions (e.g., CBA). This difference is called the backward inhibitory effect (BI effect), which reflects the process of overcoming residual inhibition from a recently performed task (i.e., deinhibition). The neural mechanism of backward inhibition and deinhibition has received a lot of attention in the past decade. Multiple brain regions, including the frontal lobe, parietal, basal ganglia, and cerebellum, are activated during deinhibition. The event-related potentials (ERP) studies have shown that deinhibition process is reflected in the P1/N1 and P3 components, which might be related to early attention control, context updating, and response selection, respectively. Future research can use a variety of new paradigms to separate the neural mechanisms of BI and deinhibition.
Collapse
Affiliation(s)
- Jiwen Chen
- School of Psychology, Jiangxi Normal University, Nanchang, China
| | - Shujie Wu
- School of Psychology, Jiangxi Normal University, Nanchang, China
| | - Fuhong Li
- School of Psychology, Jiangxi Normal University, Nanchang, China
| |
Collapse
|
33
|
van Duijvenvoorde ACK, Whitmore LB, Westhoff B, Mills KL. A methodological perspective on learning in the developing brain. NPJ SCIENCE OF LEARNING 2022; 7:12. [PMID: 35654860 PMCID: PMC9163171 DOI: 10.1038/s41539-022-00127-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2020] [Accepted: 05/05/2022] [Indexed: 05/07/2023]
Abstract
The brain undergoes profound development across childhood and adolescence, including continuous changes in brain morphology, connectivity, and functioning that are, in part, dependent on one's experiences. These neurobiological changes are accompanied by significant changes in children's and adolescents' cognitive learning. By drawing from studies in the domains of reading, reinforcement learning, and learning difficulties, we present a brief overview of methodological approaches and research designs that bridge brain- and behavioral research on learning. We argue that ultimately these methods and designs may help to unravel questions such as why learning interventions work, what learning computations change across development, and how learning difficulties are distinct between individuals.
Collapse
Affiliation(s)
- Anna C K van Duijvenvoorde
- Institute of Psychology, Leiden University, Leiden, The Netherlands.
- Leiden Institute for Brain & Cognition, Leiden University, Leiden, The Netherlands.
| | - Lucy B Whitmore
- Department of Psychology, University of Oregon, Eugene, OR, USA
| | - Bianca Westhoff
- Institute of Psychology, Leiden University, Leiden, The Netherlands
- Leiden Institute for Brain & Cognition, Leiden University, Leiden, The Netherlands
| | - Kathryn L Mills
- Department of Psychology, University of Oregon, Eugene, OR, USA
- PROMENTA Research Center, Department of Psychology, University of Oslo, Oslo, Norway
| |
Collapse
|
34
|
Eckstein MK, Master SL, Dahl RE, Wilbrecht L, Collins AGE. Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal. Dev Cogn Neurosci 2022; 55:101106. [PMID: 35537273 PMCID: PMC9108470 DOI: 10.1016/j.dcn.2022.101106] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2021] [Revised: 03/01/2022] [Accepted: 03/25/2022] [Indexed: 12/02/2022] Open
Abstract
During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated how performance changes across adolescent development in a stochastic, volatile reversal-learning task that uniquely taxes the balance of persistence and flexibility. In a sample of 291 participants aged 8-30, we found that in the mid-teen years, adolescents outperformed both younger and older participants. We developed two independent cognitive models, based on Reinforcement learning (RL) and Bayesian inference (BI). The RL parameter for learning from negative outcomes and the BI parameters specifying participants' mental models were closest to optimal in mid-teen adolescents, suggesting a central role in adolescent cognitive processing. By contrast, persistence and noise parameters improved monotonically with age. We distilled the insights of RL and BI using principal component analysis and found that three shared components interacted to form the adolescent performance peak: adult-like behavioral quality, child-like time scales, and developmentally-unique processing of positive feedback. This research highlights adolescence as a neurodevelopmental window that can create performance advantages in volatile and uncertain environments. It also shows how detailed insights can be gleaned by using cognitive models in new ways.
Collapse
Affiliation(s)
| | | | - Ronald E Dahl
- Institute of Human Development, 2121 Berkeley Way West, USA
| | - Linda Wilbrecht
- Department of Psychology, 2121 Berkeley Way West, USA; Helen Wills Neuroscience Institute, 175 Li Ka Shing Center, Berkeley, CA 94720, USA
| | | |
Collapse
|
35
|
Patt VM, Palombo DJ, Esterman M, Verfaellie M. Hippocampal Contribution to Probabilistic Feedback Learning: Modeling Observation- and Reinforcement-based Processes. J Cogn Neurosci 2022; 34:1429-1446. [PMID: 35604353 DOI: 10.1162/jocn_a_01873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Simple probabilistic reinforcement learning is recognized as a striatum-based learning system, but in recent years, has also been associated with hippocampal involvement. This study examined whether such involvement may be attributed to observation-based learning (OL) processes, running in parallel to striatum-based reinforcement learning. A computational model of OL, mirroring classic models of reinforcement-based learning (RL), was constructed and applied to the neuroimaging data set of Palombo, Hayes, Reid, and Verfaellie (2019). Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia. Cognitive, Affective & Behavioral Neuroscience, 19(3), 523-536. Results suggested that OL processes may indeed take place concomitantly to reinforcement learning and involve activation of the hippocampus and central orbitofrontal cortex. However, rather than independent mechanisms running in parallel, the brain correlates of the OL and RL prediction errors indicated collaboration between systems, with direct implication of the hippocampus in computations of the discrepancy between the expected and actual reinforcing values of actions. These findings are consistent with previous accounts of a role for the hippocampus in encoding the strength of observed stimulus-outcome associations, with updating of such associations through striatal reinforcement-based computations. In addition, enhanced negative RL prediction error signaling was found in the anterior insula with greater use of OL over RL processes. This result may suggest an additional mode of collaboration between the OL and RL systems, implicating the error monitoring network.
Collapse
Affiliation(s)
- Virginie M Patt
- VA Boston Healthcare System, MA.,Boston University School of Medicine, MA
| | | | - Michael Esterman
- VA Boston Healthcare System, MA.,Boston University School of Medicine, MA
| | - Mieke Verfaellie
- VA Boston Healthcare System, MA.,Boston University School of Medicine, MA
| |
Collapse
|
36
|
Jepma M, Schaaf JV, Visser I, Huizenga HM. Impaired learning to dissociate advantageous and disadvantageous risky choices in adolescents. Sci Rep 2022; 12:6490. [PMID: 35443773 PMCID: PMC9021244 DOI: 10.1038/s41598-022-10100-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2021] [Accepted: 03/25/2022] [Indexed: 11/09/2022] Open
Abstract
Adolescence is characterized by a surge in maladaptive risk-taking behaviors, but whether and how this relates to developmental changes in experience-based learning is largely unknown. In this preregistered study, we addressed this issue using a novel task that allowed us to separate the learning-driven optimization of risky choice behavior over time from overall risk-taking tendencies. Adolescents (12-17 years old) learned to dissociate advantageous from disadvantageous risky choices less well than adults (20-35 years old), and this impairment was stronger in early than mid-late adolescents. Computational modeling revealed that adolescents' suboptimal performance was largely due to an inefficiency in core learning and choice processes. Specifically, adolescents used a simpler, suboptimal, expectation-updating process and a more stochastic choice policy. In addition, the modeling results suggested that adolescents, but not adults, overvalued the highest rewards. Finally, an exploratory latent-mixture model analysis indicated that a substantial proportion of the participants in each age group did not engage in experience-based learning but used a gambler's fallacy strategy, stressing the importance of analyzing individual differences. Our results help understand why adolescents tend to make more, and more persistent, maladaptive risky decisions than adults when the values of these decisions have to be learned from experience.
Collapse
Affiliation(s)
- Marieke Jepma
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands.
| | - Jessica V Schaaf
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
| | - Ingmar Visser
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
| | - Hilde M Huizenga
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
37
|
Rosenbaum GM, Grassie HL, Hartley CA. Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory. eLife 2022; 11:e64620. [PMID: 35072624 PMCID: PMC8786311 DOI: 10.7554/elife.64620] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Accepted: 12/24/2021] [Indexed: 12/12/2022] Open
Abstract
As individuals learn through trial and error, some are more influenced by good outcomes, while others weight bad outcomes more heavily. Such valence biases may also influence memory for past experiences. Here, we examined whether valence asymmetries in reinforcement learning change across adolescence, and whether individual learning asymmetries bias the content of subsequent memory. Participants ages 8-27 learned the values of 'point machines,' after which their memory for trial-unique images presented with choice outcomes was assessed. Relative to children and adults, adolescents overweighted worse-than-expected outcomes during learning. Individuals' valence biases modulated incidental memory, such that those who prioritized worse- (or better-) than-expected outcomes during learning were also more likely to remember images paired with these outcomes, an effect reproduced in an independent dataset. Collectively, these results highlight age-related changes in the computation of subjective value and demonstrate that a valence-asymmetric valuation process influences how information is prioritized in episodic memory.
Collapse
Affiliation(s)
- Gail M Rosenbaum
- Department of Psychology, New York UniversityNew YorkUnited States
| | - Hannah L Grassie
- Department of Psychology, New York UniversityNew YorkUnited States
| | - Catherine A Hartley
- Department of Psychology, New York UniversityNew YorkUnited States
- Center for Neural Science, New York UniversityNew YorkUnited States
| |
Collapse
|
38
|
Letkiewicz AM, Cochran AL, Cisler JM. Frontoparietal network activity during model-based reinforcement learning updates is reduced among adolescents with severe sexual abuse. J Psychiatr Res 2022; 145:256-262. [PMID: 33199053 PMCID: PMC8195295 DOI: 10.1016/j.jpsychires.2020.10.046] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/16/2020] [Revised: 10/10/2020] [Accepted: 10/31/2020] [Indexed: 01/03/2023]
Abstract
Trauma and trauma-related disorders are characterized by impaired learning processes, including reinforcement learning (RL). Identifying which aspects of learning are altered by trauma is critical endeavor, as this may reveal key mechanisms of impairment and potential intervention targets. There are at least two types of RL that have been delineated using computational modeling: model-free and model-based RL. Although these RL processes differentially predict decision-making behavior, most research has examined the impact of trauma on model-free RL. Currently unclear whether model-based RL, which involves building abstract and nuanced representations of stimulus-outcome relationships, is impaired among individuals with a history of trauma. The present study sought to test the hypothesis of impaired model-based RL among adolescent females exposed to assaultive trauma. Participants (n = 60; 29 without a history of assault and 31 with a history of assault with and without PTSD) completed a three-arm bandit task during fMRI acquisition. Two computational models compared the degree to which participants' task behavior fit the use of a model-free versus model-based RL strategy. Although a history of assaultive trauma did not predict poorer model-based RL, greater sexual abuse severity predicted less use of model-based compared to model-free RL. Additionally, severe sexual abuse predicted less left frontoparietal network encoding of model-based RL updates. Altered model-based RL, which supports goal-directed behavior, may be an important route through which clinical impairment emerges among individuals with a history of severe sexual abuse and should be examined further in future studies.
Collapse
Affiliation(s)
- Allison M. Letkiewicz
- Department of Psychiatry and Behavioral Sciences, Northwestern University, Chicago, IL, USA
| | - Amy L. Cochran
- Departments of Math and Population Health Sciences, University of Wisconsin, Madison, WI, USA
| | - Josh M. Cisler
- Department of Psychiatry, University of Wisconsin, Madison, WI, USA
| |
Collapse
|
39
|
Marzuki AA, Tomić I, Ip SHY, Gottwald J, Kanen JW, Kaser M, Sule A, Conway-Morris A, Sahakian BJ, Robbins TW. Association of Environmental Uncertainty With Altered Decision-making and Learning Mechanisms in Youths With Obsessive-Compulsive Disorder. JAMA Netw Open 2021; 4:e2136195. [PMID: 34842925 PMCID: PMC8630570 DOI: 10.1001/jamanetworkopen.2021.36195] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Accepted: 10/01/2021] [Indexed: 02/05/2023] Open
Abstract
Importance Adults with obsessive-compulsive disorder (OCD) display perseverative behavior in stable environments but exhibit vacillating choice when payoffs are uncertain. These findings may be associated with intolerance of uncertainty and compulsive behaviors; however, little is known about the mechanisms underlying learning and decision-making in youths with OCD because research into this population has been limited. Objective To investigate cognitive mechanisms associated with decision-making in youths with OCD by using executive functioning tasks and computational modeling. Design, Setting, and Participants In this cross-sectional study, 50 youths with OCD (patients) and 53 healthy participants (controls) completed a probabilistic reversal learning (PRL) task between January 2014 and March 2020. A separate sample of 27 patients and 46 controls completed the Wisconsin Card Sorting Task (WCST) between January 2018 and November 2020. The study took place at the University of Cambridge in the UK. Main Outcomes and Measures Decision-making mechanisms were studied by fitting hierarchical bayesian reinforcement learning models to the 2 data sets and comparing model parameters between participant groups. Model parameters included reward and punishment learning rates (feedback sensitivity), reinforcement sensitivity and decision consistency (exploitation), and stickiness (perseveration). Associations of receipt of serotonergic medication with performance were assessed. Results In total, 50 patients (29 female patients [58%]; median age, 16.6 years [IQR, 15.3-18.0 years]) and 53 controls (30 female participants [57%]; median age, 16.4 years [IQR, 14.8-18.0 years]) completed the PRL task. A total of 27 patients (18 female patients [67%]; median age, 16.1 years [IQR, 15.2-17.2 years]) and 46 controls (28 female participants [61%]; median age, 17.2 [IQR, 16.3-17.6 years]) completed the WCST. During the reversal phase of the PRL task, patients made fewer correct responses (mean [SD] proportion: 0.83 [0.16] for controls and 0.61 [0.31] for patients; 95% CI, -1.31 to -0.64) and switched choices more often following false-negative feedback (mean [SD] proportion: 0.09 [0.16] for controls vs 0.27 [0.34] for patients; 95% CI, 0.60-1.26) and true-positive feedback (mean [SD] proportion: 0.93 [0.17] for controls vs 0.73 [0.34] for patients; 95% CI, -2.17 to -1.31). Computational modeling revealed that patients displayed enhanced reward learning rates (mean difference [MD], 0.21; 95% highest density interval [HDI], 0.04-0.38) but decreased punishment learning rates (MD, -0.29; 95% HDI, -0.39 to -0.18), reinforcement sensitivity (MD, -4.91; 95% HDI, -9.38 to -1.12), and stickiness (MD, -0.35; 95% HDI, -0.57 to -0.11) compared with controls. There were no group differences on standard WCST measures and computational model parameters. However, patients who received serotonergic medication showed slower response times (mean [SD], 1420.49 [279.71] milliseconds for controls, 1471.42 [212.81] milliseconds for patients who were unmedicated, and 1738.25 [349.23] milliseconds for patients who were medicated) (control vs medicated MD, -320.26 [95% CI, -547.00 to -88.68]) and increased unique errors (mean [SD] proportion: 0.001 [0.004] for controls, 0.002 [0.004] for patients who were unmedicated, and 0.008 [0.01] for patients who were medicated) (control vs medicated MD, -0.007 [95% CI, -3.14 to -0.36]) on the WCST. Conclusions and Relevance The results of this cross-sectional study indicated that youths with OCD showed atypical probabilistic reversal learning but were generally unimpaired on the deterministic WCST, although unexpected results were observed for patients receiving serotonergic medication. These findings have implications for reframing the understanding of early-onset OCD as a disorder in which decision-making is associated with uncertainty in the environment, a potential target for therapeutic treatment. These results provide continuity with findings in adults with OCD.
Collapse
Affiliation(s)
- Aleya A. Marzuki
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, United Kingdom
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
- International University Malaya–Wales, Kuala Lumpur, Malaysia
| | - Ivan Tomić
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| | - Samantha Hiu Yan Ip
- Department of Public Health and Primary Care, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom
| | - Julia Gottwald
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, United Kingdom
- Department of Psychiatry, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom
| | - Jonathan W. Kanen
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, United Kingdom
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| | - Muzaffer Kaser
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, United Kingdom
- Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, United Kingdom
| | - Akeem Sule
- Department of Psychiatry, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom
| | - Anna Conway-Morris
- Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, United Kingdom
| | - Barbara J. Sahakian
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, United Kingdom
- Department of Psychiatry, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom
| | - Trevor W. Robbins
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, United Kingdom
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
40
|
Increased Ventromedial Prefrontal Cortex Activity in Adolescence Benefits Prosocial Reinforcement Learning. Dev Cogn Neurosci 2021; 52:101018. [PMID: 34678671 PMCID: PMC8529395 DOI: 10.1016/j.dcn.2021.101018] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2021] [Revised: 09/01/2021] [Accepted: 09/30/2021] [Indexed: 02/03/2023] Open
Abstract
Learning which of our behaviors benefit others contributes to forming social relationships. An important period for the development of (pro)social behavior is adolescence, which is characterized by transitions in social connections. It is, however, unknown how learning to benefit others develops across adolescence and what the underlying cognitive and neural mechanisms are. In this functional neuroimaging study, we assessed learning for self and others (i.e., prosocial learning) and the concurring neural tracking of prediction errors across adolescence (ages 9-21, N = 74). Participants performed a two-choice probabilistic reinforcement learning task in which outcomes resulted in monetary consequences for themselves, an unknown other, or no one. Participants from all ages were able to learn for themselves and others, but learning for others showed a more protracted developmental trajectory. Prediction errors for self were observed in the ventral striatum and showed no age-related differences. However, prediction error coding for others showed an age-related increase in the ventromedial prefrontal cortex. These results reveal insights into the computational mechanisms of learning for others across adolescence, and highlight that learning for self and others show different age-related patterns.
Collapse
|
41
|
Enkhtaivan E, Nishimura J, Ly C, Cochran AL. A Competition of Critics in Human Decision-Making. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2021; 5:81-101. [PMID: 38773993 PMCID: PMC11104313 DOI: 10.5334/cpsy.64] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Accepted: 07/19/2021] [Indexed: 11/20/2022]
Abstract
Recent experiments and theories of human decision-making suggest positive and negative errors are processed and encoded differently by serotonin and dopamine, with serotonin possibly serving to oppose dopamine and protect against risky decisions. We introduce a temporal difference (TD) model of human decision-making to account for these features. Our model involves two critics, an optimistic learning system and a pessimistic learning system, whose predictions are integrated in time to control how potential decisions compete to be selected. Our model predicts that human decision-making can be decomposed along two dimensions: the degree to which the individual is sensitive to (1) risk and (2) uncertainty. In addition, we demonstrate that the model can learn about the mean and standard deviation of rewards, and provide information about reaction time despite not modeling these variables directly. Lastly, we simulate a recent experiment to show how updates of the two learning systems could relate to dopamine and serotonin transients, thereby providing a mathematical formalism to serotonin's hypothesized role as an opponent to dopamine. This new model should be useful for future experiments on human decision-making.
Collapse
Affiliation(s)
| | - Joel Nishimura
- School of Mathematical and Natural Sciences, Arizona State University, Glendale, AZ, US
| | - Cheng Ly
- Department of Statistical Sciences and Operations Research, Virginia Commonwealth University, Richmond, VA, US
| | - Amy L. Cochran
- Department of Mathematics, University of Wisconsin, Madison, WI, US
- Department of Population Health Sciences, University of Wisconsin, Madison, WI, US
| |
Collapse
|
42
|
Dannenhoffer CA, Robertson MM, Macht VA, Mooney SM, Boettiger CA, Robinson DL. Chronic alcohol exposure during critical developmental periods differentially impacts persistence of deficits in cognitive flexibility and related circuitry. INTERNATIONAL REVIEW OF NEUROBIOLOGY 2021; 160:117-173. [PMID: 34696872 DOI: 10.1016/bs.irn.2021.07.004] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Cognitive flexibility in decision making depends on prefrontal cortical function and is used by individuals to adapt to environmental changes in circumstances. Cognitive flexibility can be measured in the laboratory using a variety of discrete, translational tasks, including those that involve reversal learning and/or set-shifting ability. Distinct components of flexible behavior rely upon overlapping brain circuits, including different prefrontal substructures that have separable impacts on decision making. Cognitive flexibility is impaired after chronic alcohol exposure, particularly during development when the brain undergoes rapid maturation. This review examines how cognitive flexibility, as indexed by reversal and set-shifting tasks, is impacted by chronic alcohol exposure in adulthood, adolescent, and prenatal periods in humans and animal models. We also discuss areas for future study, including mechanisms that may contribute to the persistence of cognitive deficits after developmental alcohol exposure and the compacting consequences from exposure across multiple critical periods.
Collapse
Affiliation(s)
- C A Dannenhoffer
- Bowles Center for Alcohol Studies, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - M M Robertson
- Department of Psychology and Neuroscience, University of North Carolina, Chapel Hill, NC, United States
| | - Victoria A Macht
- Bowles Center for Alcohol Studies, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - S M Mooney
- Nutrition Research Institute and Department of Nutrition, University of North Carolina, Chapel Hill, NC, United States
| | - C A Boettiger
- Bowles Center for Alcohol Studies, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States; Department of Psychology and Neuroscience, University of North Carolina, Chapel Hill, NC, United States; Neuroscience Curriculum, University of North Carolina, Chapel Hill, NC, United States; Biomedical Research Imaging Center, University of North Carolina, Chapel Hill, NC, United States
| | - Donita L Robinson
- Bowles Center for Alcohol Studies, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States; Department of Psychiatry, University of North Carolina, Chapel Hill, NC, United States; Neuroscience Curriculum, University of North Carolina, Chapel Hill, NC, United States.
| |
Collapse
|
43
|
Jepma M, Schaaf JV, Visser I, Huizenga HM. Effects of advice on experienced-based learning in adolescents and adults. J Exp Child Psychol 2021; 211:105230. [PMID: 34256185 DOI: 10.1016/j.jecp.2021.105230] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Revised: 03/24/2021] [Accepted: 06/16/2021] [Indexed: 01/04/2023]
Abstract
Recent studies that compared effects of pre-learning advice on experience-based learning in adolescents and adults have yielded mixed results. Previous studies on this topic used choice tasks in which age-related differences in advice-related learning bias and exploratory choice behavior are difficult to dissociate. Moreover, these studies did not examine whether effects of advice depend on working memory load. In this preregistered study (in adolescents [13-15 years old] and adults [18-31 years old]), we addressed these issues by factorially combining advice and working memory load manipulations in an estimation task that does not require choices and hence eliminates the influence of known age-related differences in exploration. We found that advice guided participants' initial estimates in both age groups. When advice was correct, this improved estimation performance, especially in adolescents when working memory load was high. When advice was incorrect, it had a longer-lasting effect on adolescents' performance than on adults' performance. In contrast to previous findings in choice tasks, we found no evidence that advice biased learning in either age group. Taken together, our results suggest that learning in an estimation task improves between adolescence and adulthood but that the effects of advice on learning do not differ substantially between adolescents and adults.
Collapse
Affiliation(s)
- Marieke Jepma
- Department of Psychology, University of Amsterdam, 1012 WX Amsterdam, the Netherlands.
| | - Jessica V Schaaf
- Department of Psychology, University of Amsterdam, 1012 WX Amsterdam, the Netherlands
| | - Ingmar Visser
- Department of Psychology, University of Amsterdam, 1012 WX Amsterdam, the Netherlands
| | - Hilde M Huizenga
- Department of Psychology, University of Amsterdam, 1012 WX Amsterdam, the Netherlands
| |
Collapse
|
44
|
Hartung SL, Mandonnet E, de Witt Hamer P, Klein M, Wager M, Rech F, Pallud J, Pessanha Viegas C, Ille S, Krieg SM, Robe PA, van Zandvoort MJE. Impaired Set-Shifting from Dorsal Stream Disconnection: Insights from a European Series of Right Parietal Lower-Grade Glioma Resection. Cancers (Basel) 2021; 13:cancers13133337. [PMID: 34283043 PMCID: PMC8267741 DOI: 10.3390/cancers13133337] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2021] [Revised: 06/18/2021] [Accepted: 06/24/2021] [Indexed: 11/16/2022] Open
Abstract
Simple Summary Awake surgery with cognitive monitoring has increasingly been implemented to preserve brain networks and functionality. More recently, not only surgery in the left but also in the right hemisphere, i.c., the parietal lobe, was associated with potential risk for deficits in cognitive functions, such as cognitive flexibility. We describe an explorative pilot study in an international consortium within clinical care as usual. Careful interpretation of our findings indicates that disconnection of the lateral part of the dorsal stream correlated with impaired set-shifting. More importantly, it illustrates the need for international collaboration on neuropsychological tests and methodologies to improve our understanding of white matter networks at risk during awake surgery. Abstract Awake surgery with cognitive monitoring has increasingly been implemented to preserve brain networks and functionality. More recently, not only surgery in the left but also in the right hemisphere, i.c., the parietal lobe, was associated with potential risk for deficits in cognitive functions, such as cognitive flexibility. In this explorative pilot study, we compare cognitive performance more than three months after surgery with baseline measurements and explore the association between cognitive decline and subcortical tracts that may have been severed during surgery in the right hemisphere. Twenty-two patients who underwent surgery for a right parietal low-grade glioma were assessed pre- and postoperatively using the Trail Making Test and the Stroop task to administer set-shifting abilities and inhibition. Volume measurements and lesion–symptom mapping analyses were performed on postoperative MRI scans. Careful interpretation of the results shows a change in TMT performance and not on the Stroop Task when the lateral part of the arcuate fasciculus is damaged, indicating that disconnection of the lateral part of the dorsal stream might be correlated specifically with impaired set-shifting and not with inhibition. More importantly, this study underlines the need for international concertation to allow larger studies to increase power and perform more detailed analyses.
Collapse
Affiliation(s)
- Suzanne L. Hartung
- Department of Neurology and Neurosurgery, University Medical Center Utrecht, 3584 CX Utrecht, The Netherlands; (P.A.R.); (M.J.E.v.Z.)
- Correspondence:
| | | | - Philip de Witt Hamer
- Department of Neurosurgery, Location VUmc, Cancer Center Amsterdam, Amsterdam UMC, 1081 HV Amsterdam, The Netherlands;
| | - Martin Klein
- Department of Medical Psychology and Brain Tumor Center Amsterdam at Amsterdam UMC, Vrije Universiteit Amsterdam, 1081 HV Amsterdam, The Netherlands;
| | - Michel Wager
- Department of Neurological Surgery, Poitiers University Hospital, 86021 Poitiers, France;
| | - Fabien Rech
- CHRU-Nancy, Service de Neurochirurgie, Université de Lorraine, F-54000 Nancy, France;
- CNRS, CRAN, Université de Lorraine, F-54000 Nancy, France
| | - Johan Pallud
- Department of Neursurgery, Saint-Anne Hospital, 75014 Paris, France;
| | | | - Sebastian Ille
- Department of Neurosurgery, School of Medicine, Klinikum rechts der Isar, Technical University Munich, 80333 Munich, Germany; (S.I.); (S.M.K.)
| | - Sandro M. Krieg
- Department of Neurosurgery, School of Medicine, Klinikum rechts der Isar, Technical University Munich, 80333 Munich, Germany; (S.I.); (S.M.K.)
| | - Pierre A. Robe
- Department of Neurology and Neurosurgery, University Medical Center Utrecht, 3584 CX Utrecht, The Netherlands; (P.A.R.); (M.J.E.v.Z.)
| | - Martine J. E. van Zandvoort
- Department of Neurology and Neurosurgery, University Medical Center Utrecht, 3584 CX Utrecht, The Netherlands; (P.A.R.); (M.J.E.v.Z.)
- Department of Experimental Psychology, Utrecht University, 3584 CS Utrecht, The Netherlands
| |
Collapse
|
45
|
Xia L, Master SL, Eckstein MK, Baribault B, Dahl RE, Wilbrecht L, Collins AGE. Modeling changes in probabilistic reinforcement learning during adolescence. PLoS Comput Biol 2021; 17:e1008524. [PMID: 34197447 PMCID: PMC8279421 DOI: 10.1371/journal.pcbi.1008524] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Revised: 07/14/2021] [Accepted: 05/26/2021] [Indexed: 01/17/2023] Open
Abstract
In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggest probabilistic learning may be inefficient in youths compared to adults, while others suggest it may be more efficient in youths in mid adolescence. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants' performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time scale); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development.
Collapse
Affiliation(s)
- Liyu Xia
- Department of Mathematics, University of California Berkeley, Berkeley, California, United States of America
| | - Sarah L. Master
- Department of Psychology, New York University, New York, New York, United States of America
| | - Maria K. Eckstein
- Department of Psychology, University of California Berkeley, Berkeley, California, United States of America
| | - Beth Baribault
- Department of Psychology, University of California Berkeley, Berkeley, California, United States of America
| | - Ronald E. Dahl
- School of Public Health, University of California Berkeley, Berkeley, California, United States of America
| | - Linda Wilbrecht
- Department of Psychology, University of California Berkeley, Berkeley, California, United States of America
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, United States of America
| | - Anne Gabrielle Eva Collins
- Department of Psychology, University of California Berkeley, Berkeley, California, United States of America
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, United States of America
| |
Collapse
|
46
|
Willinger D, Karipidis II, Neuer S, Emery S, Rauch C, Häberling I, Berger G, Walitza S, Brem S. Maladaptive Avoidance Learning in the Orbitofrontal Cortex in Adolescents With Major Depression. BIOLOGICAL PSYCHIATRY: COGNITIVE NEUROSCIENCE AND NEUROIMAGING 2021; 7:293-301. [PMID: 34144217 DOI: 10.1016/j.bpsc.2021.06.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/29/2020] [Revised: 04/30/2021] [Accepted: 06/08/2021] [Indexed: 11/25/2022]
Abstract
BACKGROUND Understanding the mechanisms in the brain's incentive network that give rise to symptoms of major depressive disorder (MDD) during adolescence provides new perspectives to address MDD in early stages of development. This functional magnetic resonance imaging study determines whether instrumental vigor and brain responses to appetitive and aversive monetary incentives are altered in adolescent MDD and associated with symptom severity. METHODS Adolescents with moderate to severe MDD (n = 30, mean age [SD] = 16.1 [1.4] years) and healthy control subjects (n = 33, mean age = 16.2 [1.9] years) matched for age, sex, and IQ performed a monetary incentive delay task. During outcome presentation, prediction error signals were used to study the response and coupling of the incentive network during learning of cue-outcome associations. A computational reinforcement model was used to assess adaptation of response vigor. Brain responses and effective connectivity to model-derived prediction errors were assessed and related to depression severity and anhedonia levels. RESULTS Participants with MDD behaved according to a more simplistic learning model and exhibited slower learning. Effective connectivity analysis of functional magnetic resonance imaging data revealed that impaired loss error processing in the orbitofrontal cortex was associated with aberrant gain control. Anhedonia scores correlated with loss-related error signals in the posterior insula and habenula. CONCLUSIONS Adolescent MDD is selectively related to impaired processing of error signals during loss, but not reward, in the orbitofrontal cortex. Aberrant evaluation of loss outcomes might reflect an early mechanism of how negative bias and helplessness manifest in the brain. This approach sheds light on pathomechanisms in MDD and may improve early diagnosis and treatment selection.
Collapse
Affiliation(s)
- David Willinger
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Iliana I Karipidis
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland; Center for Interdisciplinary Brain Sciences Research, Department of Psychiatry and Behavioral Sciences, School of Medicine, Stanford University, Stanford, California
| | - Selina Neuer
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland
| | - Sophie Emery
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland
| | - Carolina Rauch
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland
| | - Isabelle Häberling
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland
| | - Gregor Berger
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland
| | - Susanne Walitza
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Silvia Brem
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Hospital of Psychiatry Zurich, University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland.
| |
Collapse
|
47
|
Neurodevelopment of the incentive network facilitates motivated behaviour from adolescence to adulthood. Neuroimage 2021; 237:118186. [PMID: 34020019 DOI: 10.1016/j.neuroimage.2021.118186] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Revised: 05/11/2021] [Accepted: 05/17/2021] [Indexed: 12/28/2022] Open
Abstract
The ability to enhance motivated performance through incentives is crucial to guide and ultimately optimise the outcome of goal-directed behaviour. It remains largely unclear how motivated behaviour and performance develops particularly across adolescence. Here, we used computational fMRI to assess how response speed and its underlying neural circuitry are modulated by reward and loss in a monetary incentive delay paradigm. We demonstrate that maturational fine-tuning of functional coupling within the cortico-striatal incentive circuitry from adolescence to adulthood facilitates the ability to enhance performance selectively for higher subjective values. Additionally, during feedback, we found developmental sex differences of striatal representations of reward prediction errors in an exploratory analysis. Our findings suggest that a reduced capacity to utilise subjective value for motivated behaviour in adolescence is rooted in immature information processing in the incentive system. This indicates that the neurocircuitry for coordination of incentivised, motivated cognitive control acts as a bottleneck for behavioural adjustments in adolescence.
Collapse
|
48
|
Abstract
Cognitive and behavioural flexibility permit the appropriate adjustment of thoughts and behaviours in response to changing environmental demands. Brain mechanisms enabling flexibility have been examined using non-invasive neuroimaging and behavioural approaches in humans alongside pharmacological and lesion studies in animals. This work has identified large-scale functional brain networks encompassing lateral and orbital frontoparietal, midcingulo-insular and frontostriatal regions that support flexibility across the lifespan. Flexibility can be compromised in early-life neurodevelopmental disorders, clinical conditions that emerge during adolescence and late-life dementias. We critically evaluate evidence for the enhancement of flexibility through cognitive training, physical activity and bilingual experience.
Collapse
Affiliation(s)
- Lucina Q Uddin
- Department of Psychology, University of Miami, Coral Gables, FL, USA.
- Neuroscience Program, University of Miami Miller School of Medicine, Miami, FL, USA.
| |
Collapse
|
49
|
Bowler A, Habicht J, Moses-Payne ME, Steinbeis N, Moutoussis M, Hauser TU. Children perform extensive information gathering when it is not costly. Cognition 2021; 208:104535. [PMID: 33370652 PMCID: PMC7871012 DOI: 10.1016/j.cognition.2020.104535] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Revised: 11/27/2020] [Accepted: 12/03/2020] [Indexed: 01/21/2023]
Abstract
Humans often face decisions where little is known about the choice options. Gathering information prior to making a choice is an important strategy to improve decision making under uncertainty. This is of particular importance during childhood and adolescence, when knowledge about the world is still limited. To examine how much information youths gather, we asked 107 children (8-9 years, N = 30), early (12-13 years, N = 41) and late adolescents (16-17 years, N = 36) to perform an information sampling task. We find that children gather significantly more information before making a decision compared to adolescents, but only if it does not come with explicit costs. Using computational modelling, we find that this is because children have reduced subjective costs for gathering information. Our findings thus demonstrate how children overcome their limited knowledge and neurocognitive constraints by deploying excessive information gathering, a developmental feature that could inform aberrant information gathering in psychiatric disorders.
Collapse
Affiliation(s)
- Aislinn Bowler
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, United Kingdom; Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, United Kingdom; Centre for Brain and Cognitive Development, Birkbeck, University of London, London WC1E 7HX, United Kingdom
| | - Johanna Habicht
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, United Kingdom; Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, United Kingdom
| | | | - Niko Steinbeis
- Division of Psychology and Language Sciences, University College London, London WC1H 0AP, United Kingdom
| | - Michael Moutoussis
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, United Kingdom; Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, United Kingdom
| | - Tobias U Hauser
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, United Kingdom; Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, United Kingdom.
| |
Collapse
|
50
|
Gracia-Tabuenca Z, Moreno MB, Barrios FA, Alcauter S. Development of the brain functional connectome follows puberty-dependent nonlinear trajectories. Neuroimage 2021; 229:117769. [PMID: 33482398 DOI: 10.1016/j.neuroimage.2021.117769] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Revised: 12/18/2020] [Accepted: 01/05/2021] [Indexed: 01/06/2023] Open
Abstract
Adolescence is a developmental period that dramatically impacts body and behavior, with pubertal hormones playing an important role not only in the morphological changes in the body but also in brain structure and function. Understanding brain development during adolescence has become a priority in neuroscience because it coincides with the onset of many psychiatric and behavioral disorders. However, little is known about how puberty influences the brain functional connectome. In this study, taking a longitudinal human sample of typically developing children and adolescents (of both sexes), we demonstrate that the development of the brain functional connectome better fits pubertal status than chronological age. In particular, centrality, segregation, efficiency, and integration of the brain functional connectome increase after the onset of the pubertal markers. We found that these effects are stronger in attention and task control networks. Lastly, after controlling for this effect, we showed that functional connectivity between these networks is related to better performance in cognitive flexibility. This study points out the importance of considering longitudinal nonlinear trends when exploring developmental trajectories, and emphasizes the impact of puberty on the functional organization of the brain in adolescence.
Collapse
Affiliation(s)
- Zeus Gracia-Tabuenca
- Instituto de Neurobiología, Universidad Nacional Autónoma de México, Blvd Juriquilla 3001, Querétaro 76230, Mexico
| | - Martha Beatriz Moreno
- Instituto de Neurobiología, Universidad Nacional Autónoma de México, Blvd Juriquilla 3001, Querétaro 76230, Mexico
| | - Fernando A Barrios
- Instituto de Neurobiología, Universidad Nacional Autónoma de México, Blvd Juriquilla 3001, Querétaro 76230, Mexico
| | - Sarael Alcauter
- Instituto de Neurobiología, Universidad Nacional Autónoma de México, Blvd Juriquilla 3001, Querétaro 76230, Mexico.
| |
Collapse
|