1
|
Do KT, Paolizzi SG, Hallquist MN. How adolescents learn to build social bonds: A developmental computational account of social explore-exploit decision-making. Dev Cogn Neurosci 2024; 69:101415. [PMID: 39089173 DOI: 10.1016/j.dcn.2024.101415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Revised: 07/01/2024] [Accepted: 07/12/2024] [Indexed: 08/03/2024] Open
Abstract
Building social bonds is a critical task of adolescence that affords opportunities for learning, identity formation, and social support. Failing to develop close relationships in adolescence hinders adult interpersonal functioning and contributes to problems such as loneliness and depression. During adolescence, increased reward sensitivity and greater social flexibility both contribute to healthy social development, yet we lack a clear theory of how these processes interact to support social functioning. Here, we propose synthesizing these two literatures using a computational reinforcement learning framework that recasts how adolescents pursue and learn from social rewards as a social explore-exploit problem. To become socially skilled, adolescents must balance both their efforts to form individual bonds within specific groups and manage memberships across multiple groups to maximize access to social resources. We draw on insights from sociological studies on social capital in collective networks and neurocognitive research on foraging and cooperation to describe the social explore-exploit dilemma faced by adolescents navigating a modern world with increasing access to diverse resources and group memberships. Our account provides important new directions for examining the dynamics of adolescent behavior in social groups and understanding how social value computations can support positive relationships into adulthood.
Collapse
Affiliation(s)
- Kathy T Do
- Department of Psychology and Neuroscience, University of North Carolina Chapel Hill, 235 E. Cameron Avenue, Chapel Hill, NC 27599-3270, United States.
| | - Sophie G Paolizzi
- Department of Psychology and Neuroscience, University of North Carolina Chapel Hill, 235 E. Cameron Avenue, Chapel Hill, NC 27599-3270, United States
| | - Michael N Hallquist
- Department of Psychology and Neuroscience, University of North Carolina Chapel Hill, 235 E. Cameron Avenue, Chapel Hill, NC 27599-3270, United States
| |
Collapse
|
2
|
Chung YS, van den Berg B, Roberts KC, Woldorff MG, Gaffrey MS. Electrical brain activations in preadolescents during a probabilistic reward-learning task reflect cognitive processes and behavioral strategy. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.10.16.562326. [PMID: 37905129 PMCID: PMC10614771 DOI: 10.1101/2023.10.16.562326] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/02/2023]
Abstract
Both adults and children learn through feedback which environmental events and choices are associated with higher probability of reward, an ability thought to be supported by the development of fronto-striatal reward circuits. Recent developmental studies have applied computational models of reward learning to investigate such learning in children. However, tasks and measures effective for assaying the cascade of reward-learning neural processes in children have been limited. Using a child-version of a probabilistic reward-learning task while recording event-related-potential (ERP) measures of electrical brain activity, this study examined key processes of reward learning in preadolescents (8-12 years old; n=30), namely: (1) reward-feedback sensitivity, as measured by the early-latency, reward-related, frontal ERP positivity, (2) rapid attentional shifting of processing toward favored visual stimuli, as measured by the N2pc component, and (3) longer-latency attention-related responses to reward feedback as a function of behavioral strategies (i.e., Win-Stay-Lose-Shift), as measured by the central-parietal P300. Consistent with our prior work in adults, the behavioral findings indicate preadolescents can learn stimulus-reward outcome associations, but at varying levels of performance. Neurally, poor preadolescent learners (those with slower learning rates) showed greater reward-related positivity amplitudes relative to good learners, suggesting greater reward-feedback sensitivity. We also found attention shifting towards to-be-chosen stimuli, as evidenced by the N2pc, but not to more highly rewarded stimuli as we have observed in adults. Lastly, we found the behavioral learning strategy (i.e., Win-Stay-Lose-Shift) reflected by the feedback-elicited parietal P300. These findings provide novel insights into the key neural processes underlying reinforcement learning in preadolescents.
Collapse
Affiliation(s)
- Yu Sun Chung
- Department of Psychology and Neuroscience, Duke University, Reuben-Cooke Building, 417 Chapel Drive, Durham, NC 27708, USA
| | | | - Kenneth C. Roberts
- Center for Cognitive Neuroscience, Department of Psychiatry, Psychology & Neuroscience and Neurobiology, Duke University, Durham, NC, 27708 USA
| | - Marty G. Woldorff
- Department of Psychology and Neuroscience, Duke University, Reuben-Cooke Building, 417 Chapel Drive, Durham, NC 27708, USA
- Center for Cognitive Neuroscience, Department of Psychiatry, Psychology & Neuroscience and Neurobiology, Duke University, Durham, NC, 27708 USA
| | - Michael S. Gaffrey
- Department of Psychology and Neuroscience, Duke University, Reuben-Cooke Building, 417 Chapel Drive, Durham, NC 27708, USA
- Children’s Wisconsin, 9000 W. Wisconsin Avenue, Milwaukee, WI, 53226
- Medical College of Wisconsin, Division of Pediatric Psychology and Developmental Medicine, Department of Pediatrics, 8701 Watertown Plank Road, Milwaukee, WI, 53226
| |
Collapse
|
3
|
Schreuders E, van Buuren M, Walsh RJ, Sijtsma H, Hollarek M, Lee NC, Krabbendam L. Learning whom not to trust across early and middle adolescence: A longitudinal neuroimaging study to trusting behavior involving an uncooperative other. Child Dev 2024; 95:368-390. [PMID: 37583272 DOI: 10.1111/cdev.13986] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Revised: 07/04/2023] [Accepted: 07/16/2023] [Indexed: 08/17/2023]
Abstract
Longitudinal changes in trusting behavior across adolescence and their neural correlates were examined. Neural regions of interest (ROIs) included the medial prefrontal cortex (mPFC), dorsal anterior cingulate cortex (dACC), left anterior insula (AI), bilateral ventral striatum (VS), and right dorsal striatum (DS). Participants (wave 1 age: M = 12.90) played the investor in a Trust Game with an uncooperative trustee three times (1-year interval). Analyses included 77 primarily Dutch participants (33 females). Participants decreased their investments with wave. Furthermore, activity was heightened in mPFC, dACC, and DS during investment and repayment, and in right VS (investment) and AI (repayment). Finally, DS activity during repayment increased with wave. These findings highlight early-middle adolescence as an important period for developing sensitivity to uncooperative behavior.
Collapse
Affiliation(s)
- E Schreuders
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit, Amsterdam, The Netherlands
- Department of Developmental and Educational Psychology, Leiden University, Leiden, The Netherlands
| | - M van Buuren
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit, Amsterdam, The Netherlands
| | - R J Walsh
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit, Amsterdam, The Netherlands
| | - H Sijtsma
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit, Amsterdam, The Netherlands
| | - M Hollarek
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit, Amsterdam, The Netherlands
| | - N C Lee
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit, Amsterdam, The Netherlands
- Department of Developmental Psychology, Utrecht University, Utrecht, The Netherlands
| | - L Krabbendam
- Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit, Amsterdam, The Netherlands
| |
Collapse
|
4
|
Wilbrecht L, Davidow JY. Goal-directed learning in adolescence: neurocognitive development and contextual influences. Nat Rev Neurosci 2024; 25:176-194. [PMID: 38263216 DOI: 10.1038/s41583-023-00783-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/12/2023] [Indexed: 01/25/2024]
Abstract
Adolescence is a time during which we transition to independence, explore new activities and begin pursuit of major life goals. Goal-directed learning, in which we learn to perform actions that enable us to obtain desired outcomes, is central to many of these processes. Currently, our understanding of goal-directed learning in adolescence is itself in a state of transition, with the scientific community grappling with inconsistent results. When we examine metrics of goal-directed learning through the second decade of life, we find that many studies agree there are steady gains in performance in the teenage years, but others report that adolescent goal-directed learning is already adult-like, and some find adolescents can outperform adults. To explain the current variability in results, sophisticated experimental designs are being applied to test learning in different contexts. There is also increasing recognition that individuals of different ages and in different states will draw on different neurocognitive systems to support goal-directed learning. Through adoption of more nuanced approaches, we can be better prepared to recognize and harness adolescent strengths and to decipher the purpose (or goals) of adolescence itself.
Collapse
Affiliation(s)
- Linda Wilbrecht
- Department of Psychology, University of California, Berkeley, CA, USA.
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA.
| | - Juliet Y Davidow
- Department of Psychology, Northeastern University, Boston, MA, USA.
| |
Collapse
|
5
|
van Duijvenvoorde ACK, Whitmore LB, Westhoff B, Mills KL. A methodological perspective on learning in the developing brain. NPJ SCIENCE OF LEARNING 2022; 7:12. [PMID: 35654860 PMCID: PMC9163171 DOI: 10.1038/s41539-022-00127-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2020] [Accepted: 05/05/2022] [Indexed: 05/07/2023]
Abstract
The brain undergoes profound development across childhood and adolescence, including continuous changes in brain morphology, connectivity, and functioning that are, in part, dependent on one's experiences. These neurobiological changes are accompanied by significant changes in children's and adolescents' cognitive learning. By drawing from studies in the domains of reading, reinforcement learning, and learning difficulties, we present a brief overview of methodological approaches and research designs that bridge brain- and behavioral research on learning. We argue that ultimately these methods and designs may help to unravel questions such as why learning interventions work, what learning computations change across development, and how learning difficulties are distinct between individuals.
Collapse
Affiliation(s)
- Anna C K van Duijvenvoorde
- Institute of Psychology, Leiden University, Leiden, The Netherlands.
- Leiden Institute for Brain & Cognition, Leiden University, Leiden, The Netherlands.
| | - Lucy B Whitmore
- Department of Psychology, University of Oregon, Eugene, OR, USA
| | - Bianca Westhoff
- Institute of Psychology, Leiden University, Leiden, The Netherlands
- Leiden Institute for Brain & Cognition, Leiden University, Leiden, The Netherlands
| | - Kathryn L Mills
- Department of Psychology, University of Oregon, Eugene, OR, USA
- PROMENTA Research Center, Department of Psychology, University of Oslo, Oslo, Norway
| |
Collapse
|
6
|
Pechtel P, Harris J, Karl A, Clunies-Ross C, Bower S, Moberly NJ, Pizzagalli DA, Watkins ER. Emerging ecophenotype: reward anticipation is linked to high-risk behaviours after sexual abuse. Soc Cogn Affect Neurosci 2022; 17:1035-1043. [PMID: 35438797 PMCID: PMC9629466 DOI: 10.1093/scan/nsac030] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Revised: 02/28/2022] [Accepted: 04/19/2022] [Indexed: 01/12/2023] Open
Abstract
Adolescents frequently engage in high-risk behaviours (HRB) following childhood sexual abuse (CSA). Aberrant reward processes are implicated in HRB, and their underlying fronto-striatal networks are vulnerable to neurodevelopmental changes during adversity representing a promising candidate for understanding links between CSA and HRB. We examined whether fronto-striatal responses during reward anticipation and feedback (i) are altered in depressed adolescents with CSA compared to depressed, non-abused peers and (ii) moderate the relationship between CSA and HRB irrespective of depression. Forty-eight female adolescents {14 with CSA and depression [CSA + major depressive disorder (MDD)]; 17 with MDD but no CSA (MDD); 17 healthy, non-abused controls} completed a monetary reward task during functional magnetic resonance imaging. No differences in fronto-striatal response to reward emerged between CSA + MDD and MDD. Critically, high left nucleus accumbens activation during reward anticipation was associated with greater HRB in CSA + MDD compared to MDD and controls. Low left putamen activation during reward feedback was associated with the absence of HRB in CSA + MDD compared to MDD. Striatal reward responses appear to play a key role in HRB for adolescents with CSA irrespective of depression, providing initial support for a CSA ecophenotype. Such information is pivotal to identify at-risk youth and prevent HRB in adolescents after CSA.
Collapse
Affiliation(s)
- Pia Pechtel
- Correspondence should be addressed to Pia Pechtel, Department of Psychology, University of Exeter, Sir Henry Wellcome Building for Mood Disorders Research, Perry Road, Exeter EX4 4QQ, UK. E-mail:
| | - Jennifer Harris
- Department of Psychology, College of Life and Environmental Sciences (CLES), University of Exeter, Exeter EX4 4QG, UK
| | - Anke Karl
- Department of Psychology, College of Life and Environmental Sciences (CLES), University of Exeter, Exeter EX4 4QG, UK
| | - Caroline Clunies-Ross
- Child and Adolescent Mental Health Services, Children and Family Health Devon, Exeter EX2 4NU, UK
| | - Susie Bower
- Child and Adolescent Mental Health Services, Children and Family Health Devon, Exeter EX2 4NU, UK
| | - Nicholas J Moberly
- Department of Psychology, College of Life and Environmental Sciences (CLES), University of Exeter, Exeter EX4 4QG, UK
| | - Diego A Pizzagalli
- Center for Depression, Anxiety and Stress Research, McLean Hospital, Belmont MA 02478, USA,Department of Psychiatry, Harvard Medical School, Boston, MA 02215, USA,McLean Imaging Center, McLean Hospital, Belmont, MA 02478 USA
| | - Edward R Watkins
- Department of Psychology, College of Life and Environmental Sciences (CLES), University of Exeter, Exeter EX4 4QG, UK
| |
Collapse
|
7
|
Eckstein MK, Master SL, Dahl RE, Wilbrecht L, Collins AG. Reinforcement learning and bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal. Dev Cogn Neurosci 2022; 55:101106. [PMID: 35537273 PMCID: PMC9108470 DOI: 10.1016/j.dcn.2022.101106] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2021] [Revised: 03/01/2022] [Accepted: 03/25/2022] [Indexed: 12/02/2022] Open
Abstract
During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated how performance changes across adolescent development in a stochastic, volatile reversal-learning task that uniquely taxes the balance of persistence and flexibility. In a sample of 291 participants aged 8–30, we found that in the mid-teen years, adolescents outperformed both younger and older participants. We developed two independent cognitive models, based on Reinforcement learning (RL) and Bayesian inference (BI). The RL parameter for learning from negative outcomes and the BI parameters specifying participants’ mental models were closest to optimal in mid-teen adolescents, suggesting a central role in adolescent cognitive processing. By contrast, persistence and noise parameters improved monotonically with age. We distilled the insights of RL and BI using principal component analysis and found that three shared components interacted to form the adolescent performance peak: adult-like behavioral quality, child-like time scales, and developmentally-unique processing of positive feedback. This research highlights adolescence as a neurodevelopmental window that can create performance advantages in volatile and uncertain environments. It also shows how detailed insights can be gleaned by using cognitive models in new ways.
Collapse
|
8
|
Calabrese JR, Goetschius LG, Murray L, Kaplan MR, Lopez-Duran N, Mitchell C, Hyde LW, Monk CS. Mapping frontostriatal white matter tracts and their association with reward-related ventral striatum activation in adolescence. Brain Res 2022; 1780:147803. [PMID: 35090884 DOI: 10.1016/j.brainres.2022.147803] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Revised: 01/17/2022] [Accepted: 01/21/2022] [Indexed: 11/21/2022]
Abstract
The ventral striatum (VS) is implicated in reward processing and motivation. Human and non-human primate studies demonstrate that the VS and prefrontal cortex (PFC), which comprise the frontostriatal circuit, interact to influence motivated behavior. However, there is a lack of research that precisely maps and quantifies VS-PFC white matter tracts. Moreover, no studies have linked frontostriatal white matter to VS activation. Using a multimodal neuroimaging approach with diffusion MRI (dMRI) and functional MRI (fMRI), the present study had two objectives: 1) to chart white matter tracts between the VS and specific PFC structures and 2) assess the association between the degree of VS-PFC white matter tract connectivity and VS activation in 187 adolescents. White matter connectivity was assessed with probabilistic tractography and functional activation was examined with two fMRI tasks (one task with social reward and another task using monetary reward). We found widespread but variable white matter connectivity between the VS and areas of the PFC, with the anterior insula and subgenual cingulate cortex demonstrating the greatest degree of connectivity with the VS. VS-PFC structural connectivity was related to functional activation in the VS though activation depended on the specific PFC region and reward task.
Collapse
Affiliation(s)
| | | | - Laura Murray
- Department of Psychology, University of Michigan, Ann Arbor, MI, USA; McLean Hospital, Harvard Medical School, Belmont, MA, USA
| | - Megan R Kaplan
- Department of Psychology, University of Michigan, Ann Arbor, MI, USA
| | | | - Colter Mitchell
- Institute for Social Research, University of Michigan, Ann Arbor, MI, USA; Survey Research Center of the Institute for Social Research, University of Michigan, Ann Arbor, MI, USA; Population Studies Center of the Institute for Social Research, University of Michigan, Ann Arbor, MI, USA
| | - Luke W Hyde
- Department of Psychology, University of Michigan, Ann Arbor, MI, USA; Survey Research Center of the Institute for Social Research, University of Michigan, Ann Arbor, MI, USA
| | - Christopher S Monk
- Department of Psychology, University of Michigan, Ann Arbor, MI, USA; Survey Research Center of the Institute for Social Research, University of Michigan, Ann Arbor, MI, USA; Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, USA; Department of Psychiatry, University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
9
|
McKeon PN, Bunce GW, Patton MH, Chen R, Mathur BN. Cortical control of striatal fast-spiking interneuron synchrony. J Physiol 2022; 600:2189-2202. [PMID: 35332539 PMCID: PMC9058232 DOI: 10.1113/jp282850] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Accepted: 03/16/2022] [Indexed: 11/08/2022] Open
Abstract
KEY POINTS Electrical synapses between striatal fast-spiking interneurons in adult mice occur in ∼8% of assayed pairs. Coincident, convergent cortical input onto fast-spiking interneurons significantly contributes to fast-spiking interneuron synchrony Electrical synapses between fast-spiking interneurons provide only minor enhancement of fast-spiking interneuron synchrony. These results suggest a mechanism by which adult mouse fast-spiking interneurons of the striatum synchronize in the face of declining expression of the electrical synapse-forming connexin-36 protein. ABSTRACT Inhibitory fast-spiking interneurons in the dorsal striatum regulate actions and action strategies, including habits. Fast-spiking interneurons are widely believed to synchronize their firing due to the electrical synapses formed between these neurons. However, neuronal modeling data suggest convergent cortical input may also drive synchrony in fast-spiking interneuron networks. To better understand how fast-spiking interneuron synchrony arises, we performed dual whole-cell patch clamp electrophysiology experiments to inform a simple Bayesian network modeling cortico-fast-spiking interneuron circuitry. Dual whole-cell patch clamp electrophysiology revealed that while responsivity to corticostriatal input activation was high in fast-spiking interneurons, few of these neurons exhibited electrical coupling in adult mice. In simulations of a cortico-fast-spiking interneuron network informed by these data, the degree of glutamatergic cortical convergence onto fast-spiking interneurons significantly increased fast-spiking interneuron synchronization while manipulations of electrical coupling between these neurons exerted relatively little impact. These results suggest that the primary source of functional coordination of fast-spiking interneuron activity in adulthood arises from convergent corticostriatal input activation. Abstract figure legend Dual whole-cell patch clamp recordings of dorsal striatal fast-spiking interneurons (FSIs; red circles) rarely (8 percentage) form electrical synapses with other FSIs in adult mouse. In a two-layer in silico model of cortical pyramidal neuron (gray triangles) input to FSIs using empirically defined cortico-FSI synaptic weights, synchronous FSI-FSI activity (in the absence of abundant electrical synapses) is achievable by convergent cortical pyramidal excitation of FSIs. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Paige N McKeon
- Department of Pharmacology, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Garrett W Bunce
- Department of Pharmacology, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Mary H Patton
- St. Jude Children's Research Hospital, Memphis, TN, USA
| | - Rong Chen
- Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Brian N Mathur
- Department of Pharmacology, University of Maryland School of Medicine, Baltimore, MD, USA
| |
Collapse
|
10
|
Eckstein MK, Master SL, Xia L, Dahl RE, Wilbrecht L, Collins AGE. The interpretation of computational model parameters depends on the context. eLife 2022; 11:75474. [PMID: 36331872 PMCID: PMC9635876 DOI: 10.7554/elife.75474] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Accepted: 09/09/2022] [Indexed: 11/06/2022] Open
Abstract
Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g. tasks, models) and that they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8–30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. Still, generalization was significantly below the methodological ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g. reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models.
Collapse
Affiliation(s)
| | - Sarah L Master
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States,Department of Psychology, New York UniversityNew YorkUnited States
| | - Liyu Xia
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States,Department of Mathematics, University of California, BerkeleyBerkeleyUnited States
| | - Ronald E Dahl
- Institute of Human Development, University of California, BerkeleyBerkeleyUnited States
| | - Linda Wilbrecht
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States,Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| | - Anne GE Collins
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States,Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| |
Collapse
|
11
|
Galván A. Adolescent Brain Development and Contextual Influences: A Decade in Review. JOURNAL OF RESEARCH ON ADOLESCENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR RESEARCH ON ADOLESCENCE 2021; 31:843-869. [PMID: 34820955 DOI: 10.1111/jora.12687] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Adolescence is a developmental period characterized by substantial psychological, biological, and neurobiological changes. This review discusses the past decade of research on the adolescent brain, as based on the overarching framework that development is a dynamic process both within the individual and between the individual and external inputs. As such, this review focuses on research showing that the development of the brain is influenced by multiple ongoing and dynamic elements. It highlights the implications this body of work on behavioral development and offers areas of opportunity for future research in the coming decade.
Collapse
|
12
|
Grant AD, Wilbrecht L, Kriegsfeld LJ. Adolescent Development of Biological Rhythms in Female Rats: Estradiol Dependence and Effects of Combined Contraceptives. Front Physiol 2021; 12:752363. [PMID: 35615288 PMCID: PMC9126190 DOI: 10.3389/fphys.2021.752363] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Accepted: 10/11/2021] [Indexed: 01/23/2023] Open
Abstract
Adolescence is a period of continuous development, including the maturation of endogenous rhythms across systems and timescales. Although, these dynamic changes are well-recognized, their continuous structure and hormonal dependence have not been systematically characterized. Given the well-established link between core body temperature (CBT) and reproductive hormones in adults, we hypothesized that high-resolution CBT can be applied to passively monitor pubertal development and disruption with high fidelity. To examine this possibility, we used signal processing to investigate the trajectory of CBT rhythms at the within-day (ultradian), daily (circadian), and ovulatory timescales, their dependence on estradiol (E2), and the effects of hormonal contraceptives. Puberty onset was marked by a rise in fecal estradiol (fE2), followed by an elevation in CBT and circadian power. This time period marked the commencement of 4-day rhythmicity in fE2, CBT, and ultradian power marking the onset of the estrous cycle. The rise in circadian amplitude was accelerated by E2 treatment, indicating a role for this hormone in rhythmic development. Contraceptive administration in later adolescence reduced CBT and circadian power and resulted in disruption to 4-day cycles that persisted after discontinuation. Our data reveal with precise temporal resolution how biological rhythms change across adolescence and demonstrate a role for E2 in the emergence and preservation of multiscale rhythmicity. These findings also demonstrate how hormones delivered exogenously in a non-rhythmic pattern can disrupt rhythmic development. These data lay the groundwork for a future in which temperature metrics provide an inexpensive, convenient method for monitoring pubertal maturation and support the development of hormone therapies that better mimic and support human chronobiology.
Collapse
Affiliation(s)
- Azure D. Grant
- The Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, United States
| | - Linda Wilbrecht
- The Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, United States
- Department of Psychology, University of California, Berkeley, Berkeley, CA, United States
| | - Lance J. Kriegsfeld
- The Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, United States
- Department of Psychology, University of California, Berkeley, Berkeley, CA, United States
- Department of Integrative Biology, University of California, Berkeley, Berkeley, CA, United States
- Graduate Group in Endocrinology, University of California, Berkeley, Berkeley, CA, United States
| |
Collapse
|
13
|
Eckstein MK, Wilbrecht L, Collins AGE. What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience. Curr Opin Behav Sci 2021; 41:128-137. [PMID: 34984213 PMCID: PMC8722372 DOI: 10.1016/j.cobeha.2021.06.004] [Citation(s) in RCA: 37] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Reinforcement learning (RL) is a concept that has been invaluable to fields including machine learning, neuroscience, and cognitive science. However, what RL entails differs between fields, leading to difficulties when interpreting and translating findings. After laying out these differences, this paper focuses on cognitive (neuro)science to discuss how we as a field might over-interpret RL modeling results. We too often assume-implicitly-that modeling results generalize between tasks, models, and participant populations, despite negative empirical evidence for this assumption. We also often assume that parameters measure specific, unique (neuro)cognitive processes, a concept we call interpretability, when evidence suggests that they capture different functions across studies and tasks. We conclude that future computational research needs to pay increased attention to implicit assumptions when using RL models, and suggest that a more systematic understanding of contextual factors will help address issues and improve the ability of RL to explain brain and behavior.
Collapse
Affiliation(s)
- Maria K Eckstein
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
| | - Linda Wilbrecht
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
- Helen Wills Neuroscience Institute, UC Berkeley, 175 Li Ka Shing Center, Berkeley, 94720, CA, USA
| | - Anne G E Collins
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
- Helen Wills Neuroscience Institute, UC Berkeley, 175 Li Ka Shing Center, Berkeley, 94720, CA, USA
| |
Collapse
|
14
|
Tashjian SM, Galván A. Frontopolar Cortex Response to Positive Feedback Relates to Nonincentivized Task Persistence. Cereb Cortex 2021; 32:2293-2309. [PMID: 34581407 DOI: 10.1093/cercor/bhab317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023] Open
Abstract
When individuals make decisions whether to persist at a task, their decision-making is informed by whether success is pending or accomplished. If pending, the brain facilitates behavioral persistence; if the goal is accomplished or no longer desired, the brain enables switching away from the current task. Feedback, which is known to differentially engage reward neurocircuitry, may modulate goal-directed behavior such as task persistence. However, prior studies are confounded by offering external incentives for persistence. This study tested whether neural response to feedback differed as a function of nonincentivized task persistence in 99 human participants ages 13-30 (60 females). Individuals who persisted engaged the frontopolar cortex (FPC) to a greater extent during receipt of task-relevant positive feedback compared with negative feedback. For individuals who quit, task-irrelevant monetary reward engaged the FPC to a greater extent compared with positive feedback. FPC activation in response to positive feedback is identified as a key contributor to task persistence.
Collapse
Affiliation(s)
- Sarah M Tashjian
- Department of Psychology, University of California at Los Angeles, Los Angeles, CA 90095, USA
| | - Adriana Galván
- Department of Psychology, University of California at Los Angeles, Los Angeles, CA 90095, USA.,Brain Research Institute, University of California at Los Angeles, Los Angeles, CA 90095, USA
| |
Collapse
|
15
|
Xie S, Zhang X, Cheng W, Yang Z. Adolescent anxiety disorders and the developing brain: comparing neuroimaging findings in adolescents and adults. Gen Psychiatr 2021; 34:e100411. [PMID: 34423252 PMCID: PMC8340272 DOI: 10.1136/gpsych-2020-100411] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Accepted: 06/21/2021] [Indexed: 02/07/2023] Open
Abstract
Adolescence is the peak period for the incidence of anxiety disorders. Recent findings have revealed the immaturity of neural networks underlying emotional regulation in this population. Brain vulnerability to anxiety in adolescence is related to the unsynchronised development of anxiety-relevant brain functional systems. However, our current knowledge on brain deficits in adolescent anxiety is mainly borrowed from studies on adults. Understanding adolescent-specific brain deficits is essential for developing biomarkers and brain-based therapies targeting adolescent anxiety. This article reviews and compares recent neuroimaging literature on anxiety-related brain structural and functional deficits between adolescent and adult populations, and proposes a model highlighting the differences between adolescence and adulthood in anxiety-related brain networks. This model emphasises that in adolescence the emotional control system tends to be hypoactivated, the fear conditioning system is immature, and the reward and stress response systems are hypersensitive. Furthermore, the striatum’s functional links to the amygdala and the prefrontal cortex are strengthened, while the link between the prefrontal cortex and the amygdala is weakened in adolescence. This model helps to explain why adolescents are vulnerable to anxiety disorders and provides insights into potential brain-based approaches to intervene in adolescent anxiety disorders.
Collapse
Affiliation(s)
- Shuqi Xie
- Laboratory of Psychological Health and Imaging, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xiaochen Zhang
- Laboratory of Psychological Health and Imaging, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Wenhong Cheng
- Department of Child and Adolescent Psychiatry, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
- Department of Psychological Medicine, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Zhi Yang
- Laboratory of Psychological Health and Imaging, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
- Institute of Psychological and Behavioral Sciences, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
16
|
Xia L, Master SL, Eckstein MK, Baribault B, Dahl RE, Wilbrecht L, Collins AGE. Modeling changes in probabilistic reinforcement learning during adolescence. PLoS Comput Biol 2021; 17:e1008524. [PMID: 34197447 PMCID: PMC8279421 DOI: 10.1371/journal.pcbi.1008524] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Revised: 07/14/2021] [Accepted: 05/26/2021] [Indexed: 01/17/2023] Open
Abstract
In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggest probabilistic learning may be inefficient in youths compared to adults, while others suggest it may be more efficient in youths in mid adolescence. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants' performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time scale); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development.
Collapse
Affiliation(s)
- Liyu Xia
- Department of Mathematics, University of California Berkeley, Berkeley, California, United States of America
| | - Sarah L. Master
- Department of Psychology, New York University, New York, New York, United States of America
| | - Maria K. Eckstein
- Department of Psychology, University of California Berkeley, Berkeley, California, United States of America
| | - Beth Baribault
- Department of Psychology, University of California Berkeley, Berkeley, California, United States of America
| | - Ronald E. Dahl
- School of Public Health, University of California Berkeley, Berkeley, California, United States of America
| | - Linda Wilbrecht
- Department of Psychology, University of California Berkeley, Berkeley, California, United States of America
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, United States of America
| | - Anne Gabrielle Eva Collins
- Department of Psychology, University of California Berkeley, Berkeley, California, United States of America
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, United States of America
| |
Collapse
|
17
|
Noschang C, Lampert C, Krolow R, de Almeida RMM. Social isolation at adolescence: a systematic review on behaviour related to cocaine, amphetamine and nicotine use in rats and mice. Psychopharmacology (Berl) 2021; 238:927-947. [PMID: 33606060 DOI: 10.1007/s00213-021-05777-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/01/2020] [Accepted: 01/24/2021] [Indexed: 02/07/2023]
Abstract
Adolescence is known for its high level of risk-taking, and neurobiological alterations during this period may predispose to psychoactive drug initiation and progression into more severe use patterns. Stress is a risk factor for drug consumption, and post-weaning social isolation increases drug self-administration in rodents. This review aimed to provide an overview of the effects of adolescent social isolation on cocaine, amphetamine and nicotine use-related behaviours, highlighting the specific period when animals were submitted to stress and these drugs. We wondered if there was a specific period during adolescence that isolation stress would increase drug use vulnerability. A total of 323 publications from the Scopus, Web of Science and PubMed (Medline) electronic databases were identified using the words "social isolation" and "adolescence" and "drug" or "cocaine" or "amphetamine" or "nicotine", resulting in 24 articles after analyses criteria following the PRISMA statement. The main points raised were social isolation during adolescence increased cocaine self-administration, amphetamine and nicotine locomotor activity. We did not observe a pattern of a specific moment during the adolescent period that could lead to an increased vulnerability to drug use. The precise conditions under which adolescent social stress alters drug use parameters are complex and likely depend on several factors.
Collapse
Affiliation(s)
- C Noschang
- Institute of Psychology, Laboratory of Experimental Psychology, Neuroscience and Behavior, Federal University of Rio Grande do Sul (UFRGS), 2600 Ramiro Barcelos St., Room 216, Porto Alegre, Rio Grande do Sul, 90035-003, Brazil.
- Biochemistry Department, Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil.
| | - C Lampert
- Institute of Psychology, Laboratory of Experimental Psychology, Neuroscience and Behavior, Federal University of Rio Grande do Sul (UFRGS), 2600 Ramiro Barcelos St., Room 216, Porto Alegre, Rio Grande do Sul, 90035-003, Brazil
- Biochemistry Department, Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
| | - R Krolow
- Biochemistry Department, Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
| | - R M M de Almeida
- Institute of Psychology, Laboratory of Experimental Psychology, Neuroscience and Behavior, Federal University of Rio Grande do Sul (UFRGS), 2600 Ramiro Barcelos St., Room 216, Porto Alegre, Rio Grande do Sul, 90035-003, Brazil
| |
Collapse
|
18
|
Towner TT, Spear LP. Rats exposed to intermittent ethanol during late adolescence exhibit enhanced habitual behavior following reward devaluation. Alcohol 2021; 91:11-20. [PMID: 33031883 DOI: 10.1016/j.alcohol.2020.09.004] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2020] [Revised: 09/15/2020] [Accepted: 09/28/2020] [Indexed: 12/29/2022]
Abstract
The brain undergoes substantial maturation during adolescence, and repeated exposure to ethanol at this time has been shown to result in long-lasting behavioral and neural consequences. During the broad period of adolescence, different neuronal populations and circuits are refined between early and late adolescence, suggesting the possibility that ethanol exposure at these differing times may lead to differential outcomes. The goal of the current study was to evaluate the impact of adolescent intermittent ethanol (AIE) during early and late adolescence on the formation of goal-directed and habitual behavior in adulthood. Male and female Sprague-Dawley rats were exposed to ethanol via intragastric gavage (4.0 g/kg, 25% v/v) every other day from postnatal day (P) 25-45 or P45-65, considered early and late adolescence, respectively. In adulthood (~P70 early or ~ P90 late), rats were gradually food-restricted and began operant training on a fixed ratio 1 schedule. Rats were then transitioned onto random interval schedules and eventually underwent a sensory-specific satiation procedure as a model of reward devaluation. Few differences as a result of adolescent ethanol exposure were found during instrumental training. Following reward devaluation, rats exposed to water and ethanol during early adolescence exhibited reductions in lever pressing, suggestive of a goal-directed response pattern. In contrast, late AIE males and females demonstrated persistent responding following both devalued and non-devalued trials, findings representative of a habitual behavior pattern. The shifts from goal-directed to habitual behavior noted only following late AIE contribute to the growing literature identifying specific behavioral consequences as a result of ethanol exposure during distinct developmental periods within adolescence. More work is needed to determine whether the greater habit formation following late AIE is also associated with elevated habitual ethanol consumption.
Collapse
Affiliation(s)
- Trevor Theodore Towner
- Neurobiology of Adolescent Drinking in Adulthood Consortium, Center for Development and Behavioral Neuroscience, Department of Psychology, Binghamton University, Binghamton, NY, 13902-6000, United States.
| | - Linda Patia Spear
- Neurobiology of Adolescent Drinking in Adulthood Consortium, Center for Development and Behavioral Neuroscience, Department of Psychology, Binghamton University, Binghamton, NY, 13902-6000, United States
| |
Collapse
|
19
|
Chahal R, Delevich K, Kirshenbaum JS, Borchers LR, Ho TC, Gotlib IH. Sex differences in pubertal associations with fronto-accumbal white matter morphometry: Implications for understanding sensitivity to reward and punishment. Neuroimage 2020; 226:117598. [PMID: 33249215 DOI: 10.1016/j.neuroimage.2020.117598] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2020] [Revised: 11/04/2020] [Accepted: 11/21/2020] [Indexed: 12/16/2022] Open
Abstract
Researchers have reported sex-differentiated maturation of white matter (WM) during puberty. It is not clear, however, whether such distinctions contribute to documented sex differences in sensitivity to reward and punishment during adolescence. Given the role of the orbitofrontal cortex (OFC) and nucleus accumbens (NAcc) in reward and punishment-related behaviors, we tested in a cross-sectional study whether males and females (N = 156, 89 females; ages 9-14 years) differ in the association between pubertal stage and fixel-based morphometry of WM fibers connecting the OFC and NAcc (i.e., the fronto-accumbal tract). Further, we examined whether males and females differ in associations between fronto-accumbal WM measures and self-reported sensitivity to reward and punishment. Pubertal stage was positively associated with fronto-accumbal fiber density and cross-section (FDC) in males, but not in females. Consistent with previous reports, males reported higher reward sensitivity than did females, although fronto-accumbal combined FDC was not related to reward sensitivity in either sex. Meanwhile, only males showed a negative association between fronto-accumbal tract FDC and sensitivity to punishment. Follow-up analyses revealed that fiber cross-section, but not density, was related to pubertal stage and punishment sensitivity in males, as well as to reward sensitivity in all participants. Our findings suggest there are sex differences in puberty-related maturation of the fronto-accumbal tract, and this tract is related to lower punishment sensitivity in adolescent males compared to females.
Collapse
Affiliation(s)
- Rajpreet Chahal
- Department of Psychology, Stanford University, 450 Jane Stanford Way, Stanford, CA 94305, United States.
| | - Kristen Delevich
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, United States
| | - Jaclyn S Kirshenbaum
- Department of Psychology, Stanford University, 450 Jane Stanford Way, Stanford, CA 94305, United States
| | - Lauren R Borchers
- Department of Psychology, Stanford University, 450 Jane Stanford Way, Stanford, CA 94305, United States
| | - Tiffany C Ho
- Department of Psychiatry and Behavioral Sciences, University of California, San Francisco, CA, United States; Weil Institute for Neurosciences, University of California, San Francisco, CA, United States
| | - Ian H Gotlib
- Department of Psychology, Stanford University, 450 Jane Stanford Way, Stanford, CA 94305, United States.
| |
Collapse
|
20
|
Delevich K, Okada NJ, Rahane A, Zhang Z, Hall CD, Wilbrecht L. Sex and Pubertal Status Influence Dendritic Spine Density on Frontal Corticostriatal Projection Neurons in Mice. Cereb Cortex 2020; 30:3543-3557. [PMID: 32037445 DOI: 10.1093/cercor/bhz325] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
In humans, nonhuman primates, and rodents, the frontal cortices exhibit grey matter thinning and dendritic spine pruning that extends into adolescence. This maturation is believed to support higher cognition but may also confer psychiatric vulnerability during adolescence. Currently, little is known about how specific cell types in the frontal cortex mature or whether puberty plays a role in the maturation of some cell types but not others. Here, we used mice to characterize the spatial topography and adolescent development of cross-corticostriatal (cSTR) neurons that project through the corpus collosum to the dorsomedial striatum. We found that apical spine density on cSTR neurons in the medial prefrontal cortex decreased significantly between late juvenile (P29) and young adult time points (P60), with females exhibiting higher spine density than males at both ages. Adult males castrated prior to puberty onset had higher spine density compared to sham controls. Adult females ovariectomized before puberty onset showed greater variance in spine density measures on cSTR cells compared to controls, but their mean spine density did not significantly differ from sham controls. Our findings reveal that these cSTR neurons, a subtype of the broader class of intratelencephalic-type neurons, exhibit significant sex differences and suggest that spine pruning on cSTR neurons is regulated by puberty in male mice.
Collapse
Affiliation(s)
- Kristen Delevich
- Department of Psychology, University of California, Berkeley, CA 94720, USA and.,Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
| | - Nana J Okada
- Department of Psychology, University of California, Berkeley, CA 94720, USA and.,Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
| | - Ameet Rahane
- Department of Psychology, University of California, Berkeley, CA 94720, USA and.,Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
| | - Zicheng Zhang
- Department of Psychology, University of California, Berkeley, CA 94720, USA and.,Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
| | - Christopher D Hall
- Department of Psychology, University of California, Berkeley, CA 94720, USA and.,Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
| | - Linda Wilbrecht
- Department of Psychology, University of California, Berkeley, CA 94720, USA and.,Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
| |
Collapse
|
21
|
Nussenbaum K, Hartley CA. Reinforcement learning across development: What insights can we draw from a decade of research? Dev Cogn Neurosci 2019; 40:100733. [PMID: 31770715 PMCID: PMC6974916 DOI: 10.1016/j.dcn.2019.100733] [Citation(s) in RCA: 72] [Impact Index Per Article: 14.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2019] [Revised: 10/24/2019] [Accepted: 11/04/2019] [Indexed: 01/02/2023] Open
Abstract
The past decade has seen the emergence of the use of reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety of tasks and model variants, have reached convergent conclusions. In this review, we examine whether the tuning of model parameters that govern different aspects of learning and decision-making processes vary consistently as a function of age, and what neurocognitive developmental changes may account for differences in these parameter estimates across development. We explore whether patterns of developmental change in these estimates are better described by differences in the extent to which individuals adapt their learning processes to the statistics of different environments, or by more static learning biases that emerge across varied contexts. We focus specifically on learning rates and inverse temperature parameter estimates, and find evidence that from childhood to adulthood, individuals become better at optimally weighting recent outcomes during learning across diverse contexts and less exploratory in their value-based decision-making. We provide recommendations for how these two possibilities - and potential alternative accounts - can be tested more directly to build a cohesive body of research that yields greater insight into the development of core learning processes.
Collapse
|
22
|
Kennedy JT, Astafiev SV, Golosheykin S, Korucuoglu O, Anokhin AP. Shared genetic influences on adolescent body mass index and brain structure: A voxel-based morphometry study in twins. Neuroimage 2019; 199:261-272. [PMID: 31163268 DOI: 10.1016/j.neuroimage.2019.05.053] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2019] [Revised: 05/17/2019] [Accepted: 05/19/2019] [Indexed: 01/22/2023] Open
Abstract
BACKGROUND Previous research has demonstrated significant relationships between obesity and brain structure. Both phenotypes are heritable, but it is not known whether they are influenced by common genetic factors. We investigated the genetic etiology of the relationship between individual variability in brain morphology and BMIz using structural MRI in adolescent twins. METHOD The sample (n = 258) consisted of 54 monozygotic and 75 dizygotic twin pairs (mean(SD) age = 13.61(0.505), BMIz = 0.608(1.013). Brain structure (volume and density of gray and white matter) was assessed using VBM. Significant voxelwise heritability of brain structure was established using the Accelerated Permutation inference for ACE models (APACE) program, with structural heritability varying from 15 to 97%, depending on region. Bivariate heritability analyses were carried out comparing additive genetic and unique environment models with and without shared genetics on BMIz and the voxels showing significant heritability in the APACE analyses. RESULTS BMIz was positively related to gray matter volume in the brainstem and thalamus and negatively related to gray matter volume in the bilateral uncus and medial orbitofrontal cortex, gray matter density in the cerebellum, prefrontal lobe, temporal lobe, and limbic system, and white matter density in the brainstem. Bivariate heritability analyses showed that BMIz and brain structure share ∼1/3 of their genes and that ∼95% of the phenotypic correlation between BMIz and brain structure is due to shared additive genetic influences. These regions included areas related to decision-making, motivation, liking vs. wanting, taste, interoception, reward processing/learning, caloric evaluation, and inhibition. CONCLUSION These results suggested genetic factors are responsible for the relationship between BMIz and heritable BMIz related brain structure in areas related to eating behavior.
Collapse
Affiliation(s)
- James T Kennedy
- Department of Psychiatry, Washington University School of Medicine, United States.
| | - Serguei V Astafiev
- Department of Psychiatry, Washington University School of Medicine, United States
| | - Semyon Golosheykin
- Department of Psychiatry, Washington University School of Medicine, United States
| | - Ozlem Korucuoglu
- Department of Psychiatry, Washington University School of Medicine, United States
| | - Andrey P Anokhin
- Department of Psychiatry, Washington University School of Medicine, United States
| |
Collapse
|
23
|
Rosenbaum GM, Hartley CA. Developmental perspectives on risky and impulsive choice. Philos Trans R Soc Lond B Biol Sci 2019; 374:20180133. [PMID: 30966918 PMCID: PMC6335462 DOI: 10.1098/rstb.2018.0133] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/20/2018] [Indexed: 12/28/2022] Open
Abstract
Epidemiological data suggest that risk taking in the real world increases from childhood into adolescence and declines into adulthood. However, developmental patterns of behaviour in laboratory assays of risk taking and impulsive choice are inconsistent. In this article, we review a growing literature using behavioural economic approaches to understand developmental changes in risk taking and impulsivity. We present findings that have begun to elucidate both the cognitive and neural processes that contribute to risky and impulsive choice, as well as how age-related changes in these neurocognitive processes give rise to shifts in choice behaviour. We highlight how variability in task parameters can be used to identify specific aspects of decision contexts that may differentially influence risky and impulsive choice behaviour across development. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications'.
Collapse
Affiliation(s)
- Gail M. Rosenbaum
- Department of Psychology, New York University, New York, NY 10003, USA
| | - Catherine A. Hartley
- Department of Psychology, New York University, New York, NY 10003, USA
- Center for Neural Science, New York University, New York, NY 10003, USA
| |
Collapse
|