Cognitive flexibility in adolescence: neural and behavioral mechanisms of reward prediction error processing in adaptive decision making during development

Adolescent-to-adult gains in cognitive flexibility are adaptively supported by reward sensitivity, exploration, and neural variability

Cognitive flexibility exhibits dynamic changes throughout development, with different forms of flexibility showing dissociable developmental trajectories. In this review, we propose that an adolescent-specific mode of flexibility in the face of changing environmental contingencies supports the emergence of adolescent-to-adult gains in cognitive shifting efficiency. We first describe how cognitive shifting abilities monotonically improve from childhood to adulthood, accompanied by increases in brain state flexibility, neural variability, and excitatory/inhibitory balance. We next summarize evidence supporting the existence of a dopamine-driven, adolescent peak in flexible behavior that results in reward seeking, undirected exploration, and environmental sampling. We propose a neurodevelopmental framework that relates these adolescent behaviors to the refinement of neural phenotypes relevant to mature cognitive flexibility, and thus highlight the importance of the adolescent period in fostering healthy neurocognitive trajectories.

Effects of adolescent intermittent ethanol exposure on cortical perineuronal net and parvalbumin expression in adulthood mediate behavioral inflexibility

BACKGROUND

Alcohol is commonly consumed by adolescents in a binge-like pattern, which can lead to long-lasting cognitive deficits, including reduced behavioral flexibility. We and others have determined that adolescent intermittent ethanol (AIE) exposure leads to increased number of perineuronal net (PNN) numbers in brain regions that are important for behavioral flexibility. However, whether altered neurochemistry stemming from AIE exposure plays a significant role in reduced behavioral flexibility is unknown.

METHODS

We measured the number and size of parvalbumin expressing (PV+) interneurons and associated PNNs within the orbitofrontal cortex (OFC), prelimbic cortex (PrL), infralimbic cortex (IL), and anterior insular cortex (AIC) of female and male rats following AIE or control exposure and subsequent training on an attentional set-shift task (ASST). We then ran analyses to determine whether AIE-induced changes in PV and PNN measures statistically mediated the AIE-induced behavioral deficit in reversal learning.

RESULTS

We demonstrate that AIE exposure impaired behavioral flexibility on reversal two of the ASST (i.e., recalling the initial learned associations), and led to smaller PV+ cells and increased PNN numbers in the AIC. Interestingly, PNN size and number were not altered in the PrL or IL following AIE exposure, in contrast to prior reports. Mediation analyses suggest that AIE alters behavioral flexibility, at least in part through changes in PV and PNN fluorescent measures in the AIC.

CONCLUSIONS

This study reveals a significant link between AIE exposure, neural alterations, and diminished behavioral flexibility in rats, and highlights a potential novel mechanism comprising changes in PV and PNN measures within the AIC. Future studies should explore the impact of PNN degradation within the AIC on behavioral flexibility.

Model of a striatal circuit exploring biological mechanisms underlying decision-making during normal and disordered states

Decision-making requires continuous adaptation to internal and external contexts. Changes in decision-making are reliable transdiagnostic symptoms of neuropsychiatric disorders. We created a computational model demonstrating how the striosome compartment of the striatum constructs a mathematical space for decision-making computations depending on context, and how the matrix compartment defines action value depending on the space. The model explains multiple experimental results and unifies other theories like reward prediction error, roles of the direct versus indirect pathways, and roles of the striosome versus matrix, under one framework. We also found, through new analyses, that striosome and matrix neurons increase their synchrony during difficult tasks, caused by a necessary increase in dimensionality of the space. The model makes testable predictions about individual differences in disorder susceptibility, decision-making symptoms shared among neuropsychiatric disorders, and differences in neuropsychiatric disorder symptom presentation. The model reframes the role of the striosomal circuit in neuroeconomic and disorder-affected decision-making.

Highlights

Striosomes prioritize decision-related data used by matrix to set action values. Striosomes and matrix have different roles in the direct and indirect pathways. Abnormal information organization/valuation alters disorder presentation. Variance in data prioritization may explain individual differences in disorders.

eTOC

Beck et al. developed a computational model of how a striatal circuit functions during decision-making. The model unifies and extends theories about the direct versus indirect pathways. It further suggests how aberrant circuit function underlies decision-making phenomena observed in neuropsychiatric disorders.

Compulsive avoidance in youths and adults with OCD: an aversive pavlovian-to-instrumental transfer study

Compulsive behaviour may often be triggered by Pavlovian cues. Assessing how Pavlovian cues drive instrumental behaviour in obsessive-compulsive disorder (OCD) is therefore crucial to understand how compulsions develop and are maintained. An aversive Pavlovian-to-Instrumental transfer (PIT) paradigm, particularly one involving avoidance/cancellation of negative outcomes, can enable such investigation and has not previously been studied in clinical-OCD. Forty-one participants diagnosed with OCD (21 adults; 20 youths) and 44 controls (21 adults; 23 youths) completed an aversive PIT task. Participants had to prevent the delivery of unpleasant noises by moving a joystick in the correct direction. They could infer these correct responses by learning appropriate response-outcome (instrumental) and stimulus-outcome (Pavlovian) associations. We then assessed whether Pavlovian cues elicited specific instrumental avoidance responses (specific PIT) and induced general instrumental avoidance (general PIT). We investigated whether task learning and confidence indices influenced PIT strength differentially between groups. There was no overall group difference in PIT performance, although youths with OCD showed weaker specific PIT than youth controls. However, urge to avoid unpleasant noises and preference for safe over unsafe stimuli influenced specific and general PIT respectively in OCD, while PIT in controls was more influenced by confidence in instrumental and Pavlovian learning. Thus, in OCD, implicit motivational factors, but not learnt knowledge, may contribute to the successful integration of aversive Pavlovian and instrumental cues. This implies that compulsive avoidance may be driven by these automatic processes. Youths with OCD show deficits in specific PIT, suggesting cue integration impairments are only apparent in adolescence. These findings may be clinically relevant as they emphasise the importance of targeting such implicit motivational processes when treating OCD.

Investigating grey matter volumetric trajectories through the lifespan at the individual level

Adolescents exhibit remarkable heterogeneity in the structural architecture of brain development. However, due to limited large-scale longitudinal neuroimaging studies, existing research has largely focused on population averages, and the neurobiological basis underlying individual heterogeneity remains poorly understood. Here we identify, using the IMAGEN adolescent cohort followed up over 9 years (14-23 y), three groups of adolescents characterized by distinct developmental patterns of whole-brain gray matter volume (GMV). Group 1 show continuously decreasing GMV associated with higher neurocognitive performances than the other two groups during adolescence. Group 2 exhibit a slower rate of GMV decrease and lower neurocognitive performances compared with Group 1, which was associated with epigenetic differences and greater environmental burden. Group 3 show increasing GMV and lower baseline neurocognitive performances due to a genetic variation. Using the UK Biobank, we show these differences may be attenuated in mid-to-late adulthood. Our study reveals clusters of adolescent neurodevelopment based on GMV and the potential long-term impact.

Hippocampus and striatum show distinct contributions to longitudinal changes in value-based learning in middle childhood

The hippocampal-dependent memory system and striatal-dependent memory system modulate reinforcement learning depending on feedback timing in adults, but their contributions during development remain unclear. In a 2-year longitudinal study, 6-to-7-year-old children performed a reinforcement learning task in which they received feedback immediately or with a short delay following their response. Children's learning was found to be sensitive to feedback timing modulations in their reaction time and inverse temperature parameter, which quantifies value-guided decision-making. They showed longitudinal improvements towards more optimal value-based learning, and their hippocampal volume showed protracted maturation. Better delayed model-derived learning covaried with larger hippocampal volume longitudinally, in line with the adult literature. In contrast, a larger striatal volume in children was associated with both better immediate and delayed model-derived learning longitudinally. These findings show, for the first time, an early hippocampal contribution to the dynamic development of reinforcement learning in middle childhood, with neurally less differentiated and more cooperative memory systems than in adults.

Electrical brain activations in preadolescents during a probabilistic reward-learning task reflect cognitive processes and behavioral strategy

Both adults and children learn through feedback which environmental events and choices are associated with higher probability of reward, an ability thought to be supported by the development of fronto-striatal reward circuits. Recent developmental studies have applied computational models of reward learning to investigate such learning in children. However, tasks and measures effective for assaying the cascade of reward-learning neural processes in children have been limited. Using a child-version of a probabilistic reward-learning task while recording event-related-potential (ERP) measures of electrical brain activity, this study examined key processes of reward learning in preadolescents (8-12 years old; n=30), namely: (1) reward-feedback sensitivity, as measured by the early-latency, reward-related, frontal ERP positivity, (2) rapid attentional shifting of processing toward favored visual stimuli, as measured by the N2pc component, and (3) longer-latency attention-related responses to reward feedback as a function of behavioral strategies (i.e., Win-Stay-Lose-Shift), as measured by the central-parietal P300. Consistent with our prior work in adults, the behavioral findings indicate preadolescents can learn stimulus-reward outcome associations, but at varying levels of performance. Neurally, poor preadolescent learners (those with slower learning rates) showed greater reward-related positivity amplitudes relative to good learners, suggesting greater reward-feedback sensitivity. We also found attention shifting towards to-be-chosen stimuli, as evidenced by the N2pc, but not to more highly rewarded stimuli as we have observed in adults. Lastly, we found the behavioral learning strategy (i.e., Win-Stay-Lose-Shift) reflected by the feedback-elicited parietal P300. These findings provide novel insights into the key neural processes underlying reinforcement learning in preadolescents.

Internally Formed Preferences for Options only Influence Initial Decisions in Gambling Tasks, while the Gambling Outcomes do not Alter these Preferences

All humans must engage in decision-making. Decision-making processes can be broadly classified into internally guided decision-making (IDM), which is determined by individuals' internal value criteria, such as preference, or externally guided decision-making (EDM), which is determined by environmental external value criteria, such as monetary rewards. However, real-life decisions are never made simply using one kind of decision-making, and the relationship between IDM and EDM remains unclear. This study had individuals perform gambling tasks requiring the EDM using stimuli that formed preferences through the preference judgment task as the IDM. Computational model analysis revealed that strong preferences in the IDM affected initial choice behavior in the EDM. Moreover, through the analysis of the subjective preference evaluation after the gambling tasks, we found that even when stimuli that were preferred in the IDM were perceived as less valuable in the EDM, the preference for IDM was maintained after EDM. These results indicate that although internal criteria, such as preferences, influence EDM, the results show that internal and external criteria differ.

Artificial neural networks for model identification and parameter estimation in computational cognitive models

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

Artificial neural networks for model identification and parameter estimation in computational cognitive models

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

A cognitive-computational account of mood swings in adolescence

Teenagers have a reputation for being fickle, in both their choices and their moods. This variability may help adolescents as they begin to independently navigate novel environments. Recently, however, adolescent moodiness has also been linked to psychopathology. Here, we consider adolescents' mood swings from a novel computational perspective, grounded in reinforcement learning (RL). This model proposes that mood is determined by surprises about outcomes in the environment, and how much we learn from these surprises. It additionally suggests that mood biases learning and choice in a bidirectional manner. Integrating independent lines of research, we sketch a cognitive-computational account of how adolescents' mood, learning, and choice dynamics influence each other, with implications for normative and psychopathological development.

Examining the Association between Punishment and Reward Sensitivity and Response Inhibition to Previously-Incentivized Cues across Development

Processing and learning from affective cues to guide goal-directed behavior may be particularly important during adolescence; yet the factors that promote and/or disrupt the ability to integrate value in order to guide decision making across development remain unclear. The present study (N = 1046) assessed individual difference factors (self-reported punishment and reward sensitivity) related to whether previously-rewarded and previously-punished cues differentially impact goal-directed behavior (response inhibition) in a large developmental sample. Participants were between the ages of 8-21 years (Mage = 14.29, SD = 3.97, 50.38% female). Previously-rewarded cues improved response inhibition among participants age 14 and older. Further, punishment sensitivity predicted overall improved response inhibition among participants aged 10 to 18. The results highlight two main factors that are associated with improvements in the ability to integrate value to guide goal-directed behaviour - cues in the environment (e.g., reward-laden cues) and individual differences in punishment sensitivity. These findings have implications for both educational and social policies aimed at characterizing the ways in which youth integrate value to guide decision making.

Goal-directed learning in adolescence: neurocognitive development and contextual influences

Adolescence is a time during which we transition to independence, explore new activities and begin pursuit of major life goals. Goal-directed learning, in which we learn to perform actions that enable us to obtain desired outcomes, is central to many of these processes. Currently, our understanding of goal-directed learning in adolescence is itself in a state of transition, with the scientific community grappling with inconsistent results. When we examine metrics of goal-directed learning through the second decade of life, we find that many studies agree there are steady gains in performance in the teenage years, but others report that adolescent goal-directed learning is already adult-like, and some find adolescents can outperform adults. To explain the current variability in results, sophisticated experimental designs are being applied to test learning in different contexts. There is also increasing recognition that individuals of different ages and in different states will draw on different neurocognitive systems to support goal-directed learning. Through adoption of more nuanced approaches, we can be better prepared to recognize and harness adolescent strengths and to decipher the purpose (or goals) of adolescence itself.

Nucleus accumbens dopamine release reflects Bayesian inference during instrumental learning

Dopamine release in the nucleus accumbens has been hypothesized to signal reward prediction error, the difference between observed and predicted reward, suggesting a biological implementation for reinforcement learning. Rigorous tests of this hypothesis require assumptions about how the brain maps sensory signals to reward predictions, yet this mapping is still poorly understood. In particular, the mapping is non-trivial when sensory signals provide ambiguous information about the hidden state of the environment. Previous work using classical conditioning tasks has suggested that reward predictions are generated conditional on probabilistic beliefs about the hidden state, such that dopamine implicitly reflects these beliefs. Here we test this hypothesis in the context of an instrumental task (a two-armed bandit), where the hidden state switches repeatedly. We measured choice behavior and recorded dLight signals reflecting dopamine release in the nucleus accumbens core. Model comparison based on the behavioral data favored models that used Bayesian updating of probabilistic beliefs. These same models also quantitatively matched the dopamine measurements better than non-Bayesian alternatives. We conclude that probabilistic belief computation plays a fundamental role in instrumental performance and associated mesolimbic dopamine signaling.

Measuring flexibility in autistic adults: Exploring the factor structure of the flexibility scale self report

Cognitive flexibility differences are common for autistic individuals and have an impact on a range of clinical outcomes. However, there is currently a lack of well validated measurement tools to assess flexibility in adulthood. The Flexibility Scale was originally designed as a parent-report measure of real-world flexibility challenges in youth. The original Flexibility Scale provides a total score and five subscales: Routines and Rituals, Transitions and Change, Special Interests, Social Flexibility, and Generativity. In this study, we evaluate the factorial validity of the Flexibility Scale as a self-report (Flexibility Scale Self Report) measure of cognitive flexibility, adapted from the original Flexibility Scale, for use by autistic adults. This study includes both a primary sample (n = 813; mean age = 40.3; 59% female) and an independently recruited replication sample (n = 120; mean age = 32.8; 74% female) of individuals who completed the Flexibility Scale Self Report. The analysis consisted of an initial confirmatory factor analysis (CFA) of the original Flexibility Scale structure, followed by exploratory factor analysis (EFA) and factor optimization within a structural equation modeling framework to identify the optimal structure for the questionnaire in adults. The identified structure was then replicated through CFA in the replication sample. Our results indicate an alternative optimal scale structure from the original Flexibility Scale, which includes fewer items, and only three (Routines/Rituals, Transitions and Change, Special Interests) of the five subscales contributing to the flexibility total score. Comparisons revealed no structural differences within the scale based on sex assigned at birth. Here the Generativity and Social Flexibility scales are treated as independent but related scales. The implications for measurement of cognitive flexibility in clinical and research settings, as well as theoretical underpinnings are discussed.

Structural neurodevelopment at the individual level - a life-course investigation using ABCD, IMAGEN and UK Biobank data

Adolescents exhibit remarkable heterogeneity in the structural architecture of brain development. However, due to the lack of large-scale longitudinal neuroimaging studies, existing research has largely focused on population averages and the neurobiological basis underlying individual heterogeneity remains poorly understood. Using structural magnetic resonance imaging from the IMAGEN cohort (n=1,543), we show that adolescents can be clustered into three groups defined by distinct developmental patterns of whole-brain gray matter volume (GMV). Genetic and epigenetic determinants of group clustering and long-term impacts of neurodevelopment in mid-to-late adulthood were investigated using data from the ABCD, IMAGEN and UK Biobank cohorts. Group 1, characterized by continuously decreasing GMV, showed generally the best neurocognitive performances during adolescence. Compared to Group 1, Group 2 exhibited a slower rate of GMV decrease and worsened neurocognitive development, which was associated with epigenetic changes and greater environmental burden. Further, Group 3 showed increasing GMV and delayed neurocognitive development during adolescence due to a genetic variation, while these disadvantages were attenuated in mid-to-late adulthood. In summary, our study revealed novel clusters of adolescent structural neurodevelopment and suggested that genetically-predicted delayed neurodevelopment has limited long-term effects on mental well-being and socio-economic outcomes later in life. Our results could inform future research on policy interventions aimed at reducing the financial and emotional burden of mental illness.

Action initiation and punishment learning differ from childhood to adolescence while reward learning remains stable

Theoretical and empirical accounts suggest that adolescence is associated with heightened reward learning and impulsivity. Experimental tasks and computational models that can dissociate reward learning from the tendency to initiate actions impulsively (action initiation bias) are thus critical to characterise the mechanisms that drive developmental differences. However, existing work has rarely quantified both learning ability and action initiation, or it has relied on small samples. Here, using computational modelling of a learning task collected from a large sample (N = 742, 9-18 years, 11 countries), we test differences in reward and punishment learning and action initiation from childhood to adolescence. Computational modelling reveals that whilst punishment learning rates increase with age, reward learning remains stable. In parallel, action initiation biases decrease with age. Results are similar when considering pubertal stage instead of chronological age. We conclude that heightened reward responsivity in adolescence can reflect differences in action initiation rather than enhanced reward learning.

Prediction errors arising from switches between major and minor modes in music: An fMRI study

The major and minor modes in Western music have positive and negative connotations, respectively. The present fMRI study examined listeners' neural responses to switches between major and minor modes. We manipulated the final chords of J. S. Bach's keyboard pieces so that each major-mode passage ended with either the major (Major-Major) or minor (Major-Minor) tonic chord, and each minor-mode passage ended with either the minor (Minor-Minor) or major (Minor-Major) tonic chord. If the final major and minor chords have positive and negative reward values respectively, the Major-Minor and Minor-Major stimuli would cause negative and positive reward prediction errors (RPEs) respectively in a listener's brain. We found that activity in a frontoparietal network was significantly higher for Major-Minor than for Major-Major. Based on previous research, these results support the idea that a major-to-minor switch causes negative RPE. The contrast of Minor-Major minus Minor-Minor yielded activation in the ventral insula and visual cortex, speaking against the idea that a minor-to-major switch causes positive RPE. We discuss our results in relation to executive functions and the emotional connotations of major versus minor modes.

Expecting the unexpected: a review of learning under uncertainty across development

Many of our decisions take place under uncertainty. To successfully navigate the environment, individuals need to estimate the degree of uncertainty and adapt their behaviors accordingly by learning from experiences. However, uncertainty is a broad construct and distinct types of uncertainty may differentially influence our learning. We provide a semi-systematic review to illustrate cognitive and neurobiological processes involved in learning under two types of uncertainty: learning in environments with stochastic outcomes, and with volatile outcomes. We specifically reviewed studies (N = 26 studies) that included an adolescent population, because adolescence is a period in life characterized by heightened exploration and learning, as well as heightened uncertainty due to experiencing many new, often social, environments. Until now, reviews have not comprehensively compared learning under distinct types of uncertainties in this age range. Our main findings show that although the overall developmental patterns were mixed, most studies indicate that learning from stochastic outcomes, as indicated by increased accuracy in performance, improved with age. We also found that adolescents tended to have an advantage compared with adults and children when learning from volatile outcomes. We discuss potential mechanisms explaining these age-related differences and conclude by outlining future research directions.

Confirmatory reinforcement learning changes with age during adolescence

Understanding how learning changes during human development has been one of the long-standing objectives of developmental science. Recently, advances in computational biology have demonstrated that humans display a bias when learning to navigate novel environments through rewards and punishments: they learn more from outcomes that confirm their expectations than from outcomes that disconfirm them. Here, we ask whether confirmatory learning is stable across development, or whether it might be attenuated in developmental stages in which exploration is beneficial, such as in adolescence. In a reinforcement learning (RL) task, 77 participants aged 11-32 years (four men, mean age = 16.26) attempted to maximize monetary rewards by repeatedly sampling different pairs of novel options, which varied in their reward/punishment probabilities. Mixed-effect models showed an age-related increase in accuracy as long as learning contingencies remained stable across trials, but less so when they reversed halfway through the trials. Age was also associated with a greater tendency to stay with an option that had just delivered a reward, more than to switch away from an option that had just delivered a punishment. At the computational level, a confirmation model provided increasingly better fit with age. This model showed that age differences are captured by decreases in noise or exploration, rather than in the magnitude of the confirmation bias. These findings provide new insights into how learning changes during development and could help better tailor learning environments to people of different ages. RESEARCH HIGHLIGHTS: Reinforcement learning shows age-related improvement during adolescence, but more in stable learning environments compared with volatile learning environments. People tend to stay with an option after a win more than they shift from an option after a loss, and this asymmetry increases with age during adolescence. Computationally, these changes are captured by a developing confirmatory learning style, in which people learn more from outcomes that confirm rather than disconfirm their choices. Age-related differences in confirmatory learning are explained by decreases in stochasticity, rather than changes in the magnitude of the confirmation bias.

Diminished prospective mental representations of reward mediate reward learning strategies among youth with internalizing symptoms

BACKGROUND

Adolescent internalizing symptoms and trauma exposure have been linked with altered reward learning processes and decreased ventral striatal responses to rewarding cues. Recent computational work on decision-making highlights an important role for prospective representations of the imagined outcomes of different choices. This study tested whether internalizing symptoms and trauma exposure among youth impact the generation of prospective reward representations during decision-making and potentially mediate altered behavioral strategies during reward learning.

METHODS

Sixty-one adolescent females with varying exposure to interpersonal violence exposure (n = 31 with histories of physical or sexual assault) and severity of internalizing symptoms completed a social reward learning task during fMRI. Multivariate pattern analyses (MVPA) were used to decode neural reward representations at the time of choice.

RESULTS

MVPA demonstrated that rewarding outcomes could accurately be decoded within several large-scale distributed networks (e.g. frontoparietal and striatum networks), that these reward representations were reactivated prospectively at the time of choice in proportion to the expected probability of receiving reward, and that youth with behavioral strategies that favored exploiting high reward options demonstrated greater prospective generation of reward representations. Youth internalizing symptoms, but not trauma exposure characteristics, were negatively associated with both the behavioral strategy of exploiting high reward options as well as the prospective generation of reward representations in the striatum.

CONCLUSIONS

These data suggest diminished prospective mental simulation of reward as a mechanism of altered reward learning strategies among youth with internalizing symptoms.

Diminished reinforcement sensitivity in adolescence is associated with enhanced response switching and reduced coding of choice probability in the medial frontal pole

Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.

Local and global reward learning in the lateral frontal cortex show differential development during human adolescence

Reward-guided choice is fundamental for adaptive behaviour and depends on several component processes supported by prefrontal cortex. Here, across three studies, we show that two such component processes, linking reward to specific choices and estimating the global reward state, develop during human adolescence and are linked to the lateral portions of the prefrontal cortex. These processes reflect the assignment of rewards contingently to local choices, or noncontingently, to choices that make up the global reward history. Using matched experimental tasks and analysis platforms, we show the influence of both mechanisms increase during adolescence (study 1) and that lesions to lateral frontal cortex (that included and/or disconnected both orbitofrontal and insula cortex) in human adult patients (study 2) and macaque monkeys (study 3) impair both local and global reward learning. Developmental effects were distinguishable from the influence of a decision bias on choice behaviour, known to depend on medial prefrontal cortex. Differences in local and global assignments of reward to choices across adolescence, in the context of delayed grey matter maturation of the lateral orbitofrontal and anterior insula cortex, may underlie changes in adaptive behaviour.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

Flexibility in valenced reinforcement learning computations across development

Optimal integration of positive and negative outcomes during learning varies depending on an environment's reward statistics. The present study investigated the extent to which children, adolescents, and adults (N = 142 8-25 year-olds, 55% female, 42% White, 31% Asian, 17% mixed race, and 8% Black; data collected in 2021) adapt their weighting of better-than-expected and worse-than-expected outcomes when learning from reinforcement. Participants made choices across two contexts: one in which weighting positive outcomes more heavily than negative outcomes led to better performance, and one in which the reverse was true. Reinforcement learning modeling revealed that across age, participants shifted their valence biases in accordance with environmental structure. Exploratory analyses revealed strengthening of context-dependent flexibility with increasing age.

Cortical functionality during reversal learning on a decision-making task in young men

The evaluation of external and internal stimuli permits the ongoing actualization of choice-related information and, thus, the association between stimuli and outcomes. This process is essential to decision-making as it allows constant adaptation to environmental changes in order to maximize gains and minimize losses. Reversal learning paradigms are used to study this process, which has been associated with prefrontal cortex activity (frontopolar, dorsolateral) in conjunction with posterior areas (parietal, temporal), due to their participation in integrating and processing the stimuli-reward relation. The aim of this study was to assess the cortical functionality associated with reversal learning during the decision-making process. The EEG activity of 22 young men was recorded while performing a decision-making task in a reversal learning condition compared to an initial learning condition. EEG data were analyzed during evaluation of the stimuli, before motor execution (formation of preferences), and during task feedback (outcome evaluation). The formation of preferences stage was characterized by a higher correlation of the alpha2 band between the parietal cortices. In the feedback stage of the reversal learning condition, a higher absolute power of the theta band in the left dorsolateral (F3), and a lower correlation of the alpha1 band between the right frontopolar and dorsolateral (Fp2-F4), as well as between the right frontopolar and temporal (Fp2-T4), were observed. The data obtained show that the EEG activity of the areas recorded changed in the evaluation of the stimuli information in the reversal learning condition during a decision-making task.

Different brain systems support learning from received and avoided pain during human pain-avoidance learning

Both unexpected pain and unexpected pain absence can drive avoidance learning, but whether they do so via shared or separate neural and neurochemical systems is largely unknown. To address this issue, we combined an instrumental pain-avoidance learning task with computational modeling, functional magnetic resonance imaging (fMRI), and pharmacological manipulations of the dopaminergic (100 mg levodopa) and opioidergic (50 mg naltrexone) systems (N = 83). Computational modeling provided evidence that untreated participants learned more from received than avoided pain. Our dopamine and opioid manipulations negated this learning asymmetry by selectively increasing learning rates for avoided pain. Furthermore, our fMRI analyses revealed that pain prediction errors were encoded in subcortical and limbic brain regions, whereas no-pain prediction errors were encoded in frontal and parietal cortical regions. However, we found no effects of our pharmacological manipulations on the neural encoding of prediction errors. Together, our results suggest that human pain-avoidance learning is supported by separate threat- and safety-learning systems, and that dopamine and endogenous opioids specifically regulate learning from successfully avoided pain.

Prevalência e Caracterização do Consumo de Tabaco entre Adolescentes de Montes Claros, Minas Gerais, 2019-2020

Introdução: As doenças tabaco-relacionadas são responsáveis por cinco milhões de óbitos a cada ano. Em 2020, estimou-se que o número de óbitos anuais aumentaria para dez milhões, dos quais 70% ocorreriam nos países em desenvolvimento. Objetivo: Estimar a prevalência do tabagismo e caracterizar o consumo de tabaco entre adolescentes de 12 e de 15 anos na cidade de Montes Claros, Minas Gerais, em 2019-2020. Método: Estudo epidemiológico quantitativo transversal, parte do projeto “Levantamento epidemiológico sobre condições de saúde bucal entre escolares de Montes Claros, Minas Gerais, Brasil”, realizado em 2019-2020. A amostra estimada foi de 354 escolares de 12 anos e 355 de 15 anos, calculada entre 4.458 escolares de 12 anos e 4.524 de 15 anos, respectivamente. Os dados foram analisados descritivamente por meio de medidas de tendência central, variância e frequência. Resultados: Houve prevalência do uso de tabaco entre adolescentes alguma vez na vida de 8,6 %, sendo 4,6% relativos ao consumo do cigarro Paiol. Conclusão: A prevalência do consumo encontrada entre os adolescentes e preocupante por causa dos efeitos deletérios a saúde. Conhecer o perfil dos desses usuários pode contribuir para as políticas de saúde coletiva com o objetivo de reduzir o seu uso.

Cognitive Neural Mechanism of Backward Inhibition and Deinhibition: A Review

Task switching is one of the typical paradigms to study cognitive control. When switching back to a recently inhibited task (e.g., “A” in an ABA sequence), the performance is often worse compared to a task without N-2 task repetitions (e.g., CBA). This difference is called the backward inhibitory effect (BI effect), which reflects the process of overcoming residual inhibition from a recently performed task (i.e., deinhibition). The neural mechanism of backward inhibition and deinhibition has received a lot of attention in the past decade. Multiple brain regions, including the frontal lobe, parietal, basal ganglia, and cerebellum, are activated during deinhibition. The event-related potentials (ERP) studies have shown that deinhibition process is reflected in the P1/N1 and P3 components, which might be related to early attention control, context updating, and response selection, respectively. Future research can use a variety of new paradigms to separate the neural mechanisms of BI and deinhibition.

A methodological perspective on learning in the developing brain

The brain undergoes profound development across childhood and adolescence, including continuous changes in brain morphology, connectivity, and functioning that are, in part, dependent on one's experiences. These neurobiological changes are accompanied by significant changes in children's and adolescents' cognitive learning. By drawing from studies in the domains of reading, reinforcement learning, and learning difficulties, we present a brief overview of methodological approaches and research designs that bridge brain- and behavioral research on learning. We argue that ultimately these methods and designs may help to unravel questions such as why learning interventions work, what learning computations change across development, and how learning difficulties are distinct between individuals.

Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal

During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated how performance changes across adolescent development in a stochastic, volatile reversal-learning task that uniquely taxes the balance of persistence and flexibility. In a sample of 291 participants aged 8-30, we found that in the mid-teen years, adolescents outperformed both younger and older participants. We developed two independent cognitive models, based on Reinforcement learning (RL) and Bayesian inference (BI). The RL parameter for learning from negative outcomes and the BI parameters specifying participants' mental models were closest to optimal in mid-teen adolescents, suggesting a central role in adolescent cognitive processing. By contrast, persistence and noise parameters improved monotonically with age. We distilled the insights of RL and BI using principal component analysis and found that three shared components interacted to form the adolescent performance peak: adult-like behavioral quality, child-like time scales, and developmentally-unique processing of positive feedback. This research highlights adolescence as a neurodevelopmental window that can create performance advantages in volatile and uncertain environments. It also shows how detailed insights can be gleaned by using cognitive models in new ways.

Hippocampal Contribution to Probabilistic Feedback Learning: Modeling Observation- and Reinforcement-based Processes

Simple probabilistic reinforcement learning is recognized as a striatum-based learning system, but in recent years, has also been associated with hippocampal involvement. This study examined whether such involvement may be attributed to observation-based learning (OL) processes, running in parallel to striatum-based reinforcement learning. A computational model of OL, mirroring classic models of reinforcement-based learning (RL), was constructed and applied to the neuroimaging data set of Palombo, Hayes, Reid, and Verfaellie (2019). Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia. Cognitive, Affective & Behavioral Neuroscience, 19(3), 523-536. Results suggested that OL processes may indeed take place concomitantly to reinforcement learning and involve activation of the hippocampus and central orbitofrontal cortex. However, rather than independent mechanisms running in parallel, the brain correlates of the OL and RL prediction errors indicated collaboration between systems, with direct implication of the hippocampus in computations of the discrepancy between the expected and actual reinforcing values of actions. These findings are consistent with previous accounts of a role for the hippocampus in encoding the strength of observed stimulus-outcome associations, with updating of such associations through striatal reinforcement-based computations. In addition, enhanced negative RL prediction error signaling was found in the anterior insula with greater use of OL over RL processes. This result may suggest an additional mode of collaboration between the OL and RL systems, implicating the error monitoring network.

Impaired learning to dissociate advantageous and disadvantageous risky choices in adolescents

Adolescence is characterized by a surge in maladaptive risk-taking behaviors, but whether and how this relates to developmental changes in experience-based learning is largely unknown. In this preregistered study, we addressed this issue using a novel task that allowed us to separate the learning-driven optimization of risky choice behavior over time from overall risk-taking tendencies. Adolescents (12-17 years old) learned to dissociate advantageous from disadvantageous risky choices less well than adults (20-35 years old), and this impairment was stronger in early than mid-late adolescents. Computational modeling revealed that adolescents' suboptimal performance was largely due to an inefficiency in core learning and choice processes. Specifically, adolescents used a simpler, suboptimal, expectation-updating process and a more stochastic choice policy. In addition, the modeling results suggested that adolescents, but not adults, overvalued the highest rewards. Finally, an exploratory latent-mixture model analysis indicated that a substantial proportion of the participants in each age group did not engage in experience-based learning but used a gambler's fallacy strategy, stressing the importance of analyzing individual differences. Our results help understand why adolescents tend to make more, and more persistent, maladaptive risky decisions than adults when the values of these decisions have to be learned from experience.

Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory

As individuals learn through trial and error, some are more influenced by good outcomes, while others weight bad outcomes more heavily. Such valence biases may also influence memory for past experiences. Here, we examined whether valence asymmetries in reinforcement learning change across adolescence, and whether individual learning asymmetries bias the content of subsequent memory. Participants ages 8-27 learned the values of 'point machines,' after which their memory for trial-unique images presented with choice outcomes was assessed. Relative to children and adults, adolescents overweighted worse-than-expected outcomes during learning. Individuals' valence biases modulated incidental memory, such that those who prioritized worse- (or better-) than-expected outcomes during learning were also more likely to remember images paired with these outcomes, an effect reproduced in an independent dataset. Collectively, these results highlight age-related changes in the computation of subjective value and demonstrate that a valence-asymmetric valuation process influences how information is prioritized in episodic memory.

Frontoparietal network activity during model-based reinforcement learning updates is reduced among adolescents with severe sexual abuse

Trauma and trauma-related disorders are characterized by impaired learning processes, including reinforcement learning (RL). Identifying which aspects of learning are altered by trauma is critical endeavor, as this may reveal key mechanisms of impairment and potential intervention targets. There are at least two types of RL that have been delineated using computational modeling: model-free and model-based RL. Although these RL processes differentially predict decision-making behavior, most research has examined the impact of trauma on model-free RL. Currently unclear whether model-based RL, which involves building abstract and nuanced representations of stimulus-outcome relationships, is impaired among individuals with a history of trauma. The present study sought to test the hypothesis of impaired model-based RL among adolescent females exposed to assaultive trauma. Participants (n = 60; 29 without a history of assault and 31 with a history of assault with and without PTSD) completed a three-arm bandit task during fMRI acquisition. Two computational models compared the degree to which participants' task behavior fit the use of a model-free versus model-based RL strategy. Although a history of assaultive trauma did not predict poorer model-based RL, greater sexual abuse severity predicted less use of model-based compared to model-free RL. Additionally, severe sexual abuse predicted less left frontoparietal network encoding of model-based RL updates. Altered model-based RL, which supports goal-directed behavior, may be an important route through which clinical impairment emerges among individuals with a history of severe sexual abuse and should be examined further in future studies.

Association of Environmental Uncertainty With Altered Decision-making and Learning Mechanisms in Youths With Obsessive-Compulsive Disorder

Importance

Adults with obsessive-compulsive disorder (OCD) display perseverative behavior in stable environments but exhibit vacillating choice when payoffs are uncertain. These findings may be associated with intolerance of uncertainty and compulsive behaviors; however, little is known about the mechanisms underlying learning and decision-making in youths with OCD because research into this population has been limited.

Objective

To investigate cognitive mechanisms associated with decision-making in youths with OCD by using executive functioning tasks and computational modeling.

Design, Setting, and Participants

In this cross-sectional study, 50 youths with OCD (patients) and 53 healthy participants (controls) completed a probabilistic reversal learning (PRL) task between January 2014 and March 2020. A separate sample of 27 patients and 46 controls completed the Wisconsin Card Sorting Task (WCST) between January 2018 and November 2020. The study took place at the University of Cambridge in the UK.

Main Outcomes and Measures

Decision-making mechanisms were studied by fitting hierarchical bayesian reinforcement learning models to the 2 data sets and comparing model parameters between participant groups. Model parameters included reward and punishment learning rates (feedback sensitivity), reinforcement sensitivity and decision consistency (exploitation), and stickiness (perseveration). Associations of receipt of serotonergic medication with performance were assessed.

Results

In total, 50 patients (29 female patients [58%]; median age, 16.6 years [IQR, 15.3-18.0 years]) and 53 controls (30 female participants [57%]; median age, 16.4 years [IQR, 14.8-18.0 years]) completed the PRL task. A total of 27 patients (18 female patients [67%]; median age, 16.1 years [IQR, 15.2-17.2 years]) and 46 controls (28 female participants [61%]; median age, 17.2 [IQR, 16.3-17.6 years]) completed the WCST. During the reversal phase of the PRL task, patients made fewer correct responses (mean [SD] proportion: 0.83 [0.16] for controls and 0.61 [0.31] for patients; 95% CI, -1.31 to -0.64) and switched choices more often following false-negative feedback (mean [SD] proportion: 0.09 [0.16] for controls vs 0.27 [0.34] for patients; 95% CI, 0.60-1.26) and true-positive feedback (mean [SD] proportion: 0.93 [0.17] for controls vs 0.73 [0.34] for patients; 95% CI, -2.17 to -1.31). Computational modeling revealed that patients displayed enhanced reward learning rates (mean difference [MD], 0.21; 95% highest density interval [HDI], 0.04-0.38) but decreased punishment learning rates (MD, -0.29; 95% HDI, -0.39 to -0.18), reinforcement sensitivity (MD, -4.91; 95% HDI, -9.38 to -1.12), and stickiness (MD, -0.35; 95% HDI, -0.57 to -0.11) compared with controls. There were no group differences on standard WCST measures and computational model parameters. However, patients who received serotonergic medication showed slower response times (mean [SD], 1420.49 [279.71] milliseconds for controls, 1471.42 [212.81] milliseconds for patients who were unmedicated, and 1738.25 [349.23] milliseconds for patients who were medicated) (control vs medicated MD, -320.26 [95% CI, -547.00 to -88.68]) and increased unique errors (mean [SD] proportion: 0.001 [0.004] for controls, 0.002 [0.004] for patients who were unmedicated, and 0.008 [0.01] for patients who were medicated) (control vs medicated MD, -0.007 [95% CI, -3.14 to -0.36]) on the WCST.

Conclusions and Relevance

The results of this cross-sectional study indicated that youths with OCD showed atypical probabilistic reversal learning but were generally unimpaired on the deterministic WCST, although unexpected results were observed for patients receiving serotonergic medication. These findings have implications for reframing the understanding of early-onset OCD as a disorder in which decision-making is associated with uncertainty in the environment, a potential target for therapeutic treatment. These results provide continuity with findings in adults with OCD.

Increased Ventromedial Prefrontal Cortex Activity in Adolescence Benefits Prosocial Reinforcement Learning

Learning which of our behaviors benefit others contributes to forming social relationships. An important period for the development of (pro)social behavior is adolescence, which is characterized by transitions in social connections. It is, however, unknown how learning to benefit others develops across adolescence and what the underlying cognitive and neural mechanisms are. In this functional neuroimaging study, we assessed learning for self and others (i.e., prosocial learning) and the concurring neural tracking of prediction errors across adolescence (ages 9-21, N = 74). Participants performed a two-choice probabilistic reinforcement learning task in which outcomes resulted in monetary consequences for themselves, an unknown other, or no one. Participants from all ages were able to learn for themselves and others, but learning for others showed a more protracted developmental trajectory. Prediction errors for self were observed in the ventral striatum and showed no age-related differences. However, prediction error coding for others showed an age-related increase in the ventromedial prefrontal cortex. These results reveal insights into the computational mechanisms of learning for others across adolescence, and highlight that learning for self and others show different age-related patterns.

A Competition of Critics in Human Decision-Making

Recent experiments and theories of human decision-making suggest positive and negative errors are processed and encoded differently by serotonin and dopamine, with serotonin possibly serving to oppose dopamine and protect against risky decisions. We introduce a temporal difference (TD) model of human decision-making to account for these features. Our model involves two critics, an optimistic learning system and a pessimistic learning system, whose predictions are integrated in time to control how potential decisions compete to be selected. Our model predicts that human decision-making can be decomposed along two dimensions: the degree to which the individual is sensitive to (1) risk and (2) uncertainty. In addition, we demonstrate that the model can learn about the mean and standard deviation of rewards, and provide information about reaction time despite not modeling these variables directly. Lastly, we simulate a recent experiment to show how updates of the two learning systems could relate to dopamine and serotonin transients, thereby providing a mathematical formalism to serotonin's hypothesized role as an opponent to dopamine. This new model should be useful for future experiments on human decision-making.

Chronic alcohol exposure during critical developmental periods differentially impacts persistence of deficits in cognitive flexibility and related circuitry

Cognitive flexibility in decision making depends on prefrontal cortical function and is used by individuals to adapt to environmental changes in circumstances. Cognitive flexibility can be measured in the laboratory using a variety of discrete, translational tasks, including those that involve reversal learning and/or set-shifting ability. Distinct components of flexible behavior rely upon overlapping brain circuits, including different prefrontal substructures that have separable impacts on decision making. Cognitive flexibility is impaired after chronic alcohol exposure, particularly during development when the brain undergoes rapid maturation. This review examines how cognitive flexibility, as indexed by reversal and set-shifting tasks, is impacted by chronic alcohol exposure in adulthood, adolescent, and prenatal periods in humans and animal models. We also discuss areas for future study, including mechanisms that may contribute to the persistence of cognitive deficits after developmental alcohol exposure and the compacting consequences from exposure across multiple critical periods.

Effects of advice on experienced-based learning in adolescents and adults

Recent studies that compared effects of pre-learning advice on experience-based learning in adolescents and adults have yielded mixed results. Previous studies on this topic used choice tasks in which age-related differences in advice-related learning bias and exploratory choice behavior are difficult to dissociate. Moreover, these studies did not examine whether effects of advice depend on working memory load. In this preregistered study (in adolescents [13-15 years old] and adults [18-31 years old]), we addressed these issues by factorially combining advice and working memory load manipulations in an estimation task that does not require choices and hence eliminates the influence of known age-related differences in exploration. We found that advice guided participants' initial estimates in both age groups. When advice was correct, this improved estimation performance, especially in adolescents when working memory load was high. When advice was incorrect, it had a longer-lasting effect on adolescents' performance than on adults' performance. In contrast to previous findings in choice tasks, we found no evidence that advice biased learning in either age group. Taken together, our results suggest that learning in an estimation task improves between adolescence and adulthood but that the effects of advice on learning do not differ substantially between adolescents and adults.

Impaired Set-Shifting from Dorsal Stream Disconnection: Insights from a European Series of Right Parietal Lower-Grade Glioma Resection

Simple Summary

Awake surgery with cognitive monitoring has increasingly been implemented to preserve brain networks and functionality. More recently, not only surgery in the left but also in the right hemisphere, i.c., the parietal lobe, was associated with potential risk for deficits in cognitive functions, such as cognitive flexibility. We describe an explorative pilot study in an international consortium within clinical care as usual. Careful interpretation of our findings indicates that disconnection of the lateral part of the dorsal stream correlated with impaired set-shifting. More importantly, it illustrates the need for international collaboration on neuropsychological tests and methodologies to improve our understanding of white matter networks at risk during awake surgery.

Abstract

Awake surgery with cognitive monitoring has increasingly been implemented to preserve brain networks and functionality. More recently, not only surgery in the left but also in the right hemisphere, i.c., the parietal lobe, was associated with potential risk for deficits in cognitive functions, such as cognitive flexibility. In this explorative pilot study, we compare cognitive performance more than three months after surgery with baseline measurements and explore the association between cognitive decline and subcortical tracts that may have been severed during surgery in the right hemisphere. Twenty-two patients who underwent surgery for a right parietal low-grade glioma were assessed pre- and postoperatively using the Trail Making Test and the Stroop task to administer set-shifting abilities and inhibition. Volume measurements and lesion–symptom mapping analyses were performed on postoperative MRI scans. Careful interpretation of the results shows a change in TMT performance and not on the Stroop Task when the lateral part of the arcuate fasciculus is damaged, indicating that disconnection of the lateral part of the dorsal stream might be correlated specifically with impaired set-shifting and not with inhibition. More importantly, this study underlines the need for international concertation to allow larger studies to increase power and perform more detailed analyses.

Modeling changes in probabilistic reinforcement learning during adolescence

In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggest probabilistic learning may be inefficient in youths compared to adults, while others suggest it may be more efficient in youths in mid adolescence. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants' performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time scale); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development.

Maladaptive Avoidance Learning in the Orbitofrontal Cortex in Adolescents With Major Depression

BACKGROUND

Understanding the mechanisms in the brain's incentive network that give rise to symptoms of major depressive disorder (MDD) during adolescence provides new perspectives to address MDD in early stages of development. This functional magnetic resonance imaging study determines whether instrumental vigor and brain responses to appetitive and aversive monetary incentives are altered in adolescent MDD and associated with symptom severity.

METHODS

Adolescents with moderate to severe MDD (n = 30, mean age [SD] = 16.1 [1.4] years) and healthy control subjects (n = 33, mean age = 16.2 [1.9] years) matched for age, sex, and IQ performed a monetary incentive delay task. During outcome presentation, prediction error signals were used to study the response and coupling of the incentive network during learning of cue-outcome associations. A computational reinforcement model was used to assess adaptation of response vigor. Brain responses and effective connectivity to model-derived prediction errors were assessed and related to depression severity and anhedonia levels.

RESULTS

Participants with MDD behaved according to a more simplistic learning model and exhibited slower learning. Effective connectivity analysis of functional magnetic resonance imaging data revealed that impaired loss error processing in the orbitofrontal cortex was associated with aberrant gain control. Anhedonia scores correlated with loss-related error signals in the posterior insula and habenula.

CONCLUSIONS

Adolescent MDD is selectively related to impaired processing of error signals during loss, but not reward, in the orbitofrontal cortex. Aberrant evaluation of loss outcomes might reflect an early mechanism of how negative bias and helplessness manifest in the brain. This approach sheds light on pathomechanisms in MDD and may improve early diagnosis and treatment selection.

Neurodevelopment of the incentive network facilitates motivated behaviour from adolescence to adulthood

The ability to enhance motivated performance through incentives is crucial to guide and ultimately optimise the outcome of goal-directed behaviour. It remains largely unclear how motivated behaviour and performance develops particularly across adolescence. Here, we used computational fMRI to assess how response speed and its underlying neural circuitry are modulated by reward and loss in a monetary incentive delay paradigm. We demonstrate that maturational fine-tuning of functional coupling within the cortico-striatal incentive circuitry from adolescence to adulthood facilitates the ability to enhance performance selectively for higher subjective values. Additionally, during feedback, we found developmental sex differences of striatal representations of reward prediction errors in an exploratory analysis. Our findings suggest that a reduced capacity to utilise subjective value for motivated behaviour in adolescence is rooted in immature information processing in the incentive system. This indicates that the neurocircuitry for coordination of incentivised, motivated cognitive control acts as a bottleneck for behavioural adjustments in adolescence.

Cognitive and behavioural flexibility: neural mechanisms and clinical considerations

Cognitive and behavioural flexibility permit the appropriate adjustment of thoughts and behaviours in response to changing environmental demands. Brain mechanisms enabling flexibility have been examined using non-invasive neuroimaging and behavioural approaches in humans alongside pharmacological and lesion studies in animals. This work has identified large-scale functional brain networks encompassing lateral and orbital frontoparietal, midcingulo-insular and frontostriatal regions that support flexibility across the lifespan. Flexibility can be compromised in early-life neurodevelopmental disorders, clinical conditions that emerge during adolescence and late-life dementias. We critically evaluate evidence for the enhancement of flexibility through cognitive training, physical activity and bilingual experience.

Children perform extensive information gathering when it is not costly

Humans often face decisions where little is known about the choice options. Gathering information prior to making a choice is an important strategy to improve decision making under uncertainty. This is of particular importance during childhood and adolescence, when knowledge about the world is still limited. To examine how much information youths gather, we asked 107 children (8-9 years, N = 30), early (12-13 years, N = 41) and late adolescents (16-17 years, N = 36) to perform an information sampling task. We find that children gather significantly more information before making a decision compared to adolescents, but only if it does not come with explicit costs. Using computational modelling, we find that this is because children have reduced subjective costs for gathering information. Our findings thus demonstrate how children overcome their limited knowledge and neurocognitive constraints by deploying excessive information gathering, a developmental feature that could inform aberrant information gathering in psychiatric disorders.

Development of the brain functional connectome follows puberty-dependent nonlinear trajectories

Adolescence is a developmental period that dramatically impacts body and behavior, with pubertal hormones playing an important role not only in the morphological changes in the body but also in brain structure and function. Understanding brain development during adolescence has become a priority in neuroscience because it coincides with the onset of many psychiatric and behavioral disorders. However, little is known about how puberty influences the brain functional connectome. In this study, taking a longitudinal human sample of typically developing children and adolescents (of both sexes), we demonstrate that the development of the brain functional connectome better fits pubertal status than chronological age. In particular, centrality, segregation, efficiency, and integration of the brain functional connectome increase after the onset of the pubertal markers. We found that these effects are stronger in attention and task control networks. Lastly, after controlling for this effect, we showed that functional connectivity between these networks is related to better performance in cognitive flexibility. This study points out the importance of considering longitudinal nonlinear trends when exploring developmental trajectories, and emphasizes the impact of puberty on the functional organization of the brain in adolescence.