The computational basis of following advice in adolescents

Wanting to matter and learning to care: A neurodevelopmental window of opportunity for (Pro) social learning?

Wanting to matter-to feel socially recognized, appreciated, and capable of actions that benefit others-represents a fundamental motivation in human development. The motivational salience of mattering appears to increase in adolescence. Evidence suggests this is related to pubertal increases in the incentive salience for gaining social value and personal agency. This can provide a useful heuristic for understanding motivational proclivities (i.e. wanting to matter) that influence action-outcome learning as young adolescents are exploring and learning how to navigate increasingly complex social and relational environments. Adolescence also brings new capacities, motives, and opportunities for learning to care about and contribute to the benefit of others. Together, these create a window of opportunity: a sensitive period for learning to gain salient feelings of mattering through caring prosocial actions and valued societal contributions. Successfully discovering ways of mattering by doing things that matter to others may contribute to formative socio-emotional learning about self/other. Advances in understanding these social and relational learning processes and their neurodevelopmental underpinnings can inform strategies to improve developmental trajectories of social competence and wellbeing among adolescents growing up in a rapidly changing and increasingly techno-centric world.

Hippocampus and striatum show distinct contributions to longitudinal changes in value-based learning in middle childhood

The hippocampal-dependent memory system and striatal-dependent memory system modulate reinforcement learning depending on feedback timing in adults, but their contributions during development remain unclear. In a 2-year longitudinal study, 6-to-7-year-old children performed a reinforcement learning task in which they received feedback immediately or with a short delay following their response. Children's learning was found to be sensitive to feedback timing modulations in their reaction time and inverse temperature parameter, which quantifies value-guided decision-making. They showed longitudinal improvements towards more optimal value-based learning, and their hippocampal volume showed protracted maturation. Better delayed model-derived learning covaried with larger hippocampal volume longitudinally, in line with the adult literature. In contrast, a larger striatal volume in children was associated with both better immediate and delayed model-derived learning longitudinally. These findings show, for the first time, an early hippocampal contribution to the dynamic development of reinforcement learning in middle childhood, with neurally less differentiated and more cooperative memory systems than in adults.

A cognitive-computational account of mood swings in adolescence

Teenagers have a reputation for being fickle, in both their choices and their moods. This variability may help adolescents as they begin to independently navigate novel environments. Recently, however, adolescent moodiness has also been linked to psychopathology. Here, we consider adolescents' mood swings from a novel computational perspective, grounded in reinforcement learning (RL). This model proposes that mood is determined by surprises about outcomes in the environment, and how much we learn from these surprises. It additionally suggests that mood biases learning and choice in a bidirectional manner. Integrating independent lines of research, we sketch a cognitive-computational account of how adolescents' mood, learning, and choice dynamics influence each other, with implications for normative and psychopathological development.

Environmental statistics and experience shape risk-taking across adolescence

Adolescents are often portrayed as reckless risk-takers because of their immature brains. Recent research has cast doubt on this portrayal, identifying the environment as a moderator of risk-taking. However, the key features of environments that drive risk-taking behaviors are often underspecified. We call for greater attention to the environment by drawing on research showing that its statistical structure impacts future risk-taking as people learn from outcomes they experience after taking a risk. This opinion shows that adolescents are unlikely to experience harm from many risks because environmental statistics are skewed and favor safe experiences. Environmental statistics and experience suggest entry points for policy interventions by carefully timing risk warnings and leveraging peers' potential to shape the statistics of rewarding experiences.

Understanding patch foraging strategies across development

Patch foraging is a near-ubiquitous behaviour across the animal kingdom and characterises many decision-making domains encountered by humans. We review how a disposition to explore in adolescence may reflect the evolutionary conditions under which hunter-gatherers foraged for resources. We propose that neurocomputational mechanisms responsible for reward processing, learning, and cognitive control facilitate the transition from exploratory strategies in adolescence to exploitative strategies in adulthood - where individuals capitalise on known resources. This developmental transition may be disrupted by psychopathology, as there is emerging evidence of biases in explore/exploit choices in mental health problems. Explore/exploit choices may be an informative marker for mental health across development and future research should consider this feature of decision-making as a target for clinical intervention.

Expecting the unexpected: a review of learning under uncertainty across development

Many of our decisions take place under uncertainty. To successfully navigate the environment, individuals need to estimate the degree of uncertainty and adapt their behaviors accordingly by learning from experiences. However, uncertainty is a broad construct and distinct types of uncertainty may differentially influence our learning. We provide a semi-systematic review to illustrate cognitive and neurobiological processes involved in learning under two types of uncertainty: learning in environments with stochastic outcomes, and with volatile outcomes. We specifically reviewed studies (N = 26 studies) that included an adolescent population, because adolescence is a period in life characterized by heightened exploration and learning, as well as heightened uncertainty due to experiencing many new, often social, environments. Until now, reviews have not comprehensively compared learning under distinct types of uncertainties in this age range. Our main findings show that although the overall developmental patterns were mixed, most studies indicate that learning from stochastic outcomes, as indicated by increased accuracy in performance, improved with age. We also found that adolescents tended to have an advantage compared with adults and children when learning from volatile outcomes. We discuss potential mechanisms explaining these age-related differences and conclude by outlining future research directions.

Sensitivity and specificity in affective and social learning in adolescence

Adolescence is a period of heightened affective and social sensitivity. In this review we address how this increased sensitivity influences associative learning. Based on recent evidence from human and rodent studies, as well as advances in computational biology, we suggest that, compared to other age groups, adolescents show features of heightened Pavlovian learning but tend to perform worse than adults at instrumental learning. Because Pavlovian learning does not involve decision-making, whereas instrumental learning does, we propose that these developmental differences might be due to heightened sensitivity to rewards and threats in adolescence, coupled with a lower specificity of responding. We discuss the implications of these findings for adolescent mental health and education.

Diminished reinforcement sensitivity in adolescence is associated with enhanced response switching and reduced coding of choice probability in the medial frontal pole

Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.

Exploration heuristics decrease during youth

Deciding between exploring new avenues and exploiting known choices is central to learning, and this exploration-exploitation trade-off changes during development. Exploration is not a unitary concept, and humans deploy multiple distinct mechanisms, but little is known about their specific emergence during development. Using a previously validated task in adults, changes in exploration mechanisms were investigated between childhood (8-9 y/o, N = 26; 16 females), early (12-13 y/o, N = 38; 21 females), and late adolescence (16-17 y/o, N = 33; 19 females) in ethnically and socially diverse schools from disadvantaged areas. We find an increased usage of a computationally light exploration heuristic in younger groups, effectively accommodating their limited neurocognitive resources. Moreover, this heuristic was associated with self-reported, attention-deficit/hyperactivity disorder symptoms in this population-based sample. This study enriches our mechanistic understanding about how exploration strategies mature during development.

Social learning across adolescence: A Bayesian neurocognitive perspective

Adolescence is a period of social re-orientation in which we are generally more prone to peer influence and the updating of our beliefs based on social information, also called social learning, than in any other stage of our life. However, how do we know when to use social information and whose information to use and how does this ability develop across adolescence? Here, we review the social learning literature from a behavioral, neural and computational viewpoint, focusing on the development of brain systems related to executive functioning, value-based decision-making and social cognition. We put forward a Bayesian reinforcement learning framework that incorporates social learning about value associated with particular behavior and uncertainty in our environment and experiences. We discuss how this framework can inform us about developmental changes in social learning, including how the assessment of uncertainty and the ability to adaptively discriminate between information from different social sources change across adolescence. By combining reward-based decision-making in the domains of both informational and normative influence, this framework explains both negative and positive social peer influence in adolescence.

Social influence in adolescence as a double-edged sword

Social learning is fundamental to human development, helping individuals adapt to changing circumstances and cooperate in groups. During the formative years of adolescence, the social environment shapes people's socio-cognitive skills needed in adulthood. Although peer influence among adolescents is traditionally associated with risky and unruly conduct, with long-term negative effects on educational, economic and health outcomes, recent findings suggest that peers may also have a positive impact. Here, we present a series of experiments with 10-20-year-olds (n = 146) showing that positive and negative peer effects reflect a domain-general factor of social information use which declines during adolescence. Exposure to disobedient peers provoked rule breaking, and selfish peers reduced prosocial behaviour, particularly in early adolescence. However, compliant peers also promoted rule compliance and fair peers increased prosociality. A belief formation task further revealed that younger adolescents tend to assimilate social information, while older adolescents prioritize personal views. Our results highlight early adolescence as a key window for peer-based interventions to improve developmental trajectories.

Impaired learning to dissociate advantageous and disadvantageous risky choices in adolescents

Adolescence is characterized by a surge in maladaptive risk-taking behaviors, but whether and how this relates to developmental changes in experience-based learning is largely unknown. In this preregistered study, we addressed this issue using a novel task that allowed us to separate the learning-driven optimization of risky choice behavior over time from overall risk-taking tendencies. Adolescents (12-17 years old) learned to dissociate advantageous from disadvantageous risky choices less well than adults (20-35 years old), and this impairment was stronger in early than mid-late adolescents. Computational modeling revealed that adolescents' suboptimal performance was largely due to an inefficiency in core learning and choice processes. Specifically, adolescents used a simpler, suboptimal, expectation-updating process and a more stochastic choice policy. In addition, the modeling results suggested that adolescents, but not adults, overvalued the highest rewards. Finally, an exploratory latent-mixture model analysis indicated that a substantial proportion of the participants in each age group did not engage in experience-based learning but used a gambler's fallacy strategy, stressing the importance of analyzing individual differences. Our results help understand why adolescents tend to make more, and more persistent, maladaptive risky decisions than adults when the values of these decisions have to be learned from experience.

I know better! Emerging metacognition allows adolescents to ignore false advice

Adolescents aspire for independence. Successful independence means knowing when to rely on one's own knowledge and when to listen to others. A critical prerequisite thus is a well-developed metacognitive ability to accurately assess the quality of one's own knowledge. Little is known about whether the strive to become an independent decision maker in adolescence is underpinned by the necessary metacognitive skills. Here, we demonstrate that metacognition matures from childhood to adolescence (N = 107) and that this process coincides with greater independent decision-making. We show that adolescents, in contrast to children, take on others' advice less often, but only when the advice is misleading. Finally, we demonstrate that adolescents' reduced reliance on others' advice is explained by their increased metacognitive skills, suggesting that a developing ability to introspect may support independent decision-making in adolescence.

I'm not a little kid anymore! Reciprocal social influence in child-adult interaction

Human decisions are often influenced by others' opinions. This process is regulated by social norms: for instance, we tend to reciprocate the consideration received from others, independently of their reliability as information sources. Nonetheless, no study to date has investigated whether and how reciprocity modulates social influence in child-adult interaction. We tested 6-, 8- and 10-year-old children in a novel joint perceptual task. A child and an adult experimenter made perceptual estimates and then took turns in making a final decision, choosing between their own and partner's response. We manipulated the final choices of the adult partner, who in one condition chose often the child's estimates, whereas in another condition tended to confirm her own response. Results reveal that 10-year-old children reciprocated the consideration received from the partner, increasing their level of conformity to the adult's judgements when the partner had shown high consideration towards them. At the same time, 10-year-old children employed more elaborate decision criteria in choosing when trusting the adult partner compared to younger children and did not show egocentric biases in their final decisions. Our results shed light on the development of the cognitive and normative mechanisms modulating reciprocal social influence in child-adult interaction.

Effects of advice on experienced-based learning in adolescents and adults

Recent studies that compared effects of pre-learning advice on experience-based learning in adolescents and adults have yielded mixed results. Previous studies on this topic used choice tasks in which age-related differences in advice-related learning bias and exploratory choice behavior are difficult to dissociate. Moreover, these studies did not examine whether effects of advice depend on working memory load. In this preregistered study (in adolescents [13-15 years old] and adults [18-31 years old]), we addressed these issues by factorially combining advice and working memory load manipulations in an estimation task that does not require choices and hence eliminates the influence of known age-related differences in exploration. We found that advice guided participants' initial estimates in both age groups. When advice was correct, this improved estimation performance, especially in adolescents when working memory load was high. When advice was incorrect, it had a longer-lasting effect on adolescents' performance than on adults' performance. In contrast to previous findings in choice tasks, we found no evidence that advice biased learning in either age group. Taken together, our results suggest that learning in an estimation task improves between adolescence and adulthood but that the effects of advice on learning do not differ substantially between adolescents and adults.

Modeling changes in probabilistic reinforcement learning during adolescence

In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggest probabilistic learning may be inefficient in youths compared to adults, while others suggest it may be more efficient in youths in mid adolescence. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants' performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time scale); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development.

Preference uncertainty accounts for developmental effects on susceptibility to peer influence in adolescence

Adolescents are prone to social influence from peers, with implications for development, both adaptive and maladaptive. Here, using a computer-based paradigm, we replicate a cross-sectional effect of more susceptibility to peer influence in a large dataset of adolescents 14 to 24 years old. Crucially, we extend this finding by adopting a longitudinal perspective, showing that a within-person susceptibility to social influence decreases over a 1.5 year follow-up time period. Exploiting this longitudinal design, we show that susceptibility to social influences at baseline predicts an improvement in peer relations over the follow-up period. Using a Bayesian computational model, we demonstrate that in younger adolescents a greater tendency to adopt others’ preferences arises out of a higher uncertainty about their own preferences in the paradigmatic case of delay discounting (a phenomenon called ‘preference uncertainty’). This preference uncertainty decreases over time and, in turn, leads to a reduced susceptibility of one’s own behaviour to an influence from others. Neuro-developmentally, we show that a measure of myelination within medial prefrontal cortex, estimated at baseline, predicts a developmental decrease in preference uncertainty at follow-up. Thus, using computational and neural evidence, we reveal adaptive mechanisms underpinning susceptibility to social influence during adolescence.

People often change their preferences to conform with others. Using a longitudinal design, the authors show that such conformity decreases over the course of adolescence and that this reduction in conformity is accompanied by a decreasing degree of uncertainty about what to like.

The actions of others act as a pseudo-reward to drive imitation in the context of social reinforcement learning

While there is no doubt that social signals affect human reinforcement learning, there is still no consensus about how this process is computationally implemented. To address this issue, we compared three psychologically plausible hypotheses about the algorithmic implementation of imitation in reinforcement learning. The first hypothesis, decision biasing (DB), postulates that imitation consists in transiently biasing the learner's action selection without affecting their value function. According to the second hypothesis, model-based imitation (MB), the learner infers the demonstrator's value function through inverse reinforcement learning and uses it to bias action selection. Finally, according to the third hypothesis, value shaping (VS), the demonstrator's actions directly affect the learner's value function. We tested these three hypotheses in 2 experiments (N = 24 and N = 44) featuring a new variant of a social reinforcement learning task. We show through model comparison and model simulation that VS provides the best explanation of learner's behavior. Results replicated in a third independent experiment featuring a larger cohort and a different design (N = 302). In our experiments, we also manipulated the quality of the demonstrators' choices and found that learners were able to adapt their imitation rate, so that only skilled demonstrators were imitated. We proposed and tested an efficient meta-learning process to account for this effect, where imitation is regulated by the agreement between the learner and the demonstrator. In sum, our findings provide new insights and perspectives on the computational mechanisms underlying adaptive imitation in human reinforcement learning.

The rational use of causal inference to guide reinforcement learning strengthens with age

Beliefs about the controllability of positive or negative events in the environment can shape learning throughout the lifespan. Previous research has shown that adults' learning is modulated by beliefs about the causal structure of the environment such that they update their value estimates to a lesser extent when the outcomes can be attributed to hidden causes. This study examined whether external causes similarly influenced outcome attributions and learning across development. Ninety participants, ages 7 to 25 years, completed a reinforcement learning task in which they chose between two options with fixed reward probabilities. Choices were made in three distinct environments in which different hidden agents occasionally intervened to generate positive, negative, or random outcomes. Participants' beliefs about hidden-agent intervention aligned with the true probabilities of the positive, negative, or random outcome manipulation in each of the three environments. Computational modeling of the learning data revealed that while the choices made by both adults (ages 18-25) and adolescents (ages 13-17) were best fit by Bayesian reinforcement learning models that incorporate beliefs about hidden-agent intervention, those of children (ages 7-12) were best fit by a one learning rate model that updates value estimates based on choice outcomes alone. Together, these results suggest that while children demonstrate explicit awareness of the causal structure of the task environment, they do not implicitly use beliefs about the causal structure of the environment to guide reinforcement learning in the same manner as adolescents and adults.

Reinforcement learning across development: What insights can we draw from a decade of research?

The past decade has seen the emergence of the use of reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety of tasks and model variants, have reached convergent conclusions. In this review, we examine whether the tuning of model parameters that govern different aspects of learning and decision-making processes vary consistently as a function of age, and what neurocognitive developmental changes may account for differences in these parameter estimates across development. We explore whether patterns of developmental change in these estimates are better described by differences in the extent to which individuals adapt their learning processes to the statistics of different environments, or by more static learning biases that emerge across varied contexts. We focus specifically on learning rates and inverse temperature parameter estimates, and find evidence that from childhood to adulthood, individuals become better at optimally weighting recent outcomes during learning across diverse contexts and less exploratory in their value-based decision-making. We provide recommendations for how these two possibilities - and potential alternative accounts - can be tested more directly to build a cohesive body of research that yields greater insight into the development of core learning processes.

Social information use in adolescents: The impact of adults, peers and household composition

Social learning strategies are key for making adaptive decisions, but their ontogeny remains poorly understood. We investigate how social information use depends on its source (adults vs. peer), and how it is shaped by household composition (extended vs. nuclear), a factor known to modulate social development. Using a simple estimation task, we show that social information strongly impacts the behaviour of adolescents aged 11 to 15 years (N = 256), especially when its source is an adult. However, social information use does not depend on household composition: the relative impact of adults and peers was similar in adolescents from both household types. Furthermore, adolescents were found to directly copy others' estimates surprisingly frequently. This study provides novel insights into adolescents' social information use and contributes to understanding the ontogeny of social learning strategies.

Social Influence in Adolescent Decision-Making: A Formal Framework

Adolescence is a period of life during which peers play a pivotal role in decision-making. The narrative of social influence during adolescence often revolves around risky and maladaptive decisions, like driving under the influence, and using illegal substances (Steinberg, 2005). However, research has also shown that social influence can lead to increased prosocial behaviors (Van Hoorn et al., 2017) and a reduction in risk-taking (Braams et al., 2019). While many studies support the notion that adolescents are more sensitive to peer influence than children or adults, the developmental processes that underlie this sensitivity remain poorly understood. We argue that one important reason for this lack of understanding is the absence of precisely formulated models. To make a first step toward formal models of social influence during adolescence, we first identify three prominent verbal models of social influence in the literature: (1) social motivation, (2) reward sensitivity, and (3) distraction. We then illustrate how these can be translated into formal models, and how such formal models can inform experimental design and help identify developmental processes. Finally, by applying our formal models to existing datasets, we demonstrate the usefulness of formalization by synthesizing different studies with seemingly disparate results. We conclude with a discussion on how formal modeling can be utilized to better investigate the development of peer influence in adolescence.