Are adolescents more optimal decision-makers in novel environments? Examining the benefits of heightened exploration in a patch foraging paradigm

Reviewing explore/exploit decision-making as a transdiagnostic target for psychosis, depression, and anxiety

In many everyday decisions, individuals choose between trialling something novel or something they know well. Deciding when to try a new option or stick with an option that is already known to you, known as the "explore/exploit" dilemma, is an important feature of cognition that characterises a range of decision-making contexts encountered by humans. Recent evidence has suggested preferences in explore/exploit biases are associated with psychopathology, although this has typically been examined within individual disorders. The current review examined whether explore/exploit decision-making represents a promising transdiagnostic target for psychosis, depression, and anxiety. A systematic search of academic databases was conducted, yielding a total of 29 studies. Studies examining psychosis were mostly consistent in showing that individuals with psychosis explored more compared with individuals without psychosis. The literature on anxiety and depression was more heterogenous; some studies found that anxiety and depression were associated with more exploration, whereas other studies demonstrated reduced exploration in anxiety and depression. However, examining a subset of studies that employed case-control methods, there was some evidence that both anxiety and depression also were associated with increased exploration. Due to the heterogeneity across the literature, we suggest that there is insufficient evidence to conclude whether explore/exploit decision-making is a transdiagnostic target for psychosis, depression, and anxiety. However, alongside our advisory groups of lived experience advisors, we suggest that this context of decision-making is a promising candidate that merits further investigation using well-powered, longitudinal designs. Such work also should examine whether biases in explore/exploit choices are amenable to intervention.

How adolescents learn to build social bonds: A developmental computational account of social explore-exploit decision-making

Building social bonds is a critical task of adolescence that affords opportunities for learning, identity formation, and social support. Failing to develop close relationships in adolescence hinders adult interpersonal functioning and contributes to problems such as loneliness and depression. During adolescence, increased reward sensitivity and greater social flexibility both contribute to healthy social development, yet we lack a clear theory of how these processes interact to support social functioning. Here, we propose synthesizing these two literatures using a computational reinforcement learning framework that recasts how adolescents pursue and learn from social rewards as a social explore-exploit problem. To become socially skilled, adolescents must balance both their efforts to form individual bonds within specific groups and manage memberships across multiple groups to maximize access to social resources. We draw on insights from sociological studies on social capital in collective networks and neurocognitive research on foraging and cooperation to describe the social explore-exploit dilemma faced by adolescents navigating a modern world with increasing access to diverse resources and group memberships. Our account provides important new directions for examining the dynamics of adolescent behavior in social groups and understanding how social value computations can support positive relationships into adulthood.

Adolescent-to-adult gains in cognitive flexibility are adaptively supported by reward sensitivity, exploration, and neural variability

Cognitive flexibility exhibits dynamic changes throughout development, with different forms of flexibility showing dissociable developmental trajectories. In this review, we propose that an adolescent-specific mode of flexibility in the face of changing environmental contingencies supports the emergence of adolescent-to-adult gains in cognitive shifting efficiency. We first describe how cognitive shifting abilities monotonically improve from childhood to adulthood, accompanied by increases in brain state flexibility, neural variability, and excitatory/inhibitory balance. We next summarize evidence supporting the existence of a dopamine-driven, adolescent peak in flexible behavior that results in reward seeking, undirected exploration, and environmental sampling. We propose a neurodevelopmental framework that relates these adolescent behaviors to the refinement of neural phenotypes relevant to mature cognitive flexibility, and thus highlight the importance of the adolescent period in fostering healthy neurocognitive trajectories.

How adverse childhood experiences get under the skin: A systematic review, integration and methodological discussion on threat and reward learning mechanisms

Adverse childhood experiences (ACEs) are a major risk factor for the development of multiple psychopathological conditions, but the mechanisms underlying this link are poorly understood. Associative learning encompasses key mechanisms through which individuals learn to link important environmental inputs to emotional and behavioral responses. ACEs may impact the normative maturation of associative learning processes, resulting in their enduring maladaptive expression manifesting in psychopathology. In this review, we lay out a systematic and methodological overview and integration of the available evidence of the proposed association between ACEs and threat and reward learning processes. We summarize results from a systematic literature search (following PRISMA guidelines) which yielded a total of 81 articles (threat: n=38, reward: n=43). Across the threat and reward learning fields, behaviorally, we observed a converging pattern of aberrant learning in individuals with a history of ACEs, independent of other sample characteristics, specific ACE types, and outcome measures. Specifically, blunted threat learning was reflected in reduced discrimination between threat and safety cues, primarily driven by diminished responding to conditioned threat cues. Furthermore, attenuated reward learning manifested in reduced accuracy and learning rate in tasks involving acquisition of reward contingencies. Importantly, this pattern emerged despite substantial heterogeneity in ACE assessment and operationalization across both fields. We conclude that blunted threat and reward learning may represent a mechanistic route by which ACEs may become physiologically and neurobiologically embedded and ultimately confer greater risk for psychopathology. In closing, we discuss potentially fruitful future directions for the research field, including methodological and ACE assessment considerations.

Intolerance of uncertainty and psychological flexibility as predictors of mental health from adolescence to old age

PURPOSE

The COVID-19 pandemic brought with it significant social, economic and health uncertainties. These were proposed to impact young people more compared to adults, leading adolescents to report more mental health problems during the pandemic. The current study examined whether differences in cognitive risk (tolerance of uncertainty) and protective (psychological flexibility) factors accounted for age-related differences in depression and anxiety.

METHODS

These associations were investigated in the COVID-19 Risks Across the Lifespan (CORAL) cohort (N = 2280, 11-89 years).

RESULTS

The results showed that adolescents experienced greater intolerance of uncertainty and lower psychological flexibility compared to adults and older adults. Tolerance of uncertainty did not account for age-related differences in depression or anxiety. However, psychological flexibility conferred more protective advantage for anxiety in adults compared to adolescents.

CONCLUSION

The observed age-related differences in risk and protective factors advance our understanding of developmental vulnerabilities to depression and anxiety. Implications for mental health interventions in the context of future pandemics are discussed.

Do adolescents use choice to learn about their preferences? Development of value refinement and its associations with depressive symptoms in adolescence

Independent decision making requires forming stable estimates of one's preferences. We assessed whether adolescents learn about their preferences through choice deliberation and whether depressive symptoms disrupt this process. Adolescents aged 11-18 (N = 214; participated 2021-22; Female: 53.9%; White/Black/Asian/Mixed/Arab or Latin American: 26/21/19/9/8%) rated multiple activities, chose between pairs of activities and re-rated those activities. As expected, overall, participants uprated chosen and downrated unchosen activities (dz = .20). This value refinement through choice was not evident in younger participants but emerged across adolescence. Contrary to our predictions, depressive symptoms were associated with greater value refinement. Despite this, more depressed adolescents reported lower value certainty and choice confidence. The cognitive processes through which choice deliberation shapes preference develop over adolescence, and are disrupted in depression.

Goal-directed learning in adolescence: neurocognitive development and contextual influences

Adolescence is a time during which we transition to independence, explore new activities and begin pursuit of major life goals. Goal-directed learning, in which we learn to perform actions that enable us to obtain desired outcomes, is central to many of these processes. Currently, our understanding of goal-directed learning in adolescence is itself in a state of transition, with the scientific community grappling with inconsistent results. When we examine metrics of goal-directed learning through the second decade of life, we find that many studies agree there are steady gains in performance in the teenage years, but others report that adolescent goal-directed learning is already adult-like, and some find adolescents can outperform adults. To explain the current variability in results, sophisticated experimental designs are being applied to test learning in different contexts. There is also increasing recognition that individuals of different ages and in different states will draw on different neurocognitive systems to support goal-directed learning. Through adoption of more nuanced approaches, we can be better prepared to recognize and harness adolescent strengths and to decipher the purpose (or goals) of adolescence itself.

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Childhood unpredictability and the development of exploration

Early in development, the process of exploration helps children gather new information that fosters learning about the world. Yet, it is unclear how childhood experiences may influence the way humans approach new learning. What influences decisions to exploit known, familiar options versus trying a novel alternative? We found that childhood unpredictability, characterized by unpredictable caregiving and unstable living environments, was associated with reduced exploratory behavior. This effect holds while controlling for individual differences, including anxiety and stress. Individuals who perceived their childhoods as unpredictable explored less and were instead more likely to repeat previous choices (habitual responding). They were also more sensitive to uncertainty than to potential rewards, even when the familiar options yielded lower rewards. We examined these effects across multiple task contexts and via both in-person (N = 78) and online replication (N = 84) studies among 10- to 13-y-olds. Results are discussed in terms of the potential cascading effects of unpredictable environments on the development of decision-making and the effects of early experience on subsequent learning.

Understanding patch foraging strategies across development

Patch foraging is a near-ubiquitous behaviour across the animal kingdom and characterises many decision-making domains encountered by humans. We review how a disposition to explore in adolescence may reflect the evolutionary conditions under which hunter-gatherers foraged for resources. We propose that neurocomputational mechanisms responsible for reward processing, learning, and cognitive control facilitate the transition from exploratory strategies in adolescence to exploitative strategies in adulthood - where individuals capitalise on known resources. This developmental transition may be disrupted by psychopathology, as there is emerging evidence of biases in explore/exploit choices in mental health problems. Explore/exploit choices may be an informative marker for mental health across development and future research should consider this feature of decision-making as a target for clinical intervention.

Uncertainty as a driver of the youth mental health crisis

Mental health problems in young people have been on the rise for over a decade, with that trend accelerating during the pandemic. This review proposes that the catalyst effect of the pandemic offers insights into a key driver of increases in youth depression and anxiety: greater uncertainty. Uncertainty about many aspects of everyday life, including social connections, education, job security and health, increased during the pandemic, and this coincided with increasing rates of depression and anxiety. Lab-based developmental cognitive and clinical neuroscience research on tolerance of uncertainty and adolescent mental health shows that when adolescents fail to show age-typical tolerance of uncertainty, they are at greater risk of mental health problems. Avenues for future research to understand and promote tolerance of uncertainty in adolescents are proposed.

Novelty and uncertainty differentially drive exploration across development

Across the lifespan, individuals frequently choose between exploiting known rewarding options or exploring unknown alternatives. A large body of work has suggested that children may explore more than adults. However, because novelty and reward uncertainty are often correlated, it is unclear how they differentially influence decision-making across development. Here, children, adolescents, and adults (ages 8-27 years, N = 122) completed an adapted version of a recently developed value-guided decision-making task that decouples novelty and uncertainty. In line with prior studies, we found that exploration decreased with increasing age. Critically, participants of all ages demonstrated a similar bias to select choice options with greater novelty, whereas aversion to reward uncertainty increased into adulthood. Computational modeling of participant choices revealed that whereas adolescents and adults demonstrated attenuated uncertainty aversion for more novel choice options, children's choices were not influenced by reward uncertainty.

Exploratory preferences explain the human fascination for imaginary worlds in fictional stories

Imaginary worlds are present and often central in many of the most culturally successful modern narrative fictions, be it in novels (e.g., Harry Potter), movies (e.g., Star Wars), video games (e.g., The Legend of Zelda), graphic novels (e.g., One Piece) and TV series (e.g., Game of Thrones). We propose that imaginary worlds are popular because they activate exploratory preferences that evolved to help us navigate the real world and find new fitness-relevant information. Therefore, we hypothesize that the attraction to imaginary worlds is intrinsically linked to the desire to explore novel environments and that both are influenced by the same underlying factors. Notably, the inter-individual and cross-cultural variability of the preference for imaginary worlds should follow the inter-individual and cross-cultural variability of exploratory preferences (with the personality trait Openness-to-experience, age, sex, and ecological conditions). We test these predictions with both experimental and computational methods. For experimental tests, we run a pre-registered online experiment about movie preferences (N = 230). For computational tests, we leverage two large cultural datasets, namely the Internet Movie Database (N = 9424 movies) and the Movie Personality Dataset (N = 3.5 million participants), and use machine-learning algorithms (i.e., random forest and topic modeling). In all, consistent with how the human preference for spatial exploration adaptively varies, we provide empirical evidence that imaginary worlds appeal more to more explorative people, people higher in Openness-to-experience, younger individuals, males, and individuals living in more affluent environments. We discuss the implications of these findings for our understanding of the cultural evolution of narrative fiction and, more broadly, the evolution of human exploratory preferences.

Sensitivity and specificity in affective and social learning in adolescence

Adolescence is a period of heightened affective and social sensitivity. In this review we address how this increased sensitivity influences associative learning. Based on recent evidence from human and rodent studies, as well as advances in computational biology, we suggest that, compared to other age groups, adolescents show features of heightened Pavlovian learning but tend to perform worse than adults at instrumental learning. Because Pavlovian learning does not involve decision-making, whereas instrumental learning does, we propose that these developmental differences might be due to heightened sensitivity to rewards and threats in adolescence, coupled with a lower specificity of responding. We discuss the implications of these findings for adolescent mental health and education.

Confirmatory reinforcement learning changes with age during adolescence

Understanding how learning changes during human development has been one of the long-standing objectives of developmental science. Recently, advances in computational biology have demonstrated that humans display a bias when learning to navigate novel environments through rewards and punishments: they learn more from outcomes that confirm their expectations than from outcomes that disconfirm them. Here, we ask whether confirmatory learning is stable across development, or whether it might be attenuated in developmental stages in which exploration is beneficial, such as in adolescence. In a reinforcement learning (RL) task, 77 participants aged 11-32 years (four men, mean age = 16.26) attempted to maximize monetary rewards by repeatedly sampling different pairs of novel options, which varied in their reward/punishment probabilities. Mixed-effect models showed an age-related increase in accuracy as long as learning contingencies remained stable across trials, but less so when they reversed halfway through the trials. Age was also associated with a greater tendency to stay with an option that had just delivered a reward, more than to switch away from an option that had just delivered a punishment. At the computational level, a confirmation model provided increasingly better fit with age. This model showed that age differences are captured by decreases in noise or exploration, rather than in the magnitude of the confirmation bias. These findings provide new insights into how learning changes during development and could help better tailor learning environments to people of different ages. RESEARCH HIGHLIGHTS: Reinforcement learning shows age-related improvement during adolescence, but more in stable learning environments compared with volatile learning environments. People tend to stay with an option after a win more than they shift from an option after a loss, and this asymmetry increases with age during adolescence. Computationally, these changes are captured by a developing confirmatory learning style, in which people learn more from outcomes that confirm rather than disconfirm their choices. Age-related differences in confirmatory learning are explained by decreases in stochasticity, rather than changes in the magnitude of the confirmation bias.

Local and global reward learning in the lateral frontal cortex show differential development during human adolescence

Reward-guided choice is fundamental for adaptive behaviour and depends on several component processes supported by prefrontal cortex. Here, across three studies, we show that two such component processes, linking reward to specific choices and estimating the global reward state, develop during human adolescence and are linked to the lateral portions of the prefrontal cortex. These processes reflect the assignment of rewards contingently to local choices, or noncontingently, to choices that make up the global reward history. Using matched experimental tasks and analysis platforms, we show the influence of both mechanisms increase during adolescence (study 1) and that lesions to lateral frontal cortex (that included and/or disconnected both orbitofrontal and insula cortex) in human adult patients (study 2) and macaque monkeys (study 3) impair both local and global reward learning. Developmental effects were distinguishable from the influence of a decision bias on choice behaviour, known to depend on medial prefrontal cortex. Differences in local and global assignments of reward to choices across adolescence, in the context of delayed grey matter maturation of the lateral orbitofrontal and anterior insula cortex, may underlie changes in adaptive behaviour.

Positive and negative risk-taking: Age patterns and relations to domain-specific risk-taking

People take risks at all ages to achieve certain goals. Although these goals may be achieved through negative risks (e.g., adolescent drinking to impress their friends), people also take positive risks. Positive risks are theorized to help individuals achieve goals in developmentally appropriate and socially acceptable ways, such as initiating a new friendship as an adolescent, applying for a promotion as a young adult, or exploring a new hobby as a retiree. To test the hypothesis that people endorse different patterns of risk-taking across life, we examined age patterns in positive and negative risk-taking with a sample of individuals ranging in age from 12 to 71 years. In adults aged 19-71, we also examined to what extent positive and negative risk-taking are associated with domain-specific risk-taking and risk-taking propensity. Results indicated that positive risk-taking varied with age in the form of an inverted-U shape and peaked in middle adulthood. Negative antisocial risk-taking varied with age in the form of a U shape and was highest in adolescence. Negative health risk-taking varied with age in the form of an inverted-U shape and peaked in middle adulthood. In adults, greater positive risk-taking was associated with greater risk-taking in the social domain and greater risk-taking propensity. Greater negative risk-taking was associated with greater risk-taking in ethical and health/safety domains, and with greater risk-taking propensity. Altogether, this study is the first to demonstrate age patterns in positive and negative risk-taking across adolescence and adulthood. It also contributes to the validity of positive risk-taking as a construct distinct from negative risk-taking.

Real-World Exploration Increases Across Adolescence and Relates to Affect, Risk Taking, and Social Connectivity

Cross-species research suggests that exploratory behaviors increase during adolescence and relate to the social, affective, and risky behaviors characteristic of this developmental stage. However, how these typical adolescent behaviors manifest and relate in real-world settings remains unclear. Using geolocation tracking to quantify exploration—variability in daily movement patterns—over a 3-month period in 58 adolescents and adults (ages 13–27) in New York City, we investigated whether daily exploration varied with age and whether exploration related to social connectivity, risk taking, and momentary positive affect. In our cross-sectional sample, we found an association between daily exploration and age, with individuals near the transition to legal adulthood exhibiting the highest exploration levels. Days of higher exploration were associated with greater positive affect irrespective of age. Higher mean exploration was associated with greater social connectivity in all participants but was linked to higher risk taking selectively among adolescents. Our results highlight the interplay of exploration and socioemotional behaviors across development and suggest that societal norms may modulate their expression in naturalistic contexts.

Delay discounting and under-valuing of recent information predict poorer adherence to social distancing measures during the COVID-19 pandemic

The COVID-19 pandemic has brought about unprecedented global changes in individual and collective behaviour. To reduce the spread of the virus, public health bodies have promoted social distancing measures while attempting to mitigate their mental health consequences. The current study aimed to identify cognitive predictors of social distancing adherence and mental health symptoms, using computational models derived from delay discounting (the preference for smaller, immediate rewards over larger, delayed rewards) and patch foraging (the ability to trade-off between exploiting a known resource and exploring an unknown one). In a representative sample of the UK population (N = 442), we find that steeper delay discounting predicted poorer adherence to social distancing measures and greater sensitivity to reward magnitude during delay discounting predicted higher levels of anxiety symptoms. Furthermore, under-valuing recently sampled information during foraging independently predicted greater violation of lockdown guidance. Our results suggest that those who show greater discounting of delayed rewards struggle to maintain social distancing. Further, those who adapt faster to new information are better equipped to change their behaviour in response to public health measures. These findings can inform interventions that seek to increase compliance with social distancing measures whilst minimising negative repercussions for mental health.