Exploration heuristics decrease during youth

Adolescent-to-adult gains in cognitive flexibility are adaptively supported by reward sensitivity, exploration, and neural variability

Cognitive flexibility exhibits dynamic changes throughout development, with different forms of flexibility showing dissociable developmental trajectories. In this review, we propose that an adolescent-specific mode of flexibility in the face of changing environmental contingencies supports the emergence of adolescent-to-adult gains in cognitive shifting efficiency. We first describe how cognitive shifting abilities monotonically improve from childhood to adulthood, accompanied by increases in brain state flexibility, neural variability, and excitatory/inhibitory balance. We next summarize evidence supporting the existence of a dopamine-driven, adolescent peak in flexible behavior that results in reward seeking, undirected exploration, and environmental sampling. We propose a neurodevelopmental framework that relates these adolescent behaviors to the refinement of neural phenotypes relevant to mature cognitive flexibility, and thus highlight the importance of the adolescent period in fostering healthy neurocognitive trajectories.

Protocol and preregistration for the CODEC project: measuring, modelling and mechanistically understanding the nature of cognitive variability in early childhood

BACKGROUND

Children's cognitive performance fluctuates across multiple timescales. However, fluctuations have often been neglected in favour of research into average cognitive performance, limiting the unique insights into cognitive abilities and development that cognitive variability may afford. Preliminary evidence suggests that greater variability is associated with increased symptoms of neurodevelopmental disorders, and differences in behavioural and neural functioning. The relative dearth of empirical work on variability, historically limited due to a lack of suitable data and quantitative methodology, has left crucial questions unanswered, which the CODEC (COgnitive Dynamics in Early Childhood) study aims to address.

METHOD

The CODEC cohort is an accelerated 3-year longitudinal study which encompasses 600 7-to-10-year-old children. Each year includes a 'burst' week (3 times per day, 5 days per week) of cognitive measurements on five cognitive domains (reasoning, working memory, processing speed, vocabulary, exploration), conducted both in classrooms and at home through experience sampling assessments. We also measure academic outcomes and external factors hypothesised to predict cognitive variability, including sleep, mood, motivation and background noise. A subset of 200 children (CODEC-MRI) are invited for two deep phenotyping sessions (in year 1 and year 3 of the study), including structural and functional magnetic resonance imaging, eye-tracking, parental measurements and questionnaire-based demographic and psychosocial measures. We will quantify developmental differences and changes in variability using Dynamic Structural Equation Modelling, allowing us to simultaneously capture variability and the multilevel structure of trials nested in sessions, days, children and classrooms.

DISCUSSION

CODEC's unique design allows us to measure variability across a range of different cognitive domains, ages, and temporal resolutions. The deep-phenotyping arm allows us to test hypotheses concerning variability, including the role of mind wandering, strategy exploration, mood, sleep, and brain structure. Due to CODEC's longitudinal nature, we are able to quantify which measures of variability at baseline predict long-term outcomes. In summary, the CODEC study is a unique longitudinal study combining experience sampling, an accelerated longitudinal 'burst' design, deep phenotyping, and cutting-edge statistical methodologies to better understand the nature, causes, and consequences of cognitive variability in children.

TRIAL REGISTRATION

ClinicalTrials.gov - NCT06330090.

Exploration, exploitation, and development: Developmental shifts in decision-making

Decision-making requires balancing exploration with exploitation, yet children are highly exploratory, with exploration decreasing with development. Less is known about what drives these changes. We examined the development of decision-making in 188 three- to eight-year-old children (M = 64 months; 98 girls) and 26 adults (M = 19 years; 13 women). Children were recruited from ethnically diverse suburban middle-class neighborhoods of Columbus, Ohio, USA. Results indicate that mature reward-based choices emerge relatively late in development, with children tending to over-explore. Computational modeling suggests that this exploration is systematic rather than random, as children tend to avoid repeating choices made on the previous trial. This pattern of exploration (reminiscent of novelty preference) decreased with development, whereas the tendency to exploit increased.

The explore/exploit trade-off: An ecologically valid and translational framework that can advance mechanistic understanding of eating disorders

The explore/exploit trade-off is a decision-making process that is conserved across species and balances exploring unfamiliar choices of unknown value with choosing familiar options of known value to maximize reward. This framework is rooted in behavioral ecology and has traditionally been used to study maladaptive versus adaptive non-human animal foraging behavior. Researchers have begun to recognize the potential utility of understanding human decision-making and psychopathology through the explore/exploit trade-off. In this article, we propose that explore/exploit trade-off holds promise for advancing our mechanistic understanding of decision-making processes that confer vulnerability for and maintain eating pathology due to its neurodevelopmental bases, conservation across species, and ability to be mathematically modeled. We present a model for how suboptimal explore/exploit decision-making can promote disordered eating and present recommendations for future research applying this framework to eating pathology. Taken together, the explore/exploit trade-off provides a translational framework for expanding etiologic and maintenance models of eating pathology, given developmental changes in explore/exploit decision-making that coincide in time with the emergence of eating pathology and evidence of biased explore/exploit decision-making in psychopathology. Additionally, understanding explore/exploit decision-making in eating disorders may improve knowledge of their underlying pathophysiology, informing targeted clinical interventions such as neuromodulation and pharmacotherapy. PUBLIC SIGNIFICANCE STATEMENT: The explore/exploit trade-off is a cross-species decision-making process whereby organisms choose between a known option with a known reward or sampling unfamiliar options. We hypothesize that imbalanced explore/exploit decision-making can promote disordered eating and present preliminary data. We propose that explore/exploit trade-off has significant potential to advance understanding of the neurocognitive and neurodevelopmental mechanisms of eating pathology, which could ultimately guide revisions of etiologic models and inform novel interventions.

Probing apathy in children and adolescents with the Apathy Motivation Index-Child version

Apathy is linked to mental health and altered neurocognitive functions such as learning and decision-making in healthy adults. Mental health problems typically begin to emerge during adolescence, yet little is known about how apathy develops due to an absence of quantitative measurements specific to young people. Here, we present and evaluate the Apathy Motivation Index-Child Version (AMI-CV) for children and adolescents. We show across two samples of young people (aged 8 to 17 years, total N = 191) tested in schools in the UK and on a smartphone app, that the AMI-CV is a short, psychometrically sound measure to assess levels of apathy and motivation in young people. Similar to adult versions, the AMI-CV captures three distinct apathy domains: Behavioural Activation, Social Motivation and Emotional Sensitivity. The AMI-CV showed excellent construct validity with an alternative measure of apathy and external validity replicating specific links with related mental health traits shown in adults. Our results provide a short measure of self-reported apathy in young people that enables research into apathy development. The AMI-CV can be used in conjunction with the adult version to investigate the impact of levels of apathy across the lifespan.

Exploring the steps of learning: computational modeling of initiatory-actions among individuals with attention-deficit/hyperactivity disorder

Attention-deficit/hyperactivity disorder (ADHD) is characterized by difficulty in acting in a goal-directed manner. While most environments require a sequence of actions for goal attainment, ADHD was never studied in the context of value-based sequence learning. Here, we made use of current advancements in hierarchical reinforcement-learning algorithms to track the internal value and choice policy of individuals with ADHD performing a three-stage sequence learning task. Specifically, 54 participants (28 ADHD, 26 controls) completed a value-based reinforcement-learning task that allowed us to estimate internal action values for each trial and stage using computational modeling. We found attenuated sensitivity to action values in ADHD compared to controls, both in choice and reaction-time variability estimates. Remarkably, this was found only for first-stage actions (i.e., initiatory actions), while for actions performed just before outcome delivery the two groups were strikingly indistinguishable. These results suggest a difficulty in following value estimation for initiatory actions in ADHD.

Staying and Returning dynamics of young children's attention

In this paper, we decompose selective sustained attending behavior into components of continuous attention maintenance and attentional transitions and study how each of these components develops in young children. Our results in two experiments suggest that changes in children's ability to return attention to a target locus after distraction ("Returning") play a crucial role in the development of selective sustained attention between the ages of 3.5-6 years, perhaps to a greater extent than changes in the ability to continuously maintain attention on the target ("Staying"). We further distinguish Returning from the behavior of transitioning attention away from task (i.e., becoming distracted) and investigate the relative contributions of bottom-up and top-down factors on these different types of attentional transitions. Overall, these results (a) suggest the importance of understanding the cognitive process of transitioning attention for understanding selective sustained attention and its development, (b) provide an empirical paradigm within which to study this process, and (c) begin to characterize basic features of this process, namely its development and its relative dependence on top-down and bottom-up influences on attention. RESEARCH HIGHLIGHTS: Young children exhibited an endogenously ability, Returning, to preferentially transition attention to task-relevant information over task-irrelevant information. Selective sustained attention and its development were decomposed into Returning and Staying, or task-selective attention maintenance, using novel eye-tracking-based measures. Returning improved between the ages of 3.5-6 years, to a greater extent than Staying. Improvements in Returning supported improvements in selective sustained attention between these ages.

Developmental changes in exploration resemble stochastic optimization

Human development is often described as a 'cooling off' process, analogous to stochastic optimization algorithms that implement a gradual reduction in randomness over time. Yet there is ambiguity in how to interpret this analogy, due to a lack of concrete empirical comparisons. Using data from n = 281 participants ages 5 to 55, we show that cooling off does not only apply to the single dimension of randomness. Rather, human development resembles an optimization process of multiple learning parameters, for example, reward generalization, uncertainty-directed exploration and random temperature. Rapid changes in parameters occur during childhood, but these changes plateau and converge to efficient values in adulthood. We show that while the developmental trajectory of human parameters is strikingly similar to several stochastic optimization algorithms, there are important differences in convergence. None of the optimization algorithms tested were able to discover reliably better regions of the strategy space than adult participants on this task.

Understanding patch foraging strategies across development

Patch foraging is a near-ubiquitous behaviour across the animal kingdom and characterises many decision-making domains encountered by humans. We review how a disposition to explore in adolescence may reflect the evolutionary conditions under which hunter-gatherers foraged for resources. We propose that neurocomputational mechanisms responsible for reward processing, learning, and cognitive control facilitate the transition from exploratory strategies in adolescence to exploitative strategies in adulthood - where individuals capitalise on known resources. This developmental transition may be disrupted by psychopathology, as there is emerging evidence of biases in explore/exploit choices in mental health problems. Explore/exploit choices may be an informative marker for mental health across development and future research should consider this feature of decision-making as a target for clinical intervention.

Neurocognitive basis of model-based decision making and its metacontrol in childhood

Human behavior is supported by both goal-directed (model-based) and habitual (model-free) decision-making, each differing in its flexibility, accuracy, and computational cost. The arbitration between habitual and goal-directed systems is thought to be regulated by a process known as metacontrol. However, how these systems emerge and develop remains poorly understood. Recently, we found that while children between 5 and 11 years displayed robust signatures of model-based decision-making, which increased during this developmental period, there were substantial individual differences in the display of metacontrol. Here, we inspect the neurocognitive basis of model-based decision-making and metacontrol in childhood and focus this investigation on executive functions, fluid reasoning, and brain structure. A total of 69 participants between the ages of 6-13 completed a two-step decision-making task and an extensive behavioral test battery. A subset of 44 participants also completed a structural magnetic resonance imaging scan. We find that individual differences in metacontrol are specifically associated with performance on an inhibition task and individual differences in thickness of dorsolateral prefrontal, temporal, and superior-parietal cortices. These brain regions likely reflect the involvement of cognitive processes crucial to metacontrol, such as cognitive control and contextual processing.

Diminished reinforcement sensitivity in adolescence is associated with enhanced response switching and reduced coding of choice probability in the medial frontal pole

Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.

Value-free random exploration is linked to impulsivity

Deciding whether to forgo a good choice in favour of exploring a potentially more rewarding alternative is one of the most challenging arbitrations both in human reasoning and in artificial intelligence. Humans show substantial variability in their exploration, and theoretical (but only limited empirical) work has suggested that excessive exploration is a critical mechanism underlying the psychiatric dimension of impulsivity. In this registered report, we put these theories to test using large online samples, dimensional analyses, and computational modelling. Capitalising on recent advances in disentangling distinct human exploration strategies, we not only demonstrate that impulsivity is associated with a specific form of exploration—value-free random exploration—but also explore links between exploration and other psychiatric dimensions.

The Stage 1 protocol for this Registered Report was accepted in principle on 19/03/2021. The protocol, as accepted by the journal, can be found at 10.6084/m9.figshare.14346506.v1.

Deciding between known rewarding options and exploring novel avenues is central to decision making. Humans show variability in their exploration. Here, the authors show that impulsivity is associated to an increased usage of a cognitively cheap (and sometimes sub-optimal) exploration strategy.