The exploration-exploitation trade-off in a foraging task is affected by mood-related arousal and valence

Locus coeruleus integrity is related to an exploitation-based decision-making bias in older adulthood

Optimal decision-making balances exploration for new information against exploitation of known rewards, a process mediated by the locus coeruleus and its norepinephrine projections. We predicted that an exploitation-bias that emerges in older adulthood would be associated with lower microstructural integrity of the locus coeruleus. Leveraging in vivo histological methods from quantitative MRI-magnetic transfer saturation-we provide evidence that older age is associated with lower locus coeruleus integrity. Critically, we demonstrate that an exploitation bias in older adulthood, assessed with a foraging task, is sensitive and specific to lower locus coeruleus integrity. Because the locus coeruleus is uniquely vulnerable to Alzheimer's disease pathology, our findings suggest that aging, and a presymptomatic trajectory of Alzheimer's related decline, may fundamentally alter decision-making abilities in later life.

Models need mechanisms, but not labels

The target article proposes a model involving the important but not well-investigated topics of curiosity and creativity. The model, however, falls short of providing convincing explanations of the basic mechanisms underlying these phenomena. We outline the importance of mechanistic thinking in dealing with the concepts outlined in this article specifically and within psychology and cognitive neuroscience in general.

Dynamic computational phenotyping of human cognition

Computational phenotyping has emerged as a powerful tool for characterizing individual variability across a variety of cognitive domains. An individual's computational phenotype is defined as a set of mechanistically interpretable parameters obtained from fitting computational models to behavioural data. However, the interpretation of these parameters hinges critically on their psychometric properties, which are rarely studied. To identify the sources governing the temporal variability of the computational phenotype, we carried out a 12-week longitudinal study using a battery of seven tasks that measure aspects of human learning, memory, perception and decision making. To examine the influence of state effects, each week, participants provided reports tracking their mood, habits and daily activities. We developed a dynamic computational phenotyping framework, which allowed us to tease apart the time-varying effects of practice and internal states such as affective valence and arousal. Our results show that many phenotype dimensions covary with practice and affective factors, indicating that what appears to be unreliability may reflect previously unmeasured structure. These results support a fundamentally dynamic understanding of cognitive variability within an individual.

Meta-Analysis Reveals That Explore-Exploit Decisions are Dissociable by Activation in the Dorsal Lateral Prefrontal Cortex and the Anterior Cingulate Cortex

Explore-exploit research has challenges in generalizability due to a limited theoretical basis of exploration and exploitation. Neuroimaging can help identify whether explore-exploit decisions use an opponent processing system to address this issue. Thus, we conducted a coordinate-based meta-analysis (N=23 studies) where we found activation in the dorsal lateral prefrontal cortex and anterior cingulate cortex during exploration versus exploitation, providing some evidence for opponent processing. However, the conjunction of explore-exploit decisions was associated with activation in the dorsal anterior cingulate cortex, dorsal medial prefrontal cortex, and anterior insula, suggesting that these brain regions do not engage in opponent processing. Further, exploratory analyses revealed heterogeneity in brain responses between task types during exploration and exploitation respectively. Coupled with results suggesting that activation in exploration and exploitation decisions is generally more similar than it is different suggests there remain significant challenges toward characterizing explore-exploit decision making. Nonetheless, dlPFC and ACC activation differentiate explore and exploit decisions and identifying these responses can help in targeted interventions aimed at manipulating these decisions.

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Individuals who see the good in the bad engage distinctive default network coordination during post-encoding rest

Focusing on the upside of negative events often promotes resilience. Yet, the underlying mechanisms that allow some people to spontaneously see the good in the bad remain unclear. The broaden-and-build theory of positive emotion has long suggested that positive affect, including positivity in the face of negative events, is linked to idiosyncratic thought patterns (i.e., atypical cognitive responses). Yet, evidence in support of this view has been limited, in part, due to difficulty in measuring idiosyncratic cognitive processes as they unfold. To overcome this barrier, we applied Inter-Subject Representational Similarity Analysis to test whether and how idiosyncratic neural responding supports positive reactions to negative experience. We found that idiosyncratic functional connectivity patterns in the brain's default network while resting after a negative experience predicts more positive descriptions of the event. This effect persisted when controlling for connectivity 1) before and during the negative experience, 2) before, during, and after a neutral experience, and 3) between other relevant brain regions (i.e., the limbic system). The relationship between idiosyncratic default network responding and positive affect was largely driven by functional connectivity patterns between the ventromedial prefrontal cortex and the rest of the default network and occurred relatively quickly during rest. We identified post-encoding rest as a key moment and the default network as a key brain system in which idiosyncratic responses correspond with seeing the good in the bad.

Prolonged Physiological Stress Is Associated With a Lower Rate of Exploratory Learning That Is Compounded by Depression

BACKGROUND

Stress is a major risk factor for depression, and both are associated with important changes in decision-making patterns. However, decades of research have only weakly connected physiological measurements of stress to the subjective experience of depression. Here, we examined the relationship between prolonged physiological stress, mood, and explore-exploit decision making in a population navigating a dynamic environment under stress: health care workers during the COVID-19 pandemic.

METHODS

We measured hair cortisol levels in health care workers who completed symptom surveys and performed an explore-exploit restless-bandit decision-making task; 32 participants were included in the final analysis. Hidden Markov and reinforcement learning models assessed task behavior.

RESULTS

Participants with higher hair cortisol exhibited less exploration (r = -0.36, p = .046). Higher cortisol levels predicted less learning during exploration (β = -0.42, false discovery rate [FDR]-corrected p [pFDR] = .022). Importantly, mood did not independently correlate with cortisol concentration, but rather explained additional variance (β = 0.46, pFDR = .022) and strengthened the relationship between higher cortisol and lower levels of exploratory learning (β = -0.47, pFDR = .022) in a joint model. These results were corroborated by a reinforcement learning model, which revealed less learning with higher hair cortisol and low mood (β = -0.67, pFDR = .002).

CONCLUSIONS

These results imply that prolonged physiological stress may limit learning from new information and lead to cognitive rigidity, potentially contributing to burnout. Decision-making measures link subjective mood states to measured physiological stress, suggesting that they should be incorporated into future biomarker studies of mood and stress conditions.

The costs and benefits of psychedelics on cognition and mood

Anecdotal evidence has indicated that psychedelic substances may acutely enhance creative task performance, although empirical support for this claim is mixed at best. Clinical research has shown that psychedelics might have enduring effects on mood and well-being. However, there is no neurocognitive framework that ties acute changes in cognition to long-term effects in mood. In this review, we operationalize creativity within an emerging cognitive control framework and assess the current empirical evidence of the effects of psychedelics on creativity. Next, we leverage insights about the mechanisms and computations by which other psychoactive drugs act to enhance versus impair cognition, in particular to those that act on catecholamines, the neurophysiological consequences of which are relatively well understood. Finally, we use the same framework to link the suggested psychedelic-induced improvements in creativity with enduring psychedelic-induced improvements in mood.

Meta-control: From psychology to computational neuroscience

Research in the past decades shed light on the different mechanisms that underlie our capacity for cognitive control. However, the meta-level processes that regulate cognitive control itself remain poorly understood. Following the terminology from artificial intelligence, meta-control can be defined as a collection of mechanisms that (a) monitor the progress of controlled processing and (b) regulate the underlying control parameters in the service of current task goals and in response to internal or external constraints. From a psychological perspective, meta-control is an important concept because it may help explain and predict how and when human agents select different types of behavioral strategies. From a cognitive neuroscience viewpoint, meta-control is a useful concept for understanding the complex networks in the prefrontal cortex that guide higher-level behavior as well as their interactions with neuromodulatory systems (such as the dopamine or norepinephrine system). The purpose of the special issue is to integrate hitherto segregated strands of research across three different perspectives: 1) a psychological perspective that specifies meta-control processes on a functional level and aims to operationalize them in experimental tasks; 2) a computational perspective that builds on ideas from artificial intelligence to formalize normative solutions to meta-control problems; and 3) a cognitive neuroscience perspective that identifies neural correlates of and mechanisms underlying meta-control.