Impaired adaptation of learning to contingency volatility in internalizing psychopathology

Adapting to uncertainty: The role of anxiety and fear of negative evaluation in learning in social and non-social contexts

BACKGROUND

Navigating social situations can be challenging due to uncertainty surrounding the intentions and strategies of others, which remain hidden and subject to change. Prior research suggests that individuals with anxiety-related symptoms struggle to adapt their learning in uncertain, non-social environments. Anxiety-prone individuals encounter challenges in social functioning, yet research on learning under uncertainty in social contexts is limited. In this preregistered study, we investigated whether individuals with higher levels of trait anxiety and fear of negative evaluation encounter difficulties in adjusting their learning rates in social contexts with stable or volatile outcome contingencies.

METHODS

We implemented a modified trust game (N = 190), where participants either retained or lost their investments based on their interactions with two players in volatile or stable environments. Participants also completed a matching non-social control task involving interactions with slot machines.

RESULTS

Results from computational modeling revealed significantly higher learning rates in social compared to non-social settings. Trait anxiety did not affect the adaptability of learning rates. Individuals with heightened fear of negative evaluation were more sensitive to social compared to non-social outcomes, as reflected in their stay/switch behavior and, though less conclusive, in their learning rates.

LIMITATIONS

While transdiagnostic and dimensional approaches are important for investigating disturbed social functioning, the inclusion of clinical samples in future studies may contribute to a broader generalization of these findings regarding behavioral variances in uncertain social environments.

CONCLUSIONS

Individuals with increased fear of negative evaluation may demonstrate heightened sensitivity to learning in uncertain social contexts. This leads to heightened responsiveness to recent outcomes in their interactions with others, potentially contributing to their problems in social functioning.

The association between social rewards and anxiety: Links from neurophysiological analysis in virtual reality and social interaction game

Individuals' affective experience can be intricate, influenced by various factors including monetary rewards and social factors during social interaction. However, within this array of factors, divergent evidence has been considered as potential contributors to social anxiety. To gain a better understanding of the specific factors associated with anxiety during social interaction, we combined a social interaction task with neurophysiological recordings obtained through an anxiety-elicitation task conducted in a Virtual Reality (VR) environment. Employing inter-subject representational similarity analysis (ISRSA), we explored the potential linkage between individuals' anxiety neural patterns and their affective experiences during social interaction. Our findings suggest that, after controlling for other factors, the influence of the partner's emotional cues on individuals' affective experiences is specifically linked to their neural pattern of anxiety. This indicates that the emergence of anxiety during social interaction may be particularly associated with the emotional cues provided by the social partner, rather than individuals' own reward or prediction errors during social interaction. These results provide further support for the cognitive theory of social anxiety and extend the application of VR in future cognitive and affective studies.

Individuals with anxiety and depression use atypical decision strategies in an uncertain world

Previous studies on reinforcement learning have identified three prominent phenomena: (1) individuals with anxiety or depression exhibit a reduced learning rate compared to healthy subjects; (2) learning rates may increase or decrease in environments with rapidly changing (i.e. volatile) or stable feedback conditions, a phenomenon termed learning rate adaptation; and (3) reduced learning rate adaptation is associated with several psychiatric disorders. In other words, multiple learning rate parameters are needed to account for behavioral differences across participant populations and volatility contexts in this flexible learning rate (FLR) model. Here, we propose an alternative explanation, suggesting that behavioral variation across participant populations and volatile contexts arises from the use of mixed decision strategies. To test this hypothesis, we constructed a mixture-of-strategies (MOS) model and used it to analyze the behaviors of 54 healthy controls and 32 patients with anxiety and depression in volatile reversal learning tasks. Compared to the FLR model, the MOS model can reproduce the three classic phenomena by using a single set of strategy preference parameters without introducing any learning rate differences. In addition, the MOS model can successfully account for several novel behavioral patterns that cannot be explained by the FLR model. Preferences for different strategies also predict individual variations in symptom severity. These findings underscore the importance of considering mixed strategy use in human learning and decision-making and suggest atypical strategy preference as a potential mechanism for learning deficits in psychiatric disorders.

Transdiagnostic failure to adapt interoceptive precision estimates across affective, substance use, and eating disorders: A replication and extension of previous results

Recent Bayesian theories of interoception suggest that perception of bodily states rests upon a precision-weighted integration of afferent signals and prior beliefs. In a previous study, we fit a computational model of perception to behavior on a heartbeat tapping task to test whether aberrant precision-weighting could explain misestimation of cardiac states in psychopathology. We found that, during an interoceptive perturbation designed to amplify afferent signal precision (inspiratory breath-holding), healthy individuals increased the precision-weighting assigned to ascending cardiac signals (relative to resting conditions), while individuals with anxiety, depression, substance use disorders, and/or eating disorders did not. In this pre-registered study, we aimed to replicate and extend our prior findings in a new transdiagnostic patient sample (N = 285) similar to the one in the original study. As expected, patients in this new sample were also unable to adjust beliefs about the precision of cardiac signals - preventing the ability to accurately perceive changes in their cardiac state. Follow-up analyses combining samples from the previous and current study (N = 719) also afforded power to identify group differences between narrower diagnostic categories, and to examine predictive accuracy when logistic regression models were trained on one sample and tested on the other. With this confirmatory evidence in place, future studies should examine the utility of interoceptive precision measures in predicting treatment outcomes and test whether these computational mechanisms might represent novel therapeutic targets.

Moderate confirmation bias enhances decision-making in groups of reinforcement-learning agents

Humans tend to give more weight to information confirming their beliefs than to information that disconfirms them. Nevertheless, this apparent irrationality has been shown to improve individual decision-making under uncertainty. However, little is known about this bias' impact on decision-making in a social context. Here, we investigate the conditions under which confirmation bias is beneficial or detrimental to decision-making under social influence. To do so, we develop a Collective Asymmetric Reinforcement Learning (CARL) model in which artificial agents observe others' actions and rewards, and update this information asymmetrically. We use agent-based simulations to study how confirmation bias affects collective performance on a two-armed bandit task, and how resource scarcity, group size and bias strength modulate this effect. We find that a confirmation bias benefits group learning across a wide range of resource-scarcity conditions. Moreover, we discover that, past a critical bias strength, resource abundance favors the emergence of two different performance regimes, one of which is suboptimal. In addition, we find that this regime bifurcation comes with polarization in small groups of agents. Overall, our results suggest the existence of an optimal, moderate level of confirmation bias for decision-making in a social context.

Enhanced "learning to learn" through a hierarchical dual-learning system: the case of action video game players

In contrast to conventional cognitive training paradigms, where learning effects are specific to trained parameters, playing action video games has been shown to produce broad enhancements in many cognitive functions. These remarkable generalizations challenge the conventional theory of generalization that learned knowledge can be immediately applied to novel situations (i.e., immediate generalization). Instead, a new "learning to learn" theory has recently been proposed, suggesting that these broad generalizations are attained because action video game players (AVGPs) can quickly acquire the statistical regularities of novel tasks in order to increase the learning rate and ultimately achieve better performance. Although enhanced learning rate has been found for several tasks, it remains unclear whether AVGPs efficiently learn task statistics and use learned task knowledge to guide learning. To address this question, we tested 34 AVGPs and 36 non-video game players (NVGPs) on a cue-response associative learning task. Importantly, unlike conventional cognitive tasks with fixed task statistics, in this task, cue-response associations either remain stable or change rapidly (i.e., are volatile) in different blocks. To complete the task, participants should not only learn the lower-level cue-response associations through explicit feedback but also actively estimate the high-level task statistics (i.e., volatility) to dynamically guide lower-level learning. Such a dual learning system is modelled using a hierarchical Bayesian learning framework, and we found that AVGPs indeed quickly extract the volatility information and use the estimated higher volatility to accelerate learning of the cue-response associations. These results provide strong evidence for the "learning to learn" theory of generalization in AVGPs. Taken together, our work highlights enhanced hierarchical learning of both task statistics and cognitive abilities as a mechanism underlying the broad enhancements associated with action video game play.

Keeping an Eye Out for Change: Anxiety Disrupts Adaptive Resolution of Policy Uncertainty

BACKGROUND

Human learning unfolds under uncertainty. Uncertainty is heterogeneous with different forms exerting distinct influences on learning. While one can be uncertain about what to do to maximize rewarding outcomes, known as policy uncertainty, one can also be uncertain about general world knowledge, known as epistemic uncertainty (EU). In complex and naturalistic environments such as the social world, adaptive learning may hinge on striking a balance between attending to and resolving each type of uncertainty. Prior work illustrates that people with anxiety-those with increased threat and uncertainty sensitivity-learn less from aversive outcomes, particularly as outcomes become more uncertain. How does a learner adaptively trade-off between attending to these distinct sources of uncertainty to successfully learn about their social environment?

METHODS

We developed a novel eye-tracking method to capture highly granular estimates of policy uncertainty and EU based on gaze patterns and pupil diameter (a physiological estimate of arousal).

RESULTS

These empirically derived uncertainty measures revealed that humans (N = 94) flexibly switched between resolving policy uncertainty and EU to adaptively learn about which individuals can be trusted and which should be avoided. However, those with increased anxiety (n = 49) did not flexibly switch between resolving policy uncertainty and EU and instead expressed less uncertainty overall.

CONCLUSIONS

Combining modeling and eye-tracking techniques, we show that altered learning in people with anxiety emerged from an insensitivity to policy uncertainty and rigid choice policies, leading to maladaptive behaviors with untrustworthy people.

Distinct computational mechanisms of uncertainty processing explain opposing exploratory behaviors in anxiety and apathy

Decision-making in uncertain environments often leads to varied outcomes. Understanding how individuals interpret the causes of unexpected feedback is crucial for adaptive behavior and mental well-being. Uncertainty can be broadly categorized into two components: volatility and stochasticity. Volatility is about how quickly conditions change, impacting results. Stochasticity, on the other hand, refers to outcomes affected by random chance or "luck". Understanding these factors enables individuals to have more effective environmental analysis and strategy implementation (explore or exploit) for future decisions. This study investigates how anxiety and apathy, two prevalent affective states, influence the perceptions of uncertainty and exploratory behavior. Participants (N = 1001) completed a restless three-armed bandit task that was analyzed using latent state models. Anxious individuals perceived uncertainty as more volatile, leading to increased exploration and learning rates, especially after reward omission. Conversely, apathetic individuals viewed uncertainty as more stochastic, resulting in decreased exploration and learning rates. The perceived volatility-to-stochasticity ratio mediated the anxiety-exploration relationship post-adverse outcomes. Dimensionality reduction showed exploration and uncertainty estimation to be distinct but related latent factors shaping a manifold of adaptive behavior that is modulated by anxiety and apathy. These findings reveal distinct computational mechanisms for how anxiety and apathy influence decision-making, providing a framework for understanding cognitive and affective processes in neuropsychiatric disorders.

Heterogeneity in strategy use during arbitration between experiential and observational learning

To navigate our complex social world, it is crucial to deploy multiple learning strategies, such as learning from directly experiencing action outcomes or from observing other people's behavior. Despite the prevalence of experiential and observational learning in humans and other social animals, it remains unclear how people favor one strategy over the other depending on the environment, and how individuals vary in their strategy use. Here, we describe an arbitration mechanism in which the prediction errors associated with each learning strategy influence their weight over behavior. We designed an online behavioral task to test our computational model, and found that while a substantial proportion of participants relied on the proposed arbitration mechanism, there was some meaningful heterogeneity in how people solved this task. Four other groups were identified: those who used a fixed mixture between the two strategies, those who relied on a single strategy and non-learners with irrelevant strategies. Furthermore, groups were found to differ on key behavioral signatures, and on transdiagnostic symptom dimensions, in particular autism traits and anxiety. Together, these results demonstrate how large heterogeneous datasets and computational methods can be leveraged to better characterize individual differences.

Understanding anxiety through uncertainty quantification

Uncertainty has been a central concept in psychological theories of anxiety. However, this concept has been plagued by divergent connotations and operationalizations. The lack of consensus hinders the current search for cognitive and biological mechanisms of anxiety, jeopardizes theory creation and comparison, and restrains translation of basic research into improved diagnoses and interventions. Drawing upon uncertainty decomposition in Bayesian Decision Theory, we propose a well-defined conceptual structure of uncertainty in cognitive and clinical sciences, with a focus on anxiety. We discuss how this conceptual structure provides clarity and can be naturally applied to existing frameworks of psychopathology research. Furthermore, it allows formal quantification of various types of uncertainty that can benefit both research and clinical practice in the era of computational psychiatry.

Adaptive learning from outcome contingencies in eating-disorder risk groups

Eating disorders are characterised by altered eating patterns alongside overvaluation of body weight or shape, and have relatively low rates of successful treatment and recovery. Notably, cognitive inflexibility has been implicated in both the development and maintenance of eating disorders, and understanding the reasons for this inflexibility might indicate avenues for treatment development. We therefore investigate one potential cause of this inflexibility: an inability to adjust learning when outcome contingencies change. We recruited (n = 82) three groups of participants: those who had recovered from anorexia nervosa (RA), those who had high levels of eating disorder symptoms but no formal diagnosis (EA), and control participants (HC). They performed a reinforcement learning task (alongside eye-tracking) in which the volatility of wins and losses was independently manipulated. We predicted that both the RA and EA groups would adjust their learning rates less than the control participants. Unexpectedly, the RA group showed elevated adjustment of learning rates for both win and loss outcomes compared to control participants. The RA group also showed increased pupil dilation to stable wins and reduced pupil dilation to stable losses. Their learning rate adjustment was associated with the difference between their pupil dilation to volatile vs. stable wins. In conclusion, we find evidence that learning rate adjustment is unexpectedly higher in those who have recovered from anorexia nervosa, indicating that the relationship between eating disorders and cognitive inflexibility may be complex. Given our findings, investigation of noradrenergic agents may be valuable in the field of eating disorders.

Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning

Unavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. Such updating is however useful only if action values are sufficiently stable, something that a lack of control may impair. We examined whether self-reported stress uncontrollability during the first wave of the COVID-19 pandemic predicted impaired reward-learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning, and a greater proportion of probabilistic errors following negative feedback for correct choices, an effect mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward-structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression.

Reliability of gamified reinforcement learning in densely sampled longitudinal assessments

Reinforcement learning is a core facet of motivation and alterations have been associated with various mental disorders. To build better models of individual learning, repeated measurement of value-based decision-making is crucial. However, the focus on lab-based assessment of reward learning has limited the number of measurements and the test-retest reliability of many decision-related parameters is therefore unknown. In this paper, we present an open-source cross-platform application Influenca that provides a novel reward learning task complemented by ecological momentary assessment (EMA) of current mental and physiological states for repeated assessment over weeks. In this task, players have to identify the most effective medication by integrating reward values with changing probabilities to win (according to random Gaussian walks). Participants can complete up to 31 runs with 150 trials each. To encourage replay, in-game screens provide feedback on the progress. Using an initial validation sample of 384 players (9729 runs), we found that reinforcement learning parameters such as the learning rate and reward sensitivity show poor to fair intra-class correlations (ICC: 0.22-0.53), indicating substantial within- and between-subject variance. Notably, items assessing the psychological state showed comparable ICCs as reinforcement learning parameters. To conclude, our innovative and openly customizable app framework provides a gamified task that optimizes repeated assessments of reward learning to better quantify intra- and inter-individual differences in value-based decision-making over time.

Trait anxiety is associated with hidden state inference during aversive reversal learning

Updating beliefs in changing environments can be driven by gradually adapting expectations or by relying on inferred hidden states (i.e. contexts), and changes therein. Previous work suggests that increased reliance on context could underly fear relapse phenomena that hinder clinical treatment of anxiety disorders. We test whether trait anxiety variations in a healthy population influence how much individuals rely on hidden-state inference. In a Pavlovian learning task, participants observed cues that predicted an upcoming electrical shock with repeatedly changing probability, and were asked to provide expectancy ratings on every trial. We show that trait anxiety is associated with steeper expectation switches after contingency reversals and reduced oddball learning. Furthermore, trait anxiety is related to better fit of a state inference, compared to a gradual learning, model when contingency changes are large. Our findings support previous work suggesting hidden-state inference as a mechanism behind anxiety-related to fear relapse phenomena.

A low dimensional manifold of human exploratory behavior reveals opposing roles for apathy and anxiety

Exploration-exploitation decision-making is a feature of daily life that is altered in a number of neuropsychiatric conditions. Humans display a range of exploration and exploitation behaviors, which can be affected by apathy and anxiety. It remains unknown how factors underlying decision-making generate the spectrum of observed exploration-exploitation behavior and how they relate to states of anxiety and apathy. Here, we report a latent structure underlying sequential exploration and exploitation decisions that explains variation in anxiety and apathy. 1001 participants in a gender-balanced sample completed a three-armed restless bandit task along with psychiatric symptom surveys. Using dimensionality reduction methods, we found that decision sequences reduced to a low-dimensional manifold. The axes of this manifold explained individual differences in the balance between states of exploration and exploitation and the stability of those states, as determined by a statistical mechanics model of decision-making. Position along the balance axis was correlated with opposing symptoms of behavioral apathy and anxiety, while position along the stability axis correlated with the level of emotional apathy. This result resolves a paradox over how these symptoms can be correlated in samples but have opposite effects on behavior. Furthermore, this work provides a basis for using behavioral manifolds to reveal relationships between behavioral dynamics and affective states, with important implications for behavioral measurement approaches to neuropsychiatric conditions.

Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy

In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.

Expecting the unexpected: a review of learning under uncertainty across development

Many of our decisions take place under uncertainty. To successfully navigate the environment, individuals need to estimate the degree of uncertainty and adapt their behaviors accordingly by learning from experiences. However, uncertainty is a broad construct and distinct types of uncertainty may differentially influence our learning. We provide a semi-systematic review to illustrate cognitive and neurobiological processes involved in learning under two types of uncertainty: learning in environments with stochastic outcomes, and with volatile outcomes. We specifically reviewed studies (N = 26 studies) that included an adolescent population, because adolescence is a period in life characterized by heightened exploration and learning, as well as heightened uncertainty due to experiencing many new, often social, environments. Until now, reviews have not comprehensively compared learning under distinct types of uncertainties in this age range. Our main findings show that although the overall developmental patterns were mixed, most studies indicate that learning from stochastic outcomes, as indicated by increased accuracy in performance, improved with age. We also found that adolescents tended to have an advantage compared with adults and children when learning from volatile outcomes. We discuss potential mechanisms explaining these age-related differences and conclude by outlining future research directions.

Anxiety and depression related abnormalities in socio-affective learning

Affective distress (as observed in anxiety and depression) has been observed to be related to insufficient sensitivity to changing reinforcement during operant learning. Whether such findings are specific to anxiety or depression is unclear given a wider literature relating negative affect to abnormal learning and the possibility that relationships are not consistent across incentive types (i.e., punishment and reward) and outcomes (i.e., positive or negative). In two separate samples (n = 100; n = 88), participants completed an operant learning task with positive or negative, and neutral socio-affective feedback, designed to assess adaptive responses to changing environmental volatility. Individual parameter estimates were generated with hierarchical Bayesian modelling. Effects of manipulations were modelled by decomposing parameters into a linear combination of effects on the logit scale. While effects tended to support prior work, neither general affective distress nor anxiety or depression were consistently related to a decrease in the adaptive adjustment of learning-rates in response to changing environmental volatility (Sample 1: β_{α:volatility} = -0.01, 95 % HDI = -0.14, 0.13; Sample 2: β_{α:volatility} = -0.15, 95 % HDI = -0.37, 0.05). Interaction effects in Sample 1 suggested that while distress was associated with decrements in adaptive learning under punishment-maximisation, it was associated with improvements under reward-maximisation. While our results are broadly consistent with prior work, they suggest that the role of anxiety or depression in volatility learning, if present, is subtle and difficult to detect. Inconsistencies between our samples, along with issues of parameter identifiability complicated interpretation.

Neurocomputational mechanisms underlying fear-biased adaptation learning in changing environments

Humans are able to adapt to the fast-changing world by estimating statistical regularities of the environment. Although fear can profoundly impact adaptive behaviors, the computational and neural mechanisms underlying this phenomenon remain elusive. Here, we conducted a behavioral experiment (n = 21) and a functional magnetic resonance imaging experiment (n = 37) with a novel cue-biased adaptation learning task, during which we simultaneously manipulated emotional valence (fearful/neutral expressions of the cue) and environmental volatility (frequent/infrequent reversals of reward probabilities). Across 2 experiments, computational modeling consistently revealed a higher learning rate for the environment with frequent versus infrequent reversals following neutral cues. In contrast, this flexible adjustment was absent in the environment with fearful cues, suggesting a suppressive role of fear in adaptation to environmental volatility. This suppressive effect was underpinned by activity of the ventral striatum, hippocampus, and dorsal anterior cingulate cortex (dACC) as well as increased functional connectivity between the dACC and temporal-parietal junction (TPJ) for fear with environmental volatility. Dynamic causal modeling identified that the driving effect was located in the TPJ and was associated with dACC activation, suggesting that the suppression of fear on adaptive behaviors occurs at the early stage of bottom-up processing. These findings provide a neuro-computational account of how fear interferes with adaptation to volatility during dynamic environments.

Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty

People radically differ in how they cope with uncertainty. Clinical researchers describe a dispositional characteristic known as "intolerance of uncertainty", a tendency to find uncertainty aversive, reported to be elevated across psychiatric and neurodevelopmental conditions. Concurrently, recent research in computational psychiatry has leveraged theoretical work to characterise individual differences in uncertainty processing. Under this framework, differences in how people estimate different forms of uncertainty can contribute to mental health difficulties. In this review, we briefly outline the concept of intolerance of uncertainty within its clinical context, and we argue that the mechanisms underlying this construct may be further elucidated through modelling how individuals make inferences about uncertainty. We will review the evidence linking psychopathology to different computationally specified forms of uncertainty and consider how these findings might suggest distinct mechanistic routes towards intolerance of uncertainty. We also discuss the implications of this computational approach for behavioural and pharmacological interventions, as well as the importance of different cognitive domains and subjective experiences in studying uncertainty processing.

Identifying Transdiagnostic Mechanisms in Mental Health Using Computational Factor Modeling

Most psychiatric disorders do not occur in isolation, and most psychiatric symptom dimensions are not uniquely expressed within a single diagnostic category. Current treatments fail to work for around 25% to 40% of individuals, perhaps due at least in part to an overreliance on diagnostic categories in treatment development and allocation. In this review, we describe ongoing efforts in the field to surmount these challenges and precisely characterize psychiatric symptom dimensions using large-scale studies of unselected samples via remote, online, and "citizen science" efforts that take a dimensional, mechanistic approach. We discuss the importance that efforts to identify meaningful psychiatric dimensions be coupled with careful computational modeling to formally specify, test, and potentially falsify candidate mechanisms that underlie transdiagnostic symptom dimensions. We refer to this approach, i.e., where symptom dimensions are identified and validated against computationally well-defined neurocognitive processes, as computational factor modeling. We describe in detail some recent applications of this method to understand transdiagnostic cognitive processes that include model-based planning, metacognition, appetitive processing, and uncertainty estimation. In this context, we highlight how computational factor modeling has been used to identify specific associations between cognition and symptom dimensions and reveal previously obscured relationships, how findings generalize to smaller in-person clinical and nonclinical samples, and how the method is being adapted and optimized beyond its original instantiation. Crucially, we discuss next steps for this area of research, highlighting the value of more direct investigations of treatment response that bridge the gap between basic research and the clinic.

Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective

A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.

Belief updating in psychosis, depression and anxiety disorders: A systematic review across computational modelling approaches

Alterations in belief updating are proposed to underpin symptoms of psychiatric illness, including psychosis, depression, and anxiety. Key parameters underlying belief updating can be captured using computational modelling techniques, aiding the identification of unique and shared deficits, and improving diagnosis and treatment. We systematically reviewed research that applied computational modelling to probabilistic tasks measuring belief updating in stable and volatile (changing) environments, across clinical and subclinical psychosis (n = 17), anxiety (n = 9), depression (n = 9) and transdiagnostic samples (n = 9). Depression disorders related to abnormal belief updating in response to the valence of rewards, evidenced in both stable and volatile environments. Whereas psychosis and anxiety disorders were associated with difficulties adapting to changing contingencies specifically, indicating an inflexibility and/or insensitivity to environmental volatility. Higher-order learning models revealed additional difficulties in the estimation of overall environmental volatility across psychosis disorders, showing increased updating to irrelevant information. These findings stress the importance of investigating belief updating in transdiagnostic samples, using homogeneous experimental and computational modelling approaches.

Pupillary response in reward processing in adults with major depressive disorder in remission

OBJECTIVE

Major depressive disorder (MDD) is associated with impaired reward processing and reward learning. The literature is inconclusive regarding whether these impairments persist after remission. The current study examined reward processing during a probabilistic learning task in individuals in remission from MDD (n = 19) and never depressed healthy controls (n = 31) matched for age and sex. The outcome measures were pupil dilation (an indirect index of noradrenergic activity and arousal) and computational modeling parameters.

METHOD

Participants completed two versions (facial/nonfacial feedback) of probabilistic reward learning task with changing contingencies. Pupil dilation was measured with a corneal reflection eye tracker. The hypotheses and analysis plan were preregistered.

RESULT

Healthy controls had larger pupil dilation following losses than gains (p <.001), whereas no significant difference between outcomes was found in individuals with a history of MDD, resulting in an interaction between group and outcome (β = 0.81, SE = 0.34, t = 2.37, p = .018). The rMDD group also achieved lower mean score at the last trial (t[46.77] = 2.12, p = .040) as well as a smaller proportion of correct choices (t[46.70] = 2.09, p = .041) compared with healthy controls.

CONCLUSION

Impaired reward processing may persist after remission from MDD and could constitute a latent risk factor for relapse. Measuring pupil dilation in a reward learning task is a promising method for identifying reward processing abnormalities linked to MDD. The task is simple and noninvasive, which makes it feasible for clinical research.

Irrelevant Threats Linger and Affect Behavior in High Anxiety

Threat-related information attracts attention and disrupts ongoing behavior, and particularly so for more anxious individuals. Yet, it is unknown how and to what extent threat-related information leave lingering influences on behavior (e.g., by impeding ongoing learning processes). Here, human male and female participants (N = 47) performed probabilistic reinforcement learning tasks where irrelevant distracting faces (neutral, happy, or fearful) were presented together with relevant monetary feedback. Behavioral modeling was combined with fMRI data (N = 27) to explore the neurocomputational bases of learning relevant and irrelevant information. In two separate studies, individuals with high trait anxiety showed increased avoidance of objects previously paired with the combination of neutral monetary feedback and fearful faces (but not neutral or happy faces). Behavioral modeling revealed that high anxiety increased the integration of fearful faces during feedback learning, and fMRI results (regarded as provisional, because of a relatively small sample size) further showed that variance in the prediction error signal, uniquely accounted for by fearful faces, correlated more strongly with activity in the right DLPFC for more anxious individuals. Behavioral and neuronal dissociations indicated that the threat-related distractors did not simply disrupt learning processes. By showing that irrelevant threats exert long-lasting influences on behavior, our results extend previous research that separately showed that anxiety increases learning from aversive feedbacks and distractibility by threat-related information. Our behavioral results, combined with the proposed neurocomputational mechanism, may help explain how increased exposure to irrelevant affective information contributes to the acquisition of maladaptive behaviors in more anxious individuals.SIGNIFICANCE STATEMENT In modern-day society, people are increasingly exposed to various types of irrelevant information (e.g., intruding social media announcements). Yet, the neurocomputational mechanisms influenced by irrelevant information during learning, and their interactions with increasingly distracted personality types are largely unknown. Using a reinforcement learning task, where relevant feedback is presented together with irrelevant distractors (emotional faces), we reveal an interaction between irrelevant threat-related information (fearful faces) and interindividual anxiety levels. fMRI shows provisional evidence for an interaction between anxiety levels and the coupling between activity in the DLPFC and learning signals specifically elicited by fearful faces. Our study reveals how irrelevant threat-related information may become entrenched in the anxious psyche and contribute to long-lasting abnormal behaviors.

Deterministic and probabilistic regularities underlying risky choices are acquired in a changing decision context

Predictions supporting risky decisions could become unreliable when outcome probabilities temporarily change, making adaptation more challenging. Therefore, this study investigated whether sensitivity to the temporal structure in outcome probabilities can develop and remain persistent in a changing decision environment. In a variant of the Balloon Analogue Risk Task with 90 balloons, outcomes (rewards or balloon bursts) were predictable in the task's first and final 30 balloons and unpredictable in the middle 30 balloons. The temporal regularity underlying the predictable outcomes differed across three experimental conditions. In the deterministic condition, a repeating three-element sequence dictated the maximum number of pumps before a balloon burst. In the probabilistic condition, a single probabilistic regularity ensured that burst probability increased as a function of pumps. In the hybrid condition, a repeating sequence of three different probabilistic regularities increased burst probabilities. In every condition, the regularity was absent in the middle 30 balloons. Participants were not informed about the presence or absence of the regularity. Sensitivity to both the deterministic and hybrid regularities emerged and influenced risk taking. Unpredictable outcomes of the middle phase did not deteriorate this sensitivity. In conclusion, humans can adapt their risky choices in a changing decision environment by exploiting the statistical structure that controls how the environment changes.

Individual differences in naturalistic learning link negative emotionality to the development of anxiety

Organisms learn from prediction errors (PEs) to predict the future. Laboratory studies using small financial outcomes find that humans use PEs to update expectations and link individual differences in PE-based learning to internalizing disorders. Because of the low-stakes outcomes in most tasks, it is unclear whether PE learning emerges in naturalistic, high-stakes contexts and whether individual differences in PE learning predict psychopathology risk. Using experience sampling to assess 625 college students' expected exam grades, we found evidence of PE-based learning and a general tendency to discount negative PEs, an "optimism bias." However, individuals with elevated negative emotionality, a personality trait linked to the development of anxiety disorders, displayed a global pessimism and learning differences that impeded accurate expectations and predicted future anxiety symptoms. A sensitivity to PEs combined with an aversion to negative PEs may result in a pessimistic and inaccurate model of the world, leading to anxiety.

Trait somatic anxiety is associated with reduced directed exploration and underestimation of uncertainty

Anxiety has been related to decreased physical exploration, but past findings on the interaction between anxiety and exploration during decision making were inconclusive. Here we examined how latent factors of trait anxiety relate to different exploration strategies when facing volatility-induced uncertainty. Across two studies (total N = 985), we demonstrated that people used a hybrid of directed, random and undirected exploration strategies, which were respectively sensitive to relative uncertainty, total uncertainty and value difference. Trait somatic anxiety, that is, the propensity to experience physical symptoms of anxiety, was inversely correlated with directed exploration and undirected exploration, manifesting as a lesser likelihood for choosing the uncertain option and reducing choice stochasticity regardless of uncertainty. Somatic anxiety is also associated with underestimation of relative uncertainty. Together, these results reveal the selective role of trait somatic anxiety in modulating both uncertainty-driven and value-driven exploration strategies.

Different trajectories of depression, anxiety and anhedonia symptoms in the first 12 months of the COVID-19 pandemic in a UK longitudinal sample

BACKGROUND

While studies from the start of the COVID-19 pandemic have described initial negative effects on mental health and exacerbating mental health inequalities, longer-term studies are only now emerging.

METHOD

In total, 34 465 individuals in the UK completed online questionnaires and were re-contacted over the first 12 months of the pandemic. We used growth mixture modelling to identify trajectories of depression, anxiety and anhedonia symptoms using the 12-month data. We identified sociodemographic predictors of trajectory class membership using multinomial regression models.

RESULTS

Most participants had consistently low symptoms of depression or anxiety over the year of assessments (60%, 69% respectively), and a minority had consistently high symptoms (10%, 15%). We also identified participants who appeared to show improvements in symptoms as the pandemic progressed, and others who showed the opposite pattern, marked symptom worsening, until the second national lockdown. Unexpectedly, most participants showed stable low positive affect, indicating anhedonia, throughout the 12-month period. From regression analyses, younger age, reporting a previous mental health diagnosis, non-binary, or self-defined gender, and an unemployed or a student status were significantly associated with membership of the stable high symptom groups for depression and anxiety.

CONCLUSIONS

While most participants showed little change in their depression and anxiety symptoms across the first year of the pandemic, we highlight the divergent responses of subgroups of participants, who fared both better and worse around national lockdowns. We confirm that previously identified predictors of negative outcomes in the first months of the pandemic also predict negative outcomes over a 12-month period.

The Effects of Tinnitus in Probabilistic Learning Tasks: Protocol for an Ecological Momentary Assessment Study

BACKGROUND

Chronic tinnitus is an increasing worldwide health concern, causing a significant burden to the health care system each year. The COVID-19 pandemic has seen a further increase in reported cases. For people with tinnitus, symptoms are exacerbated because of social isolation and the elevated levels of anxiety and depression caused by quarantines and lockdowns. Although it has been reported that patients with tinnitus can experience changes in cognitive capabilities, changes in adaptive learning via decision-making tasks for people with tinnitus have not yet been investigated.

OBJECTIVE

In this study, we aim to assess state- and trait-related impairments in adaptive learning ability on probabilistic learning tasks among people with tinnitus. Given that performance in such tasks can be quantified through computational modeling methods using a small set of neural-informed model parameters, such approaches are promising in terms of the assessment of tinnitus severity. We will first examine baseline differences in the characterization of decision-making under uncertainty between healthy individuals and people with tinnitus in terms of differences in the parameters of computational models in a cross-sectional experiment. We will also investigate whether these computational markers, which capture characteristics of decision-making, can be used to understand the cognitive impact of tinnitus symptom fluctuations through a longitudinal experimental design.

METHODS

We have developed a mobile app, AthenaCX, to deliver e-consent and baseline tinnitus and psychological assessments as well as regular ecological momentary assessments (EMAs) of perceived tinnitus loudness and a web-based aversive version of a probabilistic decision-making task, which can be triggered based on the participants' responses to the EMA surveys. Computational models will be developed to fit participants' choice data in the task, and cognitive parameters will be estimated to characterize participants' current ability to adapt learning to the change of the simulated environment at each session when the task is triggered. Linear regression analysis will be conducted to evaluate the impacts of baseline tinnitus severity on adapting decision-making performance. Repeated measures linear regression analysis will be used to examine model-derived parameters of decision-making in measuring real-time perceived tinnitus loudness fluctuations.

RESULTS

Ethics approval was received in December 2020 from Dublin City University (DCUREC/2021/070). The implementation of the experiments, including both the surveys and the web-based decision-making task, has been prepared. Recruitment flyers have been shared with audiologists, and a video instruction has been created to illustrate to the participants how to participate in the experiment. We expect to finish data collection over 12 months and complete data analysis 6 months after this. The results are expected to be published in December 2023.

CONCLUSIONS

We believe that EMA with context-aware triggering can facilitate a deeper understanding of the effects of tinnitus symptom severity upon decision-making processes as measured outside of the laboratory.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/36583.

Sometimes I feel the fear of uncertainty: How intolerance of uncertainty and trait anxiety impact fear acquisition, extinction and the return of fear

It is hypothesized that the ability to discriminate between threat and safety is impaired in individuals with high dispositional negativity, resulting in maladaptive behavior. A large body of research investigated differential learning during fear conditioning and extinction protocols depending on individual differences in intolerance of uncertainty (IU) and trait anxiety (TA), two closely-related dimensions of dispositional negativity, with heterogenous results. These might be due to varying degrees of induced threat/safety uncertainty. Here, we compared two groups with high vs. low IU/TA during periods of low (instructed fear acquisition) and high levels of uncertainty (delayed non-instructed extinction training and reinstatement). Dependent variables comprised subjective (US expectancy, valence, arousal), psychophysiological (skin conductance response, SCR, and startle blink), and neural (fMRI BOLD) measures of threat responding. During fear acquisition, we found strong threat/safety discrimination for both groups. During early extinction (high uncertainty), the low IU/TA group showed an increased physiological response to the safety signal, resulting in a lack of CS discrimination. In contrast, the high IU/TA group showed strong initial threat/safety discrimination in physiology, lacking discriminative learning on startle, and reduced neural activation in regions linked to threat/safety processing throughout extinction training indicating sustained but non-adaptive and rigid responding. Similar neural patterns were found after the reinstatement test. Taken together, we provide evidence that high dispositional negativity, as indicated here by IU and TA, is associated with greater responding to threat cues during the beginning of delayed extinction, and, thus, demonstrates altered learning patterns under changing environments.

The role of intolerance of uncertainty when solving the exploration-exploitation dilemma

When making behavioral decisions, individuals need to balance between exploiting known options or exploring new ones. How individuals solve this exploration-exploitation dilemma (EED) is a key research question across psychology, leading to attempting to disentangle the cognitive mechanisms behind it. A potential predictive factor of performance in an EED is intolerance of uncertainty (IU), an individual difference factor referring to the extent to which uncertain situations are reported to be aversive. Here, we present the results of a series of exploratory analyses in which we tested the relationship between IU and performance in an EED task. For this, we compiled data from 3 experiments, in which participants received the opportunity to exploit different movements in order to avoid a painful stimulus and approach rewards. For decomposing performance in this task, we used different computational models previously employed in studies on the EED. Then, the parameters of the winning model were correlated with the scores of participants in the IU scale. Correlational and cluster analyses, within both frequentists and Bayesian frameworks, did not provide strong evidence for a relation between EED and IU, apart from the decay rate and the subscale "tendency to become paralyzed in the face of uncertainty". Given the theoretical relation between EED and IU, we propose research with different experimental paradigms.

Biased belief priors versus biased belief updating: Differential correlates of depression and anxiety

Individuals prone to anxiety and depression often report beliefs and make judgements about themselves that are more negative than those reported by others. We use computational modeling of a richly naturalistic task to disentangle the role of negative priors versus negatively biased belief updating and to investigate their association with different dimensions of Internalizing psychopathology. Undergraduate participants first provided profiles for a hypothetical tech internship. They then viewed pairs of other profiles and selected the individual they would prefer to work alongside out of each pair. In a subsequent phase of the experiment, participants made judgments about their relative popularity as hypothetical internship partners both before any feedback and after each of 20 items of feedback revealing whether or not they had been selected as the preferred teammate from a given pairing. Scores on latent factors of general negative affect, anxiety-specific affect and depression-specific affect were estimated using participants' self-report scores on standardized measures of anxiety and depression together with factor loadings from a bifactor analysis conducted previously. Higher scores on the depression-specific factor were linked to more negative prior beliefs but were not associated with differences in belief updating. In contrast, higher scores on the anxiety-specific factor were associated with a negative bias in belief updating but no difference in prior beliefs. These findings indicate that, to at least some extent, distinct processes may impact the formation of belief priors and in-the-moment belief updating and that these processes may be differentially disrupted in depression and anxiety. Future directions for enquiry include examination of the possibility that prior beliefs biases in depression might reflect generalization from prior experiences or global schema whereas belief updating biases in anxiety might be more situationally specific.

Rumination Derails Reinforcement Learning with Possible Implications for Ineffective Behavior

How does rumination affect reinforcement learning-the ubiquitous process by which we adjust behavior after error in order to behave more effectively in the future? In a within-subject design (n=49), we tested whether experimentally manipulated rumination disrupts reinforcement learning in a multidimensional learning task previously shown to rely on selective attention. Rumination impaired performance, yet unexpectedly this impairment could not be attributed to decreased attentional breadth (quantified using a "decay" parameter in a computational model). Instead, trait rumination (between subjects) was associated with higher decay rates (implying narrower attention), yet not with impaired performance. Our task-performance results accord with the possibility that state rumination promotes stress-generating behavior in part by disrupting reinforcement learning. The trait-rumination finding accords with the predictions of a prominent model of trait rumination (the attentional-scope model). More work is needed to understand the specific mechanisms by which state rumination disrupts reinforcement learning.

The computational roots of positivity and confirmation biases in reinforcement learning

Humans do not integrate new information objectively: outcomes carrying a positive affective value and evidence confirming one's own prior belief are overweighed. Until recently, theoretical and empirical accounts of the positivity and confirmation biases assumed them to be specific to 'high-level' belief updates. We present evidence against this account. Learning rates in reinforcement learning (RL) tasks, estimated across different contexts and species, generally present the same characteristic asymmetry, suggesting that belief and value updating processes share key computational principles and distortions. This bias generates over-optimistic expectations about the probability of making the right choices and, consequently, generates over-optimistic reward expectations. We discuss the normative and neurobiological roots of these RL biases and their position within the greater picture of behavioral decision-making theories.

Hyperreactivity to uncertainty is a key feature of subjective cognitive impairment

With an increasingly ageing global population, more people are presenting with concerns about their cognitive function, but not all have an underlying neurodegenerative diagnosis. Subjective cognitive impairment (SCI) is a common condition describing self-reported deficits in cognition without objective evidence of cognitive impairment. Many individuals with SCI suffer from depression and anxiety, which have been hypothesised to account for their cognitive complaints. Despite this association between SCI and affective features, the cognitive and brain mechanisms underlying SCI are poorly understood. Here, we show that people with SCI are hyperreactive to uncertainty and that this might be a key mechanism accounting for their affective burden. Twenty-seven individuals with SCI performed an information sampling task, where they could actively gather information prior to decisions. Across different conditions, SCI participants sampled faster and obtained more information than matched controls to resolve uncertainty. Remarkably, despite their ‘urgent’ sampling behaviour, SCI participants were able to maintain their efficiency. Hyperreactivity to uncertainty indexed by this sampling behaviour correlated with the severity of affective burden including depression and anxiety. Analysis of MRI resting functional connectivity revealed that SCI participants had stronger insular-hippocampal connectivity compared to controls, which also correlated with faster sampling. These results suggest that altered uncertainty processing is a key mechanism underlying the psycho-cognitive manifestations in SCI and implicate a specific brain network target for future treatment.

Childhood immuno-metabolic markers and risk of depression and psychosis in adulthood: A prospective birth cohort study

BACKGROUND

Metabolic and inflammatory disorders commonly co-occur with depression and psychosis, with emerging evidence implicating immuno-metabolic dysfunction in their aetiology. Previous studies have reported metabolic dysfunction and inflammation in adults with depression and psychosis. However, longitudinal studies testing the direction of association, and the effects of different dimensions of early-life immuno-metabolic dysfunction on adult psychopathology are limited.

METHODS

Using data from 3258 birth cohort participants we examined longitudinal associations of three metabolic hormones (leptin, adiponectin, insulin) at age 9 with risks for depression- and psychosis-spectrum outcomes at age 24. In addition, using nine immuno-metabolic biomarkers (leptin, adiponectin, insulin, interleukin-6, C-Reactive protein, low density lipoprotein, high density lipoprotein, triglycerides, and BMI), we constructed an exploratory bifactor model showing a general immuno-metabolic factor and three specific factors (adiposity, inflammation, and insulin resistance), which were also used as exposures.

RESULTS

Childhood leptin was associated with adult depressive episode (adjusted odds ratio (aOR)= 1.31; 95% CI, 1.02-1.71) and negative symptoms (aOR=1.15; 95% CI, 1.07-1.24), but not positive psychotic symptoms. The general immuno-metabolic factor was associated with atypical depressive symptoms (aOR=1.07; 95% CI, 1.01-1.14) and psychotic experiences (aOR=1.21; 95% CI, 1.02-1.44). The adiposity factor was associated with negative symptoms (aOR=1.07; 95% CI 1.02-1.12). Point estimates tended to be larger in women, though 95% credible intervals overlapped with those for men. In women, the inflammatory factor was associated with depressive episodes (aOR=1.27; 95% CI, 1.03-1.57).

CONCLUSIONS

While general immuno-metabolic dysfunction in childhood may contribute to risks for both psychotic and depressive symptoms in adulthood, childhood adiposity and inflammation appear to be particularly linked to affective (depressive and negative), but not positive psychotic symptoms.

Computational modeling of threat learning reveals links with anxiety and neuroanatomy in humans

Influential theories implicate variations in the mechanisms supporting threat learning in the severity of anxiety symptoms. We use computational models of associative learning in conjunction with structural imaging to explicate links among the mechanisms underlying threat learning, their neuroanatomical substrates, and anxiety severity in humans. We recorded skin-conductance data during a threat-learning task from individuals with and without anxiety disorders (N=251; 8-50 years; 116 females). Reinforcement-learning model variants quantified processes hypothesized to relate to anxiety: threat conditioning, threat generalization, safety learning, and threat extinction. We identified the best-fitting models for these processes and tested associations among latent learning parameters, whole-brain anatomy, and anxiety severity. Results indicate that greater anxiety severity related specifically to slower safety learning and slower extinction of response to safe stimuli. Nucleus accumbens gray-matter volume moderated learning-anxiety associations. Using a modeling approach, we identify computational mechanisms linking threat learning and anxiety severity and their neuroanatomical substrates.

Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis

IMPORTANCE

Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent.

OBJECTIVE

To assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals.

DATA SOURCES

Web of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood).

STUDY SELECTION

Studies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models.

DATA EXTRACTION AND SYNTHESIS

Articles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected.

MAIN OUTCOMES AND MEASURES

The primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate).

RESULTS

A total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004).

CONCLUSIONS AND RELEVANCE

The simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders.

A neural and behavioral trade-off between value and uncertainty underlies exploratory decisions in normative anxiety

Exploration reduces uncertainty about the environment and improves the quality of future decisions, but at the cost of provisional uncertain and suboptimal outcomes. Although anxiety promotes intolerance to uncertainty, it remains unclear whether and by which mechanisms anxiety relates to exploratory decision-making. We use a dynamic three-armed-bandit task and find that higher trait-anxiety is associated with increased exploration, which in turn harms overall performance. We identify two distinct behavioral sources: first, decisions made by anxious individuals are guided toward reduction of uncertainty; and second, decisions are less guided by immediate value gains. These findings are similar in both loss and gain domains, and further demonstrate that an affective trait relates to exploration and results in an inverse-U-shaped relationship between anxiety and overall performance. Additional imaging data (fMRI) suggests that normative anxiety correlates negatively with the representation of expected-value in the dorsal-anterior-cingulate-cortex, and in contrast, positively with the representation of uncertainty in the anterior-insula. We conclude that a trade-off between value-gains and uncertainty-reduction entails maladaptive decision-making in individuals with higher normal-range anxiety.

The role of intolerance of uncertainty in classical threat conditioning: Recent developments and directions for future research

Intolerance of uncertainty (IU), the tendency to find uncertainty aversive, is an important transdiagnostic dimension in mental health disorders. Over the last decade, there has been a surge of research on the role of IU in classical threat conditioning procedures, which serve as analogues to the development, treatment, and relapse of anxiety, obsessive-compulsive, and trauma- and stressor-related disorders. This review provides an overview of the existing literature on IU in classical threat conditioning procedures. The review integrates findings based on the shared or discrete parameters of uncertainty embedded within classical threat conditioning procedures. Under periods of unexpected uncertainty, where threat and safety contingencies change, high IU, over other self-reported measures of anxiety, is specifically associated with poorer threat extinction learning and retention, as well as overgeneralisation. Under periods of estimation and expected uncertainty, where the parameters of uncertainty are being learned or have been learned, such as threat acquisition training and avoidance learning, the findings are mixed for IU. These findings provide evidence that individual differences in IU play a significant role in maintaining learned fear and anxiety, particularly under volatile environments. Recommendations for future research are outlined, with discussion focusing on how parameters of uncertainty can be better defined to capture how IU is involved in the maintenance of learned fear and anxiety. Such work will be crucial for understanding the role of IU in neurobiological models of uncertainty-based maintenance of fear and anxiety and inform translational work aiming to improve the diagnosis and treatment of relevant psychopathology.