Trait anxiety is associated with hidden state inference during aversive reversal learning

The influence of anxiety on exploration: A review of computational modeling studies

Exploratory behaviors can serve an adaptive role within novel or changing environments. Namely, they facilitate information gain, allowing an organism to maintain accurate beliefs about the environment and select actions that better maximize reward. However, finding the optimal balance between exploration and reward-seeking behavior - the so-called explore-exploit dilemma - can be challenging, as it requires sensitivity to one's own uncertainty and to the predictability of one's surroundings. Given the close relationship between uncertainty and anxiety, a body of work has now also emerged identifying associated effects on exploration. In particular, the field of computational psychiatry has begun to use cognitive computational models to characterize how anxiety may modulate underlying information processing mechanisms, such as estimation of uncertainty and the value of information, and how this might contribute to psychopathology. Here, we review computational modeling studies investigating how exploration is influenced by anxiety. While some apparent inconsistencies remain to be resolved, studies using reinforcement learning tasks suggest that directed (but not random) forms of exploration may be elevated by trait and/or cognitive anxiety, but reduced by state and/or somatic anxiety. Anxiety is also consistently associated with less exploration in foraging tasks. Some differences in exploration may further stem from how anxiety modulates changes in uncertainty over time (learning rates). Jointly, these results highlight important directions for future work in refining choice of tasks and anxiety measures and maintaining consistent methodology across studies.

Overgeneralization of conditioned fear in patients with social anxiety disorder

Introduction

While abnormal responses to threat, including overgeneralization to conditioned fear, have been postulated to play a critical role in pathological anxiety, the relevance of previous findings to social anxiety disorder (SAD) is unclear. We investigated conditioned and generalized fear responses in patients with SAD using socially relevant stimuli.

Methods

A total of 26 patients with SAD and 25 healthy controls participated in a fear conditioning and generalization paradigm consisting of two neutral faces as conditioned stimuli (CS+ or CS-) and an angry face with contemptuous comments as unconditioned stimuli. Eight morphed faces of two conditioned stimuli in each continuum were given to test generalization. Behavioral data and physiological responses were acquired.

Results

Successful conditioning was observed in the risk ratings for both groups, while only a marginal indication of conditioning was noted in physiological measures. During the generalization phase, patients rated the risk higher than CS- when the stimuli close to CS- contained a portion of CS+ features. Larger skin conductance responses to this stimulus were linked to higher fear of negative evaluation. In addition, patients spent a longer time evaluating safe and ambiguous stimuli than healthy controls and exhibited consistently high levels of subjective arousal.

Discussion

Taken together, our findings suggest that SAD patients may exhibit a tendency towards overgeneralization of fear responses and show distinct patterns in processing generalized threat stimuli compared to healthy controls. Even though overgeneralization was not evident in physiological measures, it is necessary to consider this behavioral characteristic in the clinical management of patients with SAD.

Reduction of Aversive Learning Rates in Pavlovian Conditioning by Angiotensin II Antagonist Losartan: A Randomized Controlled Trial

BACKGROUND

Angiotensin receptor blockade has been linked to aspects of aversive learning and memory formation and to the prevention of posttraumatic stress disorder symptom development.

METHODS

We investigated the influence of the angiotensin receptor blocker losartan on aversive Pavlovian conditioning using a probabilistic learning paradigm. In a double-blind, randomized, placebo-controlled design, we tested 45 (18 female) healthy volunteers during a baseline session, after application of losartan or placebo (drug session), and during a follow-up session. During each session, participants engaged in a task in which they had to predict the probability of an electrical stimulation on every trial while the true shock contingencies switched repeatedly between phases of high and low shock threat. Computational reinforcement learning models were used to investigate learning dynamics.

RESULTS

Acute administration of losartan significantly reduced participants' adjustment during both low-to-high and high-to-low threat changes. This was driven by reduced aversive learning rates in the losartan group during the drug session compared with baseline. The 50-mg drug dose did not induce reduction of blood pressure or change in reaction times, ruling out a general reduction in attention and engagement. Decreased adjustment of aversive expectations was maintained at a follow-up session 24 hours later.

CONCLUSIONS

This study shows that losartan acutely reduces Pavlovian learning in aversive environments, thereby highlighting a potential role of the renin-angiotensin system in anxiety development.

A Prelimbic Cortex-Thalamus Circuit Bidirectionally Regulates Innate and Stress-Induced Anxiety-Like Behavior

Anxiety-related disorders respond to cognitive behavioral therapies, which involved the medial prefrontal cortex (mPFC). Previous studies have suggested that subregions of the mPFC have different and even opposite roles in regulating innate anxiety. However, the specific causal targets of their descending projections in modulating innate anxiety and stress-induced anxiety have yet to be fully elucidated. Here, we found that among the various downstream pathways of the prelimbic cortex (PL), a subregion of the mPFC, PL-mediodorsal thalamic nucleus (MD) projection, and PL-ventral tegmental area (VTA) projection exhibited antagonistic effects on anxiety-like behavior, while the PL-MD projection but not PL-VTA projection was necessary for the animal to guide anxiety-related behavior. In addition, MD-projecting PL neurons bidirectionally regulated remote but not recent fear memory retrieval. Notably, restraint stress induced high-anxiety state accompanied by strengthening the excitatory inputs onto MD-projecting PL neurons, and inhibiting PL-MD pathway rescued the stress-induced anxiety. Our findings reveal that the activity of PL-MD pathway may be an essential factor to maintain certain level of anxiety, and stress increased the excitability of this pathway, leading to inappropriate emotional expression, and suggests that targeting specific PL circuits may aid the development of therapies for the treatment of stress-related disorders.

Directed exploration is elevated in affective disorders but reduced by an aversive interoceptive state induction

Elevated anxiety and uncertainty avoidance are known to exacerbate maladaptive choice in individuals with affective disorders. However, the differential roles of state vs. trait anxiety remain unclear, and underlying computational mechanisms have not been thoroughly characterized. In the present study, we investigated how a somatic (interoceptive) state anxiety induction influences learning and decision-making under uncertainty in individuals with clinically significant levels of trait anxiety. A sample of 58 healthy comparisons (HCs) and 61 individuals with affective disorders (iADs; i.e., depression and/or anxiety) completed a previously validated explore-exploit decision task, with and without an added breathing resistance manipulation designed to induce state anxiety. Computational modeling revealed a pattern in which iADs showed greater information-seeking (i.e., directed exploration; Cohen's d=.39, p=.039) in resting conditions, but that this was reduced by the anxiety induction. The affective disorders group also showed slower learning rates across conditions (Cohen's d=.52, p=.003), suggesting more persistent uncertainty. These findings highlight a complex interplay between trait anxiety and state anxiety. Specifically, while elevated trait anxiety is associated with persistent uncertainty, acute somatic anxiety can paradoxically curtail exploratory behaviors, potentially reinforcing maladaptive decision-making patterns in affective disorders.

Positive affect modulates memory by regulating the influence of reward prediction errors

How our decisions impact our memories is not well understood. Reward prediction errors (RPEs), the difference between expected and obtained reward, help us learn to make optimal decisions-providing a signal that may influence subsequent memory. To measure this influence and how it might go awry in mood disorders, we recruited a large cohort of human participants to perform a decision-making task in which perceptually memorable stimuli were associated with probabilistic rewards, followed by a recognition test for those stimuli. Computational modeling revealed that positive RPEs enhanced both the accuracy of memory and the temporal efficiency of memory search, beyond the contribution of perceptual information. Critically, positive affect upregulated the beneficial effect of RPEs on memory. These findings demonstrate how affect selectively regulates the impact of RPEs on memory, providing a computational mechanism for biased memory in mood disorders.

Heterogeneity in strategy use during arbitration between experiential and observational learning

To navigate our complex social world, it is crucial to deploy multiple learning strategies, such as learning from directly experiencing action outcomes or from observing other people's behavior. Despite the prevalence of experiential and observational learning in humans and other social animals, it remains unclear how people favor one strategy over the other depending on the environment, and how individuals vary in their strategy use. Here, we describe an arbitration mechanism in which the prediction errors associated with each learning strategy influence their weight over behavior. We designed an online behavioral task to test our computational model, and found that while a substantial proportion of participants relied on the proposed arbitration mechanism, there was some meaningful heterogeneity in how people solved this task. Four other groups were identified: those who used a fixed mixture between the two strategies, those who relied on a single strategy and non-learners with irrelevant strategies. Furthermore, groups were found to differ on key behavioral signatures, and on transdiagnostic symptom dimensions, in particular autism traits and anxiety. Together, these results demonstrate how large heterogeneous datasets and computational methods can be leveraged to better characterize individual differences.

Blocked training facilitates learning of multiple schemas

We all possess a mental library of schemas that specify how different types of events unfold. How are these schemas acquired? A key challenge is that learning a new schema can catastrophically interfere with old knowledge. One solution to this dilemma is to use interleaved training to learn a single representation that accommodates all schemas. However, another class of models posits that catastrophic interference can be avoided by splitting off new representations when large prediction errors occur. A key differentiating prediction is that, according to splitting models, catastrophic interference can be prevented even under blocked training curricula. We conducted a series of semi-naturalistic experiments and simulations with Bayesian and neural network models to compare the predictions made by the "splitting" versus "non-splitting" hypotheses of schema learning. We found better performance in blocked compared to interleaved curricula, and explain these results using a Bayesian model that incorporates representational splitting in response to large prediction errors. In a follow-up experiment, we validated the model prediction that inserting blocked training early in learning leads to better learning performance than inserting blocked training later in learning. Our results suggest that different learning environments (i.e., curricula) play an important role in shaping schema composition.

Trait anxiety is associated with hidden state inference during aversive reversal learning

Updating beliefs in changing environments can be driven by gradually adapting expectations or by relying on inferred hidden states (i.e. contexts), and changes therein. Previous work suggests that increased reliance on context could underly fear relapse phenomena that hinder clinical treatment of anxiety disorders. We test whether trait anxiety variations in a healthy population influence how much individuals rely on hidden-state inference. In a Pavlovian learning task, participants observed cues that predicted an upcoming electrical shock with repeatedly changing probability, and were asked to provide expectancy ratings on every trial. We show that trait anxiety is associated with steeper expectation switches after contingency reversals and reduced oddball learning. Furthermore, trait anxiety is related to better fit of a state inference, compared to a gradual learning, model when contingency changes are large. Our findings support previous work suggesting hidden-state inference as a mechanism behind anxiety-related to fear relapse phenomena.