Associability-modulated loss learning is increased in posttraumatic stress disorder

Reinforcement learning processes as forecasters of depression remission

BACKGROUND

Aspects of reinforcement learning have been associated with specific depression symptoms and may inform the course of depressive illness.

METHODS

We applied support vector machines to investigate whether blood‑oxygen-level dependent (BOLD) responses linked with neural prediction error (nPE) and neural expected value (nEV) from a probabilistic learning task could forecast depression remission. We investigated whether predictions were moderated by treatment use or symptoms. Participants included 55 individuals (n = 39 female) with a depression diagnosis at baseline; 36 of these individuals completed standard cognitive behavioral therapy and 19 were followed during naturalistic course of illness. All participants were assessed for depression diagnosis at a follow-up visit.

RESULTS

Both nPE and nEV classifiers forecasted remission significantly better than null classifiers. The nEV classifier performed significantly better than the nPE classifier. We found no main or interaction effects of treatment status on nPE or nEV accuracy. We found a significant interaction between nPE-forecasted remission status and anhedonia, but not for negative affect or anxious arousal, when controlling for nEV-forecasted remission status.

LIMITATIONS

Our sample size, while comparable to that of other studies, limits options for maximizing and evaluating model performance. We addressed this with two standard methods for optimizing model performance (90:10 train and test scheme and bootstrapped sampling).

CONCLUSIONS

Results support nEV and nPE as relevant biobehavioral signals for understanding depression outcome independent of treatment status, with nEV being stronger than nPE as a predictor of remission. Reinforcement learning variables may be useful components of an individualized medicine framework for depression healthcare.

Reduction of Aversive Learning Rates in Pavlovian Conditioning by Angiotensin II Antagonist Losartan: A Randomized Controlled Trial

BACKGROUND

Angiotensin receptor blockade has been linked to aspects of aversive learning and memory formation and to the prevention of posttraumatic stress disorder symptom development.

METHODS

We investigated the influence of the angiotensin receptor blocker losartan on aversive Pavlovian conditioning using a probabilistic learning paradigm. In a double-blind, randomized, placebo-controlled design, we tested 45 (18 female) healthy volunteers during a baseline session, after application of losartan or placebo (drug session), and during a follow-up session. During each session, participants engaged in a task in which they had to predict the probability of an electrical stimulation on every trial while the true shock contingencies switched repeatedly between phases of high and low shock threat. Computational reinforcement learning models were used to investigate learning dynamics.

RESULTS

Acute administration of losartan significantly reduced participants' adjustment during both low-to-high and high-to-low threat changes. This was driven by reduced aversive learning rates in the losartan group during the drug session compared with baseline. The 50-mg drug dose did not induce reduction of blood pressure or change in reaction times, ruling out a general reduction in attention and engagement. Decreased adjustment of aversive expectations was maintained at a follow-up session 24 hours later.

CONCLUSIONS

This study shows that losartan acutely reduces Pavlovian learning in aversive environments, thereby highlighting a potential role of the renin-angiotensin system in anxiety development.

The challenge of learning adaptive mental behavior

Many psychotherapies aim to help people replace maladaptive mental behaviors (such as those leading to unproductive worry) with more adaptive ones (such as those leading to active problem solving). Yet, little is known empirically about how challenging it is to learn adaptive mental behaviors. Mental behaviors entail taking mental operations and thus may be more challenging to perform than motor actions; this challenge may enhance or impair learning. In particular, challenge when learning is often desirable because it improves retention. Yet, it is also plausible that the necessity of carrying out mental operations interferes with learning the expected values of mental actions by impeding credit assignment: the process of updating an action's value after reinforcement. Then, it may be more challenging not only to perform-but also to learn the consequences of-mental (vs. motor) behaviors. We designed a task to assess learning to take adaptive mental versus motor actions via matched probabilistic feedback. In two experiments (N = 300), most participants found it more difficult to learn to select optimal mental (vs. motor) actions, as evident in worse accuracy not only in a learning but also test (retention) phase. Computational modeling traced this impairment to an indicator of worse credit assignment (impaired construction and maintenance of expected values) when learning mental actions, accounting for worse accuracy in the learning and retention phases. The results suggest that people have particular difficulty learning adaptive mental behavior and pave the way for novel interventions to scaffold credit assignment and promote adaptive thinking. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Structural covariance, topological organization, and volumetric features of amygdala subnuclei in posttraumatic stress disorder

The amygdala is divided into functional subnuclei which have been challenging to investigate due to functional magnetic resonance imaging (MRI) limitations in mapping small neural structures. Hence their role in the neurobiology of posttraumatic stress disorder (PTSD) remains poorly understood. Examination of covariance of structural MRI measures could be an alternate approach to circumvent this issue. T1-weighted anatomical scans from a 3 T scanner from non-trauma-exposed controls (NEC; n = 71, 75 % female) and PTSD participants (n = 67, 69 % female) were parcellated into 105 brain regions. Pearson's r partial correlations were computed for three and nine bilateral amygdala subnuclei and every other brain region, corrected for age, sex, and total brain volume. Pairwise correlation comparisons were performed to examine subnuclei covariance profiles between-groups. Graph theory was employed to investigate subnuclei network topology. Volumetric measures were compared to investigate structural changes. We found differences between amygdala subnuclei in covariance with the hippocampus for both groups, and additionally with temporal brain regions for the PTSD group. Network topology demonstrated the importance of the right basal nucleus in facilitating network communication only in PTSD. There were no between-group differences for any of the three structural metrics. These findings are in line with previous work that has failed to find structural differences for amygdala subnuclei between PTSD and controls. However, differences between amygdala subnuclei covariance profiles observed in our study highlight the need to investigate amygdala subnuclei functional connectivity in PTSD using higher field strength fMRI for better spatial resolution.

Can computational models help elucidate the link between complex trauma and hallucinations?

Recently, a number of predictive coding models have been proposed to account for post-traumatic stress disorder (PTSD)'s symptomatology, including intrusions, flashbacks and hallucinations. These models were usually developed to account for traditional/type-1 PTSD. We here discuss whether these models also apply or can be translated to the case of complex/type-2 PTSD and childhood trauma (cPTSD). The distinction between PTSD and cPTSD is important because the disorders differ in terms of symptomatology and potential mechanisms, how they relate to developmental stages, but also in terms of illness trajectory and treatment. Models of complex trauma could give us insights on hallucinations in physiological/pathological conditions or more generally on the development of intrusive experiences across diagnostic classes.

Latent-state and model-based learning in PTSD

Post-traumatic stress disorder (PTSD) is characterized by altered emotional and behavioral responding following a traumatic event. In this article, we review the concepts of latent-state and model-based learning (i.e., learning and inferring abstract task representations) and discuss their relevance for clinical and neuroscience models of PTSD. Recent data demonstrate evidence for brain and behavioral biases in these learning processes in PTSD. These new data potentially recast excessive fear towards trauma cues as a problem in learning and updating abstract task representations, as opposed to traditional conceptualizations focused on stimulus-specific learning. Biases in latent-state and model-based learning may also be a common mechanism targeted in common therapies for PTSD. We highlight key knowledge gaps that need to be addressed to further elaborate how latent-state learning and its associated neurocircuitry mechanisms function in PTSD and how to optimize treatments to target these processes.

Out of control: computational dynamic control dysfunction in stress- and anxiety-related disorders

Control theory, which has played a central role in technological progress over the last 150 years, has also yielded critical insights into biology and neuroscience. Recently, there has been a surging interest in integrating control theory with computational psychiatry. Here, we review the state of the field of using control theory approaches in computational psychiatry and show that recent research has mapped a neural control circuit consisting of frontal cortex, parietal cortex, and the cerebellum. This basic feedback control circuit is modulated by estimates of reward and cost via the basal ganglia as well as by arousal states coordinated by the insula, dorsal anterior cingulate cortex, amygdala, and locus coeruleus. One major approach within the broader field of control theory, known as proportion-integral-derivative (PID) control, has shown promise as a model of human behavior which enables precise and reliable estimates of underlying control parameters at the individual level. These control parameters correlate with self-reported fear and with both structural and functional variation in affect-related brain regions. This suggests that dysfunctional engagement of stress and arousal systems may suboptimally modulate parameters of domain-general goal-directed control algorithms, impairing performance in complex tasks involving movement, cognition, and affect. Future directions include clarifying the causal role of control deficits in stress- and anxiety-related disorders and developing clinically useful tools based on insights from control theory.

Decision-making for concurrent reward and threat is differentially modulated by trauma exposure and PTSD symptom severity

Trauma exposure, particularly interpersonal violence (IPV) traumas, are significant risk factors for development of mental health disorders, particularly posttraumatic stress disorder (PTSD). Studies attempting to disentangle mechanisms by which trauma confers risk and maintenance of PTSD have often investigated threat or reward learning in isolation. However, real-world decision-making often involves navigating concurrent and conflicting probabilities for threat and reward. We sought to understand how threat and reward learning interact to impact decision-making, and how these processes are modulated by trauma exposure and PTSD symptom severity. 429 adult participants with a range of trauma exposure and symptom severities completed an online version of the two stage Markov task, where participants make a series of decisions towards the goal of obtaining a reward, that embedded an intermediate threat or neutral image along the sequence of decisions to be made. This task design afforded the possibility to differentiate between threat avoidance vs diminished reward learning in the presence of threat, and whether these two processes reflect model-based vs model-free decision-making. Results demonstrated that trauma exposure severity, particularly IPV exposure, was associated with impairment in model-based learning for reward independent of threat, as well as with model-based threat avoidance. PTSD symptom severity was associated with diminished model-based learning for reward in the presence of threat, consistent with a threat-induced impairment in cognitively-demanding strategies for reward learning, but no evidence of heightened threat avoidance. These results highlight the complex interactions between threat and reward learning as a function of trauma exposure and PTSD symptom severity. Findings have potential implications for treatment augmentation and suggest a need for continued research.

Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy

In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.

Neural valuation of rewards and punishments in posttraumatic stress disorder: a computational approach

Posttraumatic stress disorder (PTSD) is associated with changes in fear learning and decision-making, suggesting involvement of the brain's valuation system. Here we investigate the neural mechanisms of subjective valuation of rewards and punishments in combat veterans. In a functional MRI study, male combat veterans with a wide range of posttrauma symptoms (N = 48, Clinician Administered PTSD Scale, CAPS-IV) made a series of choices between sure and uncertain monetary gains and losses. Activity in the ventromedial prefrontal cortex (vmPFC) during valuation of uncertain options was associated with PTSD symptoms, an effect which was consistent for gains and losses, and specifically driven by numbing symptoms. In an exploratory analysis, computational modeling of choice behavior was used to estimate the subjective value of each option. The neural encoding of subjective value varied as a function of symptoms. Most notably, veterans with PTSD exhibited enhanced representations of the saliency of gains and losses in the neural valuation system, especially in ventral striatum. These results suggest a link between the valuation system and the development and maintenance of PTSD, and demonstrate the significance of studying reward and punishment processing within subject.

Altered predictive control during memory suppression in PTSD

Aberrant predictions of future threat lead to maladaptive avoidance in individuals with post-traumatic stress disorder (PTSD). How this disruption in prediction influences the control of memory states orchestrated by the dorsolateral prefrontal cortex is unknown. We combined computational modeling and brain connectivity analyses to reveal how individuals exposed and nonexposed to the 2015 Paris terrorist attacks formed and controlled beliefs about future intrusive re-experiencing implemented in the laboratory during a memory suppression task. Exposed individuals with PTSD used beliefs excessively to control hippocampal activity during the task. When this predictive control failed, the prediction-error associated with unwanted intrusions was poorly downregulated by reactive mechanisms. This imbalance was linked to higher severity of avoidance symptoms, but not to general disturbances such as anxiety or negative affect. Conversely, trauma-exposed participants without PTSD and nonexposed individuals were able to optimally balance predictive and reactive control during the memory suppression task. These findings highlight a potential pathological mechanism occurring in individuals with PTSD rooted in the relationship between the brain’s predictive and control mechanisms.

It remains unclear how predictions of future threat affect memory recall, specifically in the case of post-traumatic stress disorder (PTSD). Here, the authors combined computational modeling and brain connectivity analyses to show that individuals with PTSD have exaggerated predictive control and reduced reactive control in a memory suppression task.

Trauma Disrupts Reinforcement Learning in Rats-A Novel Animal Model of Chronic Stress Exposure

Trauma, as well as chronic stress that characterizes a modern fast-paced lifestyle, contributes to numerous psychopathologies and psychological problems. Psychiatric patients with traumas, as well as healthy individuals who experienced traumas in the past, are often characterized by diminished cognitive abilities. In our protocol, we used an animal model to explore the influence of chronic trauma on cognitive abilities and behavior in the group of 20 rats (Rattus norvegicus). The experimental group was introduced to chronic (12 consecutive days) exposure to predator odor (bobcat urine). We measured the reinforcement learning of each individual before and after the exposition via the Probabilistic Selection Task (PST) and we used Social Interaction Test (SIT) to assess the behavioral changes of each individual before and after the trauma. In the experimental group, there was a significant decrease in reinforcement learning after exposure to a single trauma (Wilcoxon Test, p = 0.034) as well as after 11 days of chronic trauma (Wilcoxon-test, p = 0.01) in comparison to pre-trauma performance. The control group, which was not exposed to predator odor but underwent the same testing protocol, did not present significant deterioration in reinforcement learning. In cross-group comparisons, there was no difference between the experimental and control group in PST before odor protocol (U Mann-Whitney two-sided, p = 0.909). After exposure to chronic trauma, the experimental group deteriorated in PST performance compared to control (U Mann-Whitney Two-sided, p = 0.0005). In SIT, the experimental group spent less time in an Interaction Zone with an unfamiliar rat after trauma protocol (Wilcoxon two-sided test, p = 0.019). Major strengths of our models are: (1) protocol allows investigating reinforcement learning before and after exposition to chronic trauma, with the same group of rats, (2) translational scope, as the PST is displayed on touchscreen, similarly to human studies, (3) protocol delivers chronic trauma that impairs reward learning, but behaviorally does not induce full-blown anhedonia, thus rats performed voluntarily throughout all the procedures.

Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis

IMPORTANCE

Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent.

OBJECTIVE

To assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals.

DATA SOURCES

Web of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood).

STUDY SELECTION

Studies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models.

DATA EXTRACTION AND SYNTHESIS

Articles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected.

MAIN OUTCOMES AND MEASURES

The primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate).

RESULTS

A total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004).

CONCLUSIONS AND RELEVANCE

The simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders.

Value estimation and latent-state update-related neural activity during fear conditioning predict posttraumatic stress disorder symptom severity

Learning theories of posttraumatic stress disorder (PTSD) purport that fear-learning processes, such as those that support fear acquisition and extinction, are impaired. Computational models designed to capture specific processes involved in fear learning have primarily assessed model-free, or trial-and-error, reinforcement learning (RL). Although previous studies indicated that aspects of model-free RL are disrupted among individuals with PTSD, research has yet to identify whether model-based RL, which is inferential and contextually driven, is impaired. Given empirical evidence of aberrant contextual modulation of fear in PTSD, the present study sought to identify whether model-based RL processes are altered during fear conditioning among women with interpersonal violence (IPV)-related PTSD (n = 85) using computational modeling. Model-free, hybrid, and model-based RL models were applied to skin conductance responses (SCR) collected during fear acquisition and extinction, and the model-based RL model was found to provide the best fit to the SCR data. Parameters from the model-based RL model were carried forward to neuroimaging analyses (voxel-wise and independent component analysis). Results revealed that reduced activity within visual processing regions during model-based updating uniquely predicted higher PTSD symptoms. Additionally, after controlling for model-based updating, greater value estimation encoding within the left frontoparietal network during fear acquisition and reduced value estimation encoding within the dorsomedial prefrontal cortex during fear extinction predicted greater PTSD symptoms. Results provide evidence of disrupted RL processes in women with assault-related PTSD, which may contribute to impaired fear and safety learning, and, furthermore, may relate to treatment response (e.g., poorer response to exposure therapy).

Computational Psychiatry Needs Time and Context

Why has computational psychiatry yet to influence routine clinical practice? One reason may be that it has neglected context and temporal dynamics in the models of certain mental health problems. We develop three heuristics for estimating whether time and context are important to a mental health problem: Is it characterized by a core neurobiological mechanism? Does it follow a straightforward natural trajectory? And is intentional mental content peripheral to the problem? For many problems the answers are no, suggesting that modeling time and context is critical. We review computational psychiatry advances toward this end, including modeling state variation, using domain-specific stimuli, and interpreting differences in context. We discuss complementary network and complex systems approaches. Novel methods and unification with adjacent fields may inspire a new generation of computational psychiatry.

Advances in modeling learning and decision-making in neuroscience

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

Reinforcement Learning Disruptions in Individuals With Depression and Sensitivity to Symptom Change Following Cognitive Behavioral Therapy

IMPORTANCE

Major depressive disorder is prevalent and impairing. Parsing neurocomputational substrates of reinforcement learning in individuals with depression may facilitate a mechanistic understanding of the disorder and suggest new cognitive therapeutic targets.

OBJECTIVE

To determine associations among computational model-derived reinforcement learning parameters, depression symptoms, and symptom changes after treatment.

DESIGN, SETTING, AND PARTICIPANTS

In this mixed cross-sectional-cohort study, individuals performed reward and loss variants of a probabilistic learning task during functional magnetic resonance imaging at baseline and follow-up. A volunteer sample with and without a depression diagnosis was recruited from the community. Participants were assessed from July 2011 to February 2017, and data were analyzed from May 2017 to May 2021.

MAIN OUTCOMES AND MEASURES

Computational model-based analyses of participants' choices assessed a priori hypotheses about associations between components of reward-based and loss-based learning with depression symptoms. Changes in both learning parameters and symptoms were then assessed in a subset of participants who received cognitive behavioral therapy (CBT).

RESULTS

Of 101 included adults, 69 (68.3%) were female, and the mean (SD) age was 34.4 (11.2) years. A total of 69 participants with a depression diagnosis and 32 participants without a depression diagnosis were included at baseline; 48 participants (28 with depression who received CBT and 20 without depression) were included at follow-up (mean [SD] of 115.1 [15.6] days). Computational model-based analyses of behavioral choices and neural data identified associations of learning with symptoms during reward learning and loss learning, respectively. During reward learning only, anhedonia (and not negative affect or arousal) was associated with model-derived learning parameters (learning rate: posterior mean regression β = -0.14; 95% credible interval [CrI], -0.12 to -0.03; outcome sensitivity: posterior mean regression β = 0.18; 95% CrI, 0.02 to 0.37) and neural learning signals (moderation of association between striatal prediction error and expected value signals: t97 = -2.10; P = .04). During loss learning only, negative affect (and not anhedonia or arousal) was associated with learning parameters (outcome shift: posterior mean regression β = -0.11; 95% CrI, -0.20 to -0.01) and disrupted neural encoding of learning signals (association with subgenual anterior cingulate prediction error signals: r = -0.28; P = .005). Symptom improvement following CBT was associated with normalization of learning parameters that were disrupted at baseline (reward learning rate: posterior mean regression β = 0.15; 90% CrI, 0.001 to 0.41; loss outcome shift: posterior mean regression β = 0.42; 90% CrI, 0.09 to 0.77).

CONCLUSIONS AND RELEVANCE

In this study, the mapping of reinforcement learning components to symptoms of major depression revealed mechanistic features associated with these symptoms and points to possible learning-based therapeutic processes and targets.

The interpretation of behavior-model correlations in unidentified cognitive models

The rise of computational modeling in the past decade has led to a substantial increase in the number of papers that report parameter estimates of computational cognitive models. A common application of computational cognitive models is to quantify individual differences in behavior by estimating how these are expressed in differences in parameters. For these inferences to hold, models need to be identified, meaning that one set of parameters is most likely, given the behavior under consideration. For many models, model identification can be achieved up to a scaling constraint, which means that under the assumption that one parameter has a specific value, all remaining parameters are identified. In the current note, we argue that this scaling constraint implies a strong assumption about the cognitive process that the model is intended to explain, and warn against an overinterpretation of the associative relations found in this way. We will illustrate these points using signal detection theory, reinforcement learning models, and the linear ballistic accumulator model, and provide suggestions for a clearer interpretation of modeling results.

Computational approaches and machine learning for individual-level treatment predictions

RATIONALE

The impact of neuroscience-based approaches for psychiatry on pragmatic clinical decision-making has been limited. Although neuroscience has provided insights into basic mechanisms of neural function, these insights have not improved the ability to generate better assessments, prognoses, diagnoses, or treatment of psychiatric conditions.

OBJECTIVES

To integrate the emerging findings in machine learning and computational psychiatry to address the question: what measures that are not derived from the patient's self-assessment or the assessment by a trained professional can be used to make more precise predictions about the individual's current state, the individual's future disease trajectory, or the probability to respond to a particular intervention?

RESULTS

Currently, the ability to use individual differences to predict differential outcomes is very modest possibly related to the fact that the effect sizes of interventions are small. There is emerging evidence of genetic and neuroimaging-based heterogeneity of psychiatric disorders, which contributes to imprecise predictions. Although the use of machine learning tools to generate clinically actionable predictions is still in its infancy, these approaches may identify subgroups enabling more precise predictions. In addition, computational psychiatry might provide explanatory disease models based on faulty updating of internal values or beliefs.

CONCLUSIONS

There is a need for larger studies, clinical trials using machine learning, or computational psychiatry model parameters predictions as actionable outcomes, comparing alternative explanatory computational models, and using translational approaches that apply similar paradigms and models in humans and animals.

Latent cause inference during extinction learning in trauma-exposed individuals with and without PTSD

BACKGROUND

Problems in learning that sights, sounds, or situations that were once associated with danger have become safe (extinction learning) may explain why some individuals suffer prolonged psychological distress following traumatic experiences. Although simple learning models have been unable to provide a convincing account of why this learning fails, it has recently been proposed that this may be explained by individual differences in beliefs about the causal structure of the environment.

METHODS

Here, we tested two competing hypotheses as to how differences in causal inference might be related to trauma-related psychopathology, using extinction learning data collected from clinically well-characterised individuals with varying degrees of post-traumatic stress (N = 56). Model parameters describing individual differences in causal inference were related to multiple post-traumatic stress disorder (PTSD) and depression symptom dimensions via network analysis.

RESULTS

Individuals with more severe PTSD were more likely to assign observations from conditioning and extinction stages to a single underlying cause. Specifically, greater re-experiencing symptom severity was associated with a lower likelihood of inferring that multiple causes were active in the environment.

CONCLUSIONS

We interpret these results as providing evidence of a primary deficit in discriminative learning in participants with more severe PTSD. Specifically, a tendency to attribute a greater diversity of stimulus configurations to the same underlying cause resulted in greater uncertainty about stimulus-outcome associations, impeding learning both that certain stimuli were safe, and that certain stimuli were no longer dangerous. In the future, better understanding of the role of causal inference in trauma-related psychopathology may help refine cognitive therapies for these disorders.

Symptoms of Posttraumatic Stress Disorder are Associated with Exaggerated Neural Response to Surprising Errors

Posttraumatic stress disorder (PTSD) is characterized by exaggerated salience of previously innocuous cues and associated with hyperactivity of salience-related brain regions. Recently, computational models have been deployed to operationalize salience more precisely regarding surprise-driven learning, leading to findings that such learning is altered in anxiety-related disorders. In the present study, a sample of 20 combat veterans completed a probabilistic learning task during fMRI scanning. We applied a computational model to generate a trial-by-trial surprise signal (i.e., unsigned prediction error or difference between the expected probability of an outcome and the actual observed outcome), which allowed us to examine the neural response to surprising events. We did not observe an association between PTSD symptoms and behavioral indices of learning in the task. Surprising errors were associated with increased activity in the left precuneus/inferior parietal lobule and right inferior parietal lobule, two parietal regions that are linked to salience processing. Additionally, PTSD symptom severity was positively associated with precuneus/inferior parietal lobule activation to surprising errors, r = .63, p = .004. Taken together, this pattern of results suggests that PTSD symptoms are specifically associated with an exaggerated response to surprising errors in salience-related regions of the brain. This altered pattern of neural activity could represent a target for intervention to improve PTSD symptoms.

Neural Computations of Threat

A host of learning, memory, and decision-making processes form the individual's response to threat and may be disrupted in anxiety and post-trauma psychopathology. Here we review the neural computations of threat, from the first encounter with a dangerous situation, through learning, storing, and updating cues that predict it, to making decisions about the optimal course of action. The overview highlights the interconnected nature of these processes and their reliance on shared neural and computational mechanisms. We propose an integrative approach to the study of threat-related processes, in which specific computations are studied across the various stages of threat experience rather than in isolation. This approach can generate new insights about the evolution, diagnosis, and treatment of threat-related psychopathology.

Posttraumatic Stress Disorder and the Developing Adolescent Brain

Posttraumatic stress disorder (PTSD) in adolescents is common and debilitating. In contrast to adult PTSD, relatively little is known about the neurobiology of adolescent PTSD, nor about how current treatments may alter adolescent neurodevelopment to allow recovery from PTSD. Improving our understanding of biological mechanisms of adolescent PTSD, taken in the context of neurodevelopment, is crucial for developing novel and personalized treatment approaches. In this review, we highlight prevailing constructs of PTSD and current findings on these domains in adolescent PTSD. Notably, little data exist in adolescent PTSD for prominent adult PTSD constructs, including threat learning and attentional threat bias. Most work to date has examined general threat processing, emotion regulation, and their neural substrates. These studies suggest that adolescent PTSD, while phenomenologically similar to adult PTSD, shows unique neurodevelopmental substrates that may impair recovery but could also be targeted in the context of adolescent neuroplasticity to improve outcomes. Both cross-sectional and longitudinal data suggest abnormal frontolimbic development compared with typically developing youths, a pattern that may differ from resilient youths. Whether current treatments such as trauma-focused psychotherapy engage these targets and restore healthy neurodevelopment remains an open question. We end our review by highlighting emerging areas and knowledge gaps that could be addressed to better characterize the biology underlying adolescent PTSD. Emerging studies in computational modeling of decision making, caregiver-related transmission of traumatic stress, and other areas may offer new targets that could harness adolescent neurobehavioral plasticity to improve resilience and recovery for some of our most vulnerable youths.

Where perception meets belief updating: Computational evidence for slower updating of visual expectations in anxious individuals

BACKGROUND

Surprising events are important sources of internal model updating which adjusts expectations for both decision-making and perceptual processing circuits. Anxious individuals display relatively intact updating of internal models used to make decisions, however how these individuals update their perceptual expectations remains poorly understood. Based on previous work, we hypothesized that anxious individuals experienced exaggerated surprise to predictable events, which imbues them with undue salience.

METHODS

To model the rate of updating of internal models for both decision-making and perceptual processing, we applied a hybrid Rescorla Wagner (RW)/Drift Diffusion Model (DDM) to a change point detection task in a transdiagnostic group of individuals with mood and anxiety disorders and examined the relationship between learning rates and anxiety and negative affect.

RESULTS

Model comparison provided evidence that decision-making and perceptual processing rely on separate internal models with different learning rates. Anxiety and older age were associated with slower updating of models used in perceptual processing, but not those used in decision-making.

LIMITATIONS

This was a cross-sectional study and lacked neural data to examine the role of specific brain circuits in updating of perceptual predictions.

CONCLUSIONS

Anxious individuals display slower updating of internal models used in perceptual processing, but not those used in decision-making. This deficit could contribute to exaggerated salience of harmless stimuli in anxiety. The results have implications for the assessment and treatment of basic processing dysfunctions in anxiety.

In Cocaine Dependence, Neural Prediction Errors During Loss Avoidance Are Increased With Cocaine Deprivation and Predict Drug Use

BACKGROUND

In substance-dependent individuals, drug deprivation and drug use trigger divergent behavioral responses to environmental cues. These divergent responses are consonant with data showing that short- and long-term adaptations in dopamine signaling are similarly sensitive to state of drug use. The literature suggests a drug state-dependent role of learning in maintaining substance use; evidence linking dopamine to both reinforcement learning and addiction provides a framework to test this possibility.

METHODS

In a randomized crossover design, 22 participants with current cocaine use disorder completed a probabilistic loss-learning task during functional magnetic resonance imaging while on and off cocaine (44 sessions). Another 54 participants without Axis I psychopathology served as a secondary reference group. Within-drug state and paired-subjects' learning effects were assessed with computational model-derived individual learning parameters. Model-based neuroimaging analyses evaluated effects of drug use state on neural learning signals. Relationships among model-derived behavioral learning rates (α+, α-), neural prediction error signals (δ+, δ-), cocaine use, and desire to use were assessed.

RESULTS

During cocaine deprivation, cocaine-dependent individuals exhibited heightened positive learning rates (α+), heightened neural positive prediction error (δ+) responses, and heightened association of α+ with neural δ+ responses. The deprivation-enhanced neural learning signals were specific to successful loss avoidance, comparable to participants without psychiatric conditions, and mediated a relationship between chronicity of drug use and desire to use cocaine.

CONCLUSIONS

Neurocomputational learning signals are sensitive to drug use status and suggest that heightened reinforcement by successful avoidance of negative outcomes may contribute to drug seeking during deprivation. More generally, attention to drug use state is important for delineating substrates of addiction.

Neural computations of threat in the aftermath of combat trauma

By combining computational, morphological, and functional analyses, this study relates latent markers of associative threat learning to overt post-traumatic stress disorder (PTSD) symptoms in combat veterans. Using reversal learning, we found that symptomatic veterans showed greater physiological adjustment to cues that did not predict what they had expected, indicating greater sensitivity to prediction errors for negative outcomes. This exaggerated weighting of prediction errors shapes the dynamic learning rate (associability) and value of threat predictive cues. The degree to which the striatum tracked the associability partially mediated the positive correlation between prediction-error weights and PTSD symptoms, suggesting that both increased prediction-error weights and decreased striatal tracking of associability independently contribute to PTSD symptoms. Furthermore, decreased neural tracking of value in the amygdala, in addition to smaller amygdala volume, independently corresponded to higher PTSD symptom severity. These results provide evidence for distinct neurocomputational contributions to PTSD symptoms.

The Associations of Emotion Coping Appraisal With Both the Cue-Outcome Contingency and Perceived Verbal Abuse Exposure

Previous studies have reported an association between verbal abuse in early childhood and structural and functional alterations in the young adult brain, supporting the existence of critical periods in human brain development. In addition, exposure to verbal abuse in early childhood is strongly correlated with lifetime psychiatric illness. Resilience is defined as the ability to avoid the negative psychological, biological, and social consequences of stress that impair psychological and physical homeostasis and is used to cope with these psychiatric diseases. We attempted to explain the mediatable present function of resilience and its associations with several psychiatric disorders, with verbal abuse exposure in early childhood and with the present value of the readily measurable and conceptually connected generative Bayesian model parameter. Thirty-six subjects performed a cross-modal associative learning task requiring them to learn the predictive strength of auditory cues and predict a subsequent visual stimulus. The probability of the association changed across each trial block. Subjects' responses were modeled as a hierarchical Bayesian belief-updating process using a hierarchical Gaussian filter (HGF) with three levels, a Sutton K1 model, and a Rescorla-Wagner model. Subjects completed the Korean version of the Verbal Abuse Questionnaire (VAQ) for segmented periods (aged 0 to 6, 7 to 12, and 13 to 18 years), and their positive self-appraisal was estimated using the Resilience Appraisal Scale (RAS). Random-effects Bayesian model selection identified HGF as the best model. Of the VAQ values for specific periods, only preschool VAQ scores were negatively correlated with RAS scores. The tonic volatility parameter, ω₂, of HGF showed a negative relationship with RAS emotion coping scores. The linear regression model explained 18.3% of the variance of emotion coping appraisal with ω₂ and preschool VAQ scores. Based on the results obtained from young adults, decrease in emotion coping appraisal can be explained by an increase in the number of experiences of verbal abuse in early childhood and the increased tendency to update beliefs about the cue-outcome associative probability in a volatile environment.

Shared gray matter reductions across alcohol use disorder and posttraumatic stress disorder in the anterior cingulate cortex: A dual meta-analysis

The considerable comorbidity of posttraumatic stress disorder (PTSD) and alcohol use disorders (AUD) poses a greater public health burden than either condition alone. Although there is a substantial body of evidence linking the direct neurotoxic effect of heavy drinking to gray matter (GM) deficits, as well as a growing body of literature supporting a strong association between PTSD and GM alterations, there is scant research interrogating the direct interaction of the two disorders. In order to generate data-driven, specific hypotheses regarding the overlapping neural substrates of PTSD and AUD, we conducted a meta-analysis of GM volumes in each disorder relative to healthy control subjects. We found shared GM deficits in the anterior cingulate cortex (ACC) across both disorders relative to healthy control participants. These findings suggest that reduced volumes of the ACC across PTSD and AUD may have implications for the development, expression, or treatment of symptoms linked to these frequently co-existing disorders. Recommendations are made for future work aimed at delineating the specific and shared effects of traumatic stress and alcoholism on neural integrity.

Differential Roles of the Salience Network During Prediction Error Encoding and Facial Emotion Processing Among Female Adolescent Assault Victims

BACKGROUND

Early-life assaultive violence exposure is a potent risk factor for posttraumatic stress disorder (PTSD) and other mood and anxiety disorders. Neurocircuitry models posit that increased risk is mediated by heightened emotion processing in a salience network including the dorsal anterior cingulate cortex, anterior insula, and amygdala. However, the processes of reinforcement learning (RL) also engage the salience network and are implicated in responses to early-life trauma and PTSD. To define their relative roles in response to early-life trauma and PTSD symptoms, the current study compared engagement of the salience network during emotion processing and RL as a function of early-life assault exposure.

METHODS

Adolescent girls (n = 30 girls who had previously been physically or sexually assaulted; n = 30 healthy girls for comparison) 11 to 17 years of age completed two types of tasks during functional magnetic resonance imaging: a facial emotion processing task and an RL task using either social or nonsocial stimuli. Independent component analysis was used to identify a salience network and characterize its engagement in response to emotion processing and prediction error encoding during the RL tasks.

RESULTS

Assault was related to greater reactivity of the salience network during emotion processing. By contrast, we found lesser encoding of negative prediction errors in the salience network, particularly during the social RL task, in girls who had been assaulted. The dysfunction of salience network activity during emotion processing and prediction error encoding was not associated with PTSD symptoms.

CONCLUSIONS

These results suggest that hyper- versus hypoactivity of the salience network among trauma-exposed youths depends on the cognitive-affective domain.