Associations between aversive learning processes and transdiagnostic psychiatric symptoms in a general population sample

Information search under uncertainty across transdiagnostic psychopathology and healthy ageing

When making decisions in everyday life, we often rely on an internally generated sense of confidence to help us revise and direct future behaviours. For instance, confidence directly informs whether further information should be sought prior to commitment to a final decision. Many studies have shown that aging and both clinical and sub-clinical symptoms of psychopathology are associated with systematic alterations in confidence. However, it remains unknown whether these confidence distortions influence information-seeking behaviour. We investigated this question in a large general population sample (N = 908). Participants completed a battery of psychiatric symptom questionnaires and performed a perceptual decision-making task with confidence ratings in which they were offered the option to seek helpful information (at a cost) before committing to a final decision. Replicating previous findings, an 'anxious-depression' (AD) symptom dimension was associated with systematically low confidence, despite no detriment in objective task accuracy. Conversely, a 'compulsive behaviour and intrusive thoughts' (CIT) dimension was associated with impaired task accuracy but paradoxical over-confidence. However, neither symptom dimension was significantly associated with an increased or decreased tendency to seek information. Hence, participants scoring highly for AD or CIT did not use the option to information seek any more than average to either increase their confidence (AD) or improve the accuracy of their decisions (CIT). In contrast, older age was associated with impaired accuracy and decreased confidence initially, but increased information seeking behaviour mediated increases in both accuracy and confidence for final decisions. Hence, older adults used the information seeking option to overcome initial deficits in objective performance and to increase their confidence accordingly. The results show an appropriate use of information seeking to overcome perceptual deficits and low confidence in healthy aging which was not present in transdiagnostic psychopathology.

The Role of the Medial Prefrontal Cortex in Spatial Margin of Safety Calculations

Naturalistic observations show that animals pre-empt danger by moving to locations that increase their success in avoiding future threats. To test this in humans, we created a spatial margin of safety (MOS) decision task that quantifies pre-emptive avoidance by measuring the distance subjects place themselves to safety when facing different threats whose attack locations vary in predictability. Behavioral results show that human participants place themselves closer to safe locations when facing threats that attack in spatial locations with more outliers. Using both univariate and multivariate pattern analysis (MVPA) on fMRI data collected during a 2 h session on participants of both sexes, we demonstrate a dissociable role for the vmPFC in MOS-related decision-making. MVPA results revealed that the posterior vmPFC encoded for more unpredictable threats with univariate analyses showing a functional coupling with the amygdala and hippocampus. Conversely, the anterior vmPFC was more active for the more predictable attacks and showed coupling with the striatum. Our findings converge in showing that during pre-emptive danger, the anterior vmPFC may provide a safety signal, possibly via foreseeable outcomes, while the posterior vmPFC drives unpredictable danger signals.

Reliable, rapid, and remote measurement of metacognitive bias

Metacognitive biases have been repeatedly associated with transdiagnostic psychiatric dimensions of 'anxious-depression' and 'compulsivity and intrusive thought', cross-sectionally. To progress our understanding of the underlying neurocognitive mechanisms, new methods are required to measure metacognition remotely, within individuals over time. We developed a gamified smartphone task designed to measure visuo-perceptual metacognitive (confidence) bias and investigated its psychometric properties across two studies (N = 3410 unpaid citizen scientists, N = 52 paid participants). We assessed convergent validity, split-half and test-retest reliability, and identified the minimum number of trials required to capture its clinical correlates. Convergent validity of metacognitive bias was moderate (r(50) = 0.64, p < 0.001) and it demonstrated excellent split-half reliability (r(50) = 0.91, p < 0.001). Anxious-depression was associated with decreased confidence (β =  - 0.23, SE = 0.02, p < 0.001), while compulsivity and intrusive thought was associated with greater confidence (β = 0.07, SE = 0.02, p < 0.001). The associations between metacognitive biases and transdiagnostic psychiatry dimensions are evident in as few as 40 trials. Metacognitive biases in decision-making are stable within and across sessions, exhibiting very high test-retest reliability for the 100-trial (ICC = 0.86, N = 110) and 40-trial (ICC = 0.86, N = 120) versions of Meta Mind. Hybrid 'self-report cognition' tasks may be one way to bridge the recently discussed reliability gap in computational psychiatry.

Confidence biases in problem gambling

Background and aims

Subjective confidence plays an important role in guiding behaviour, especially when objective feedback is unavailable. Systematic misjudgements in confidence can foster maladaptive behaviours and have been linked to various psychiatric disorders. In this study, we adopted a transdiagnostic approach to examine confidence biases in problem gamblers across three levels: local decision confidence, global task performance confidence, and overall self-esteem. The importance of taking a transdiagnostic perspective is increasingly recognised, as it captures the dimensional nature of psychiatric symptoms that often cut across diagnostic boundaries. Accordingly, we investigated if any observed confidence biases could be explained by transdiagnostic symptom dimensions of Anxiety-Depression and Compulsive Behaviour and Intrusive Thought. This approach allows us to gain a more comprehensive understanding of the role of metacognitive processes in problem gambling, beyond the constraints of traditional diagnostic categories.

Methods

Thirty-eight problem gamblers and 38 demographically matched control participants engaged in a gamified metacognition task and completed self-report questionnaires assessing transdiagnostic symptom dimensions.

Results

Compared to controls, problem gamblers displayed significantly elevated confidence at the local decision and global task levels, independent of their actual task performance. This elevated confidence was observed even after controlling for the heightened symptom levels of Anxiety-Depression and Compulsive Behaviour and Intrusive Thought among the problem gamblers.

Discussion

The results reveal a notable disparity in confidence levels between problem gamblers and control participants, not fully accounted for by the symptom dimensions Anxiety-Depression and Compulsive Behaviour and Intrusive Thought. This suggests the contribution of other factors, perhaps linked to gambling-specific cognitive distortions, to the observed confidence biases.

Conclusion

The findings highlight the intricate link between metacognitive confidence and psychiatric symptoms in the context of problem gambling. It underscores the need for further research into metacognitive biases, which could enhance therapeutic approaches for individuals with psychiatric conditions.

Aversive conditioning is impaired in impulsive individuals: A study on learning asymmetries

BACKGROUND AND OBJECTIVES

Appetitive and aversive conditioning are thought to be involved in the development and maintenance of mental disorders including anxiety, mood, eating, and substance use disorders. However, few studies measure the relative strength of appetitive and aversive associations, and their relevance to the risk of mental disorders. This study aims to address this gap.

METHODS

We tested how readily healthy volunteers acquire appetitive vs. aversive associations. 150 participants associated complex 3D objects with either gain or loss and made decisions to gain or avoid losing points. We investigated the relationship of a learning asymmetry with neuroticism, impulsivity, and anhedonia, to test the hypothesis that a stronger learning asymmetry corresponds to more extreme scores on these traits.

RESULTS

Impulsivity was positively associated with the learning asymmetry (R2 = .10). This resulted from an inverse relation with the strength of aversive associations, indicating that impulsive individuals are worse at aversive learning. However, appetitive associations did not differ significantly. No correlations with neuroticism or anhedonia were found.

LIMITATIONS

Conditioning studies typically use primary reinforcers and a CS-. Lacking these may make these results less comparable to other studies.

CONCLUSIONS

We demonstrate that the learning asymmetry can measure individual differences linked to personality traits, and that impulsivity, normally linked with appetitive learning, also influences aversive learning. These results enable additional studies of learning asymmetry in relation to mental disorders, which could include measurements of mental health symptoms to provide further insight into how appetitive and aversive learning interacts with mental disorders.

Dynamics Learning Rate Bias in Pigeons: Insights from Reinforcement Learning and Neural Correlates

Research in reinforcement learning indicates that animals respond differently to positive and negative reward prediction errors, which can be calculated by assuming learning rate bias. Many studies have shown that humans and other animals have learning rate bias during learning, but it is unclear whether and how the bias changes throughout the entire learning process. Here, we recorded the behavior data and the local field potentials (LFPs) in the striatum of five pigeons performing a probabilistic learning task. Reinforcement learning models with and without learning rate biases were used to dynamically fit the pigeons' choice behavior and estimate the option values. Furthemore, the correlation between the striatal LFPs power and the model-estimated option values was explored. We found that the pigeons' learning rate bias shifted from negative to positive during the learning process, and the striatal Gamma (31 to 80 Hz) power correlated with the option values modulated by dynamic learning rate bias. In conclusion, our results support the hypothesis that pigeons employ a dynamic learning strategy in the learning process from both behavioral and neural aspects, providing valuable insights into reinforcement learning mechanisms of non-human animals.

Naturalistic reinforcement learning

Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.

A core component of psychological therapy causes adaptive changes in computational learning mechanisms

BACKGROUND

Cognitive distancing is an emotion regulation strategy commonly used in psychological treatment of various mental health disorders, but its therapeutic mechanisms are unknown.

METHODS

935 participants completed an online reinforcement learning task involving choices between pairs of symbols with differing reward contingencies. Half (49.1%) of the sample was randomised to a cognitive self-distancing intervention and were trained to regulate or 'take a step back' from their emotional response to feedback throughout. Established computational (Q-learning) models were then fit to individuals' choices to derive reinforcement learning parameters capturing clarity of choice values (inverse temperature) and their sensitivity to positive and negative feedback (learning rates).

RESULTS

Cognitive distancing improved task performance, including when participants were later tested on novel combinations of symbols without feedback. Group differences in computational model-derived parameters revealed that cognitive distancing resulted in clearer representations of option values (estimated 0.17 higher inverse temperatures). Simultaneously, distancing caused increased sensitivity to negative feedback (estimated 19% higher loss learning rates). Exploratory analyses suggested this resulted from an evolving shift in strategy by distanced participants: initially, choices were more determined by expected value differences between symbols, but as the task progressed, they became more sensitive to negative feedback, with evidence for a difference strongest by the end of training.

CONCLUSIONS

Adaptive effects on the computations that underlie learning from reward and loss may explain the therapeutic benefits of cognitive distancing. Over time and with practice, cognitive distancing may improve symptoms of mental health disorders by promoting more effective engagement with negative information.

Approach-avoidance reinforcement learning as a translational and computational model of anxiety-related avoidance

Although avoidance is a prevalent feature of anxiety-related psychopathology, differences in the measurement of avoidance between humans and non-human animals hinder our progress in its theoretical understanding and treatment. To address this, we developed a novel translational measure of anxiety-related avoidance in the form of an approach-avoidance reinforcement learning task, by adapting a paradigm from the non-human animal literature to study the same cognitive processes in human participants. We used computational modelling to probe the putative cognitive mechanisms underlying approach-avoidance behaviour in this task and investigated how they relate to subjective task-induced anxiety. In a large online study (n = 372), participants who experienced greater task-induced anxiety avoided choices associated with punishment, even when this resulted in lower overall reward. Computational modelling revealed that this effect was explained by greater individual sensitivities to punishment relative to rewards. We replicated these findings in an independent sample (n = 627) and we also found fair-to-excellent reliability of measures of task performance in a sub-sample retested 1 week later (n = 57). Our findings demonstrate the potential of approach-avoidance reinforcement learning tasks as translational and computational models of anxiety-related avoidance. Future studies should assess the predictive validity of this approach in clinical samples and experimental manipulations of anxiety.

Interactive cognitive maps support flexible behavior under threat

In social environments, survival can depend upon inferring and adapting to other agents' goal-directed behavior. However, it remains unclear how humans achieve this, despite the fact that many decisions must account for complex, dynamic agents acting according to their own goals. Here, we use a predator-prey task (total n = 510) to demonstrate that humans exploit an interactive cognitive map of the social environment to infer other agents' preferences and simulate their future behavior, providing for flexible, generalizable responses. A model-based inverse reinforcement learning model explained participants' inferences about threatening agents' preferences, with participants using this inferred knowledge to enact generalizable, model-based behavioral responses. Using tree-search planning models, we then found that behavior was best explained by a planning algorithm that incorporated simulations of the threat's goal-directed behavior. Our results indicate that humans use a cognitive map to determine other agents' preferences, facilitating generalized predictions of their behavior and effective responses.

Examining the impact of cue similarity and fear learning on perceptual tuning

Past research on the effects of associative aversive learning on discrimination acuity has shown mixed results, including increases, decreases, and no changes in discrimination ability. An animal study found that the type of learning experience determined the direction and extent of learning-induced changes. The current preregistered web-based study aimed to translate these findings to humans. Experiment 1 (N = 245) compared changes in stimulus discrimination between simple learning (only one oriented grating cue), coarse differential conditioning (physically distinct cues), and fine differential conditioning (physically similar cues) as well as to their three respective control groups. The discrimination task consisted of a two-alternative-forced-choice task with oriented grating stimuli. During learning, a specific orientation was paired with unpleasant pictures. Our analysis using generative modeling demonstrated weak to moderate evidence that aversive learning did not alter discrimination acuity in any of the groups. In a follow-up experiment (N = 121), we replicated these findings despite successful learning trajectories in all three groups and a more detailed assessment of discrimination acuity. Contrary to prior assumptions, our findings indicate that aversive learning does not enhance perceptual discrimination, and the presence of additional safety cues does not appear to moderate this effect.

Trait anxiety is associated with hidden state inference during aversive reversal learning

Updating beliefs in changing environments can be driven by gradually adapting expectations or by relying on inferred hidden states (i.e. contexts), and changes therein. Previous work suggests that increased reliance on context could underly fear relapse phenomena that hinder clinical treatment of anxiety disorders. We test whether trait anxiety variations in a healthy population influence how much individuals rely on hidden-state inference. In a Pavlovian learning task, participants observed cues that predicted an upcoming electrical shock with repeatedly changing probability, and were asked to provide expectancy ratings on every trial. We show that trait anxiety is associated with steeper expectation switches after contingency reversals and reduced oddball learning. Furthermore, trait anxiety is related to better fit of a state inference, compared to a gradual learning, model when contingency changes are large. Our findings support previous work suggesting hidden-state inference as a mechanism behind anxiety-related to fear relapse phenomena.

The shadowing effect of initial expectation on learning asymmetry

Evidence for positivity and optimism bias abounds in high-level belief updates. However, no consensus has been reached regarding whether learning asymmetries exist in more elementary forms of updates such as reinforcement learning (RL). In RL, the learning asymmetry concerns the sensitivity difference in incorporating positive and negative prediction errors (PE) into value estimation, namely the asymmetry of learning rates associated with positive and negative PEs. Although RL has been established as a canonical framework in characterizing interactions between agent and environment, the direction of learning asymmetry remains controversial. Here, we propose that part of the controversy stems from the fact that people may have different value expectations before entering the learning environment. Such a default value expectation influences how PEs are calculated and consequently biases subjects' choices. We test this hypothesis in two learning experiments with stable or varying reinforcement probabilities, across monetary gains, losses, and gain-loss mixed environments. Our results consistently support the model incorporating both asymmetric learning rates and the initial value expectation, highlighting the role of initial expectation in value updating and choice preference. Further simulation and model parameter recovery analyses confirm the unique contribution of initial value expectation in accessing learning rate asymmetry.

From Computation to Clinic

Theory-driven and data-driven computational approaches to psychiatry have enormous potential for elucidating mechanism of disease and providing translational linkages between basic science findings and the clinic. These approaches have already demonstrated utility in providing clinically relevant understanding, primarily via back translation from clinic to computation, revealing how specific disorders or symptoms map onto specific computational processes. Nonetheless, forward translation, from computation to clinic, remains rare. In addition, consensus regarding specific barriers to forward translation-and on the best strategies to overcome these barriers-is limited. This perspective review brings together expert basic and computationally trained researchers and clinicians to 1) identify challenges specific to preclinical model systems and clinical translation of computational models of cognition and affect, and 2) discuss practical approaches to overcoming these challenges. In doing so, we highlight recent evidence for the ability of computational approaches to predict treatment responses in psychiatric disorders and discuss considerations for maximizing the clinical relevance of such models (e.g., via longitudinal testing) and the likelihood of stakeholder adoption (e.g., via cost-effectiveness analyses).

The Role of Expectancy Violation in Extinction Learning: A Two-Day Online Fear Conditioning Study

Background

Exposure therapy is at the core of the treatment of pathological anxiety. While the inhibitory learning model proposes a framework for the mechanisms underlying exposure therapy, in particular expectancy violation, causal evidence for its assumptions remains elusive. Therefore, the aim of the current study was to provide evidence for the influence of expectancy violation on extinction retention by manipulating the magnitude of expectancy violation during extinction learning.

Method

In total, 101 individuals completed a web-based fear conditioning protocol, consisting of a fear acquisition and extinction phase, as well as a spontaneous recovery and fear reinstatement test 24h later. To experimentally manipulate expectancy violation, participants were presented only with states of the conditioned stimulus that either weakly or strongly predicted the aversive outcome. Consequently, the absence of any aversive outcomes in the extinction phase resulted in low or high expectancy violation, respectively.

Results

We found successful fear acquisition and manipulation of expectancy violation, which was associated with reduced threat ratings for the high compared to the low expectancy violation group directly after extinction learning. On Day 2, inhibitory CS-noUS associations could be retrieved for expectancy ratings, whereas there were no substantial group differences for threat ratings.

Conclusion

These findings indicate that the magnitude of expectancy violation is related to the retrieval of conscious threat expectancies, but it is unclear how these changes translate to affective components (i.e., threat ratings) of the fear response and to symptoms of pathological anxiety.

Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy

In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.

Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty

People radically differ in how they cope with uncertainty. Clinical researchers describe a dispositional characteristic known as "intolerance of uncertainty", a tendency to find uncertainty aversive, reported to be elevated across psychiatric and neurodevelopmental conditions. Concurrently, recent research in computational psychiatry has leveraged theoretical work to characterise individual differences in uncertainty processing. Under this framework, differences in how people estimate different forms of uncertainty can contribute to mental health difficulties. In this review, we briefly outline the concept of intolerance of uncertainty within its clinical context, and we argue that the mechanisms underlying this construct may be further elucidated through modelling how individuals make inferences about uncertainty. We will review the evidence linking psychopathology to different computationally specified forms of uncertainty and consider how these findings might suggest distinct mechanistic routes towards intolerance of uncertainty. We also discuss the implications of this computational approach for behavioural and pharmacological interventions, as well as the importance of different cognitive domains and subjective experiences in studying uncertainty processing.

Identifying Transdiagnostic Mechanisms in Mental Health Using Computational Factor Modeling

Most psychiatric disorders do not occur in isolation, and most psychiatric symptom dimensions are not uniquely expressed within a single diagnostic category. Current treatments fail to work for around 25% to 40% of individuals, perhaps due at least in part to an overreliance on diagnostic categories in treatment development and allocation. In this review, we describe ongoing efforts in the field to surmount these challenges and precisely characterize psychiatric symptom dimensions using large-scale studies of unselected samples via remote, online, and "citizen science" efforts that take a dimensional, mechanistic approach. We discuss the importance that efforts to identify meaningful psychiatric dimensions be coupled with careful computational modeling to formally specify, test, and potentially falsify candidate mechanisms that underlie transdiagnostic symptom dimensions. We refer to this approach, i.e., where symptom dimensions are identified and validated against computationally well-defined neurocognitive processes, as computational factor modeling. We describe in detail some recent applications of this method to understand transdiagnostic cognitive processes that include model-based planning, metacognition, appetitive processing, and uncertainty estimation. In this context, we highlight how computational factor modeling has been used to identify specific associations between cognition and symptom dimensions and reveal previously obscured relationships, how findings generalize to smaller in-person clinical and nonclinical samples, and how the method is being adapted and optimized beyond its original instantiation. Crucially, we discuss next steps for this area of research, highlighting the value of more direct investigations of treatment response that bridge the gap between basic research and the clinic.

Disrupted state transition learning as a computational marker of compulsivity

BACKGROUND

Disorders involving compulsivity, fear, and anxiety are linked to beliefs that the world is less predictable. We lack a mechanistic explanation for how such beliefs arise. Here, we test a hypothesis that in people with compulsivity, fear, and anxiety, learning a probabilistic mapping between actions and environmental states is compromised.

METHODS

In Study 1 (n = 174), we designed a novel online task that isolated state transition learning from other facets of learning and planning. To determine whether this impairment is due to learning that is too fast or too slow, we estimated state transition learning rates by fitting computational models to two independent datasets, which tested learning in environments in which state transitions were either stable (Study 2: n = 1413) or changing (Study 3: n = 192).

RESULTS

Study 1 established that individuals with higher levels of compulsivity are more likely to demonstrate an impairment in state transition learning. Preliminary evidence here linked this impairment to a common factor comprising compulsivity and fear. Studies 2 and 3 showed that compulsivity is associated with learning that is too fast when it should be slow (i.e. when state transition are stable) and too slow when it should be fast (i.e. when state transitions change).

CONCLUSIONS

Together, these findings indicate that compulsivity is associated with a dysregulation of state transition learning, wherein the rate of learning is not well adapted to the task environment. Thus, dysregulated state transition learning might provide a key target for therapeutic intervention in compulsivity.

Anterior cingulate and medial prefrontal cortex oscillations underlie learning alterations in trait anxiety in humans

Anxiety has been linked to altered belief formation and uncertainty estimation, impacting learning. Identifying the neural processes underlying these changes is important for understanding brain pathology. Here, we show that oscillatory activity in the medial prefrontal, anterior cingulate and orbitofrontal cortex (mPFC, ACC, OFC) explains anxiety-related learning alterations. In a magnetoencephalography experiment, two groups of human participants pre-screened with high and low trait anxiety (HTA, LTA: 39) performed a probabilistic reward-based learning task. HTA undermined learning through an overestimation of volatility, leading to faster belief updating, more stochastic decisions and pronounced lose-shift tendencies. On a neural level, we observed increased gamma activity in the ACC, dmPFC, and OFC during encoding of precision-weighted prediction errors in HTA, accompanied by suppressed ACC alpha/beta activity. Our findings support the association between altered learning and belief updating in anxiety and changes in gamma and alpha/beta activity in the ACC, dmPFC, and OFC.

Computational models of subjective feelings in psychiatry

Research in computational psychiatry is dominated by models of behavior. Subjective experience during behavioral tasks is not well understood, even though it should be relevant to understanding the symptoms of psychiatric disorders. Here, we bridge this gap and review recent progress in computational models for subjective feelings. For example, happiness reflects not how well people are doing, but whether they are doing better than expected. This dependence on recent reward prediction errors is intact in major depression, although depressive symptoms lower happiness during tasks. Uncertainty predicts subjective feelings of stress in volatile environments. Social prediction errors influence feelings of self-worth more in individuals with low self-esteem despite a reduced willingness to change beliefs due to social feedback. Measuring affective state during behavioral tasks provides a tool for understanding psychiatric symptoms that can be dissociable from behavior. When smartphone tasks are collected longitudinally, subjective feelings provide a potential means to bridge the gap between lab-based behavioral tasks and real-life behavior, emotion, and psychiatric symptoms.

Trait somatic anxiety is associated with reduced directed exploration and underestimation of uncertainty

Anxiety has been related to decreased physical exploration, but past findings on the interaction between anxiety and exploration during decision making were inconclusive. Here we examined how latent factors of trait anxiety relate to different exploration strategies when facing volatility-induced uncertainty. Across two studies (total N = 985), we demonstrated that people used a hybrid of directed, random and undirected exploration strategies, which were respectively sensitive to relative uncertainty, total uncertainty and value difference. Trait somatic anxiety, that is, the propensity to experience physical symptoms of anxiety, was inversely correlated with directed exploration and undirected exploration, manifesting as a lesser likelihood for choosing the uncertain option and reducing choice stochasticity regardless of uncertainty. Somatic anxiety is also associated with underestimation of relative uncertainty. Together, these results reveal the selective role of trait somatic anxiety in modulating both uncertainty-driven and value-driven exploration strategies.

Computational perspectives on human fear and anxiety

Fear and anxiety are adaptive emotions that serve important defensive functions, yet in excess, they can be debilitating and lead to poor mental health. Computational modelling of behaviour provides a mechanistic framework for understanding the cognitive and neurobiological bases of fear and anxiety, and has seen increasing interest in the field. In this brief review, we discuss recent developments in the computational modelling of human fear and anxiety. Firstly, we describe various reinforcement learning strategies that humans employ when learning to predict or avoid threat, and how these relate to symptoms of fear and anxiety. Secondly, we discuss initial efforts to explore, through a computational lens, approach-avoidance conflict paradigms that are popular in animal research to measure fear- and anxiety-relevant behaviours. Finally, we discuss negative biases in decision-making in the face of uncertainty in anxiety.

Reproduction system development of Ceracris kiangsu Tsai female adults and its relationship with fitness characteristics

Research on the ovarian development of insect pests helps provide key information for predicting pest occurrences, and currently, there is very limited information about the reproductive system of Ceracris kiangsu Tsai. This study aimed to assess the reproductive fitness of 321 adult female insects by using traditional methods to dissect female adults, measure female ovaries, and assess the process of egg formation. The phenotypic traits including body weight and body length were also measured and used to estimate the model of ovarian developmental stages. Four ovarian developmental stages before the oviposition were identified, and the fundamental ovarian structure of C. kiangsu displayed red dots on the matured eggs inside the calyx at ovarian developmental stage V. The accessory glands of C. kiangsu had the deepest folds at stage Ⅲ. Redundancy analysis (RDA) was used to explore the correlation between ovarian development, body weight, and body length. A significant positive correlation was observed for body weight (p = 0.001) and body length (p = 0.009), which varied with the grade of ovarian development evaluated by the ovarian developmental stage, ovarian length, ovarian width, and ovarian cross-sectional area. A partial least square (PLS) regression was used to model the ovarian developmental stage, with a stage-based PLS being identified as the more effective method, which was y = 1.509x ₁ + 0.114x ₂. The model provides a potentially rapid way to identify the population source as either "native" or "immigrant" from the phenotypic traits without dissection. The aforementioned model may be used to estimate adult emergence periods and identify migratory populations from their ovarian development, potentially aiding in implementing proper prevention measures.

Model-based prioritization for acquiring protection

Protection often involves the capacity to prospectively plan the actions needed to mitigate harm. The computational architecture of decisions involving protection remains unclear, as well as whether these decisions differ from other beneficial prospective actions such as reward acquisition. Here we compare protection acquisition to reward acquisition and punishment avoidance to examine overlapping and distinct features across the three action types. Protection acquisition is positively valenced similar to reward. For both protection and reward, the more the actor gains, the more benefit. However, reward and protection occur in different contexts, with protection existing in aversive contexts. Punishment avoidance also occurs in aversive contexts, but differs from protection because punishment is negatively valenced and motivates avoidance. Across three independent studies (Total N = 600) we applied computational modeling to examine model-based reinforcement learning for protection, reward, and punishment in humans. Decisions motivated by acquiring protection evoked a higher degree of model-based control than acquiring reward or avoiding punishment, with no significant differences in learning rate. The context-valence asymmetry characteristic of protection increased deployment of flexible decision strategies, suggesting model-based control depends on the context in which outcomes are encountered as well as the valence of the outcome.

Interindividual variability in functional connectivity discovers differential development of cognition and transdiagnostic dimensions of psychopathology in youth

Cognitive and psychological development during adolescence is different from one another, which is rooted in individual differences in maturational changes in the adolescent brain. This study employed multi-modal MRI data and characterized interindividual variability in functional connectivity (IVFC) and its associations with cognition and psychopathology using the Philadelphia Neurodevelopmental Cohort (PNC) of 755 youth. We employed resting state functional MRI (rs-fMRI) and diffusion weighted images (DWIs) to estimate brain structural and functional networks. We computed the IVFC of individuals and examined its relation with structural and functional organizations. We further employed sparse partial least squares (sparse-PLS) and meta-analysis to examine the developmental associations of the IVFC with cognition and transdiagnostic dimensions of psychopathology in early, middle, and late adolescence. Our results revealed that the IVFC spatial topography reflects the brain functional integration and structure-function decoupling. Age effects on the IVFC of association networks were mediated by the FC among the triple networks, including frontoparietal, salience, and default mode networks (DMN), while those of primary and cerebellar networks were mediated by the cerebello-cortical FC. The IVFC of the triple and cerebellar networks explained the variance of executive functions and externalizing behaviors in early adolescence and then the variance of emotion and internalizing and psychosis in middle and late adolescence. We further evaluated this finding via meta-analysis on task-based studies on cognition and psychopathology. These findings implicate the emerging importance of the IVFC of the triple and cerebellar networks in cognitive, emotional, and psychopathological development during adolescence.

A corticostriatal pathway mediating self-efficacy enhancement

Forming positive beliefs about one's ability to perform challenging tasks, often termed self-efficacy, is fundamental to motivation and emotional well-being. Self-efficacy crucially depends on positive social feedback, yet people differ in the degree to which they integrate such feedback into self-beliefs (i.e., positive bias). While diminished positive bias of this sort is linked to mood and anxiety, the neural processes by which positive feedback on public performance enhances self-efficacy remain unclear. To address this, we conducted a behavioral and fMRI study wherein participants delivered a public speech and received fictitious positive and neutral feedback on their performance in the MRI scanner. Before and after receiving feedback, participants evaluated their actual and expected performance. We found that reduced positive bias in updating self-efficacy based on positive social feedback associated with a psychopathological dimension reflecting symptoms of anxiety, depression, and low self-esteem. Analysis of brain encoding of social feedback showed that a positive self-efficacy update bias associated with a stronger reward-related response in the ventral striatum (VS) and stronger coupling of the VS with a temporoparietal region involved in self-processing. Together, our findings demarcate a corticostriatal circuit that promotes positive bias in self-efficacy updating based on social feedback, and highlight the centrality of such bias to emotional well-being.

The computational roots of positivity and confirmation biases in reinforcement learning

Humans do not integrate new information objectively: outcomes carrying a positive affective value and evidence confirming one's own prior belief are overweighed. Until recently, theoretical and empirical accounts of the positivity and confirmation biases assumed them to be specific to 'high-level' belief updates. We present evidence against this account. Learning rates in reinforcement learning (RL) tasks, estimated across different contexts and species, generally present the same characteristic asymmetry, suggesting that belief and value updating processes share key computational principles and distortions. This bias generates over-optimistic expectations about the probability of making the right choices and, consequently, generates over-optimistic reward expectations. We discuss the normative and neurobiological roots of these RL biases and their position within the greater picture of behavioral decision-making theories.

Computational modeling of threat learning reveals links with anxiety and neuroanatomy in humans

Influential theories implicate variations in the mechanisms supporting threat learning in the severity of anxiety symptoms. We use computational models of associative learning in conjunction with structural imaging to explicate links among the mechanisms underlying threat learning, their neuroanatomical substrates, and anxiety severity in humans. We recorded skin-conductance data during a threat-learning task from individuals with and without anxiety disorders (N=251; 8-50 years; 116 females). Reinforcement-learning model variants quantified processes hypothesized to relate to anxiety: threat conditioning, threat generalization, safety learning, and threat extinction. We identified the best-fitting models for these processes and tested associations among latent learning parameters, whole-brain anatomy, and anxiety severity. Results indicate that greater anxiety severity related specifically to slower safety learning and slower extinction of response to safe stimuli. Nucleus accumbens gray-matter volume moderated learning-anxiety associations. Using a modeling approach, we identify computational mechanisms linking threat learning and anxiety severity and their neuroanatomical substrates.

Evaluating an internet-delivered fear conditioning and extinction protocol using response times and affective ratings

Pavlovian fear conditioning is widely used to study mechanisms of fear learning, but high-throughput studies are hampered by the labor-intensive nature of examining participants in the lab. To circumvent this bottle-neck, fear conditioning tasks have been developed for remote delivery. Previous studies have examined remotely delivered fear conditioning protocols using expectancy and affective ratings. Here we replicate and extend these findings using an internet-delivered version of the Screaming Lady paradigm, evaluating the effects on negative affective ratings and response time to an auditory probe during stimulus presentation. In a sample of 80 adults, we observed clear evidence of both fear acquisition and extinction using affective ratings. Response times were faster when probed early, but not later, during presentation of stimuli paired with an aversive scream. The response time findings are at odds with previous lab-based studies showing slower as opposed to faster responses to threat-predicting cues. The findings underscore the feasibility of employing remotely delivered fear conditioning paradigms with affective ratings as outcome. Findings further highlight the need for research examining optimal parameters for concurrent response time measures or alternate non-verbal indicators of conditioned responses in Pavlovian conditioning protocols.

The cerebellum and psychological trauma: A systematic review of neuroimaging studies

Psychological trauma is highly prevalent among psychiatric disorders, however, the relationship between trauma, neurobiology and psychopathology is not yet fully understood. The cerebellum has been recognized as a crucial structure for cognition and emotion, however, it has been relatively ignored in the literature of psychological trauma, as it is not considered as part of the traditional fear neuro-circuitry. The aim of this review is to investigate how psychological trauma affects the cerebellum and to make conclusive remarks on whether the cerebellum forms part of the trauma-affected brain circuitry. A total of 267 unique records were screened and 39 studies were included in the review. Structural cerebellar alterations and aberrant cerebellar activity and connectivity in trauma-exposed individuals were consistently reported across studies. Early-onset of adverse experiences was associated with cerebellar alterations in trauma-exposed individuals. Several studies reported alterations in connectivity between the cerebellum and nodes of large-brain networks, which are implicated in several psychiatric disorders, including the default mode network, the salience network and the central executive network. Also, trauma-exposed individuals showed altered resting state and task based cerebellar connectivity with cortical and subcortical structures that are involved in emotion and fear regulation. Our preferred interpretation of the results is through the lens of the Universal Cerebellar Transform, the hypothesis that the cerebellum, given its homogeneous cytoarchitecture, performs a common computation for motor, cognitive and emotional functions. Therefore, trauma-induced alterations in this computation might set the ground for a variety of psychiatric symptoms.

Humans perseverate on punishment avoidance goals in multigoal reinforcement learning

Managing multiple goals is essential to adaptation, yet we are only beginning to understand computations by which we navigate the resource demands entailed in so doing. Here, we sought to elucidate how humans balance reward seeking and punishment avoidance goals, and relate this to variation in its expression within anxious individuals. To do so, we developed a novel multigoal pursuit task that includes trial-specific instructed goals to either pursue reward (without risk of punishment) or avoid punishment (without the opportunity for reward). We constructed a computational model of multigoal pursuit to quantify the degree to which participants could disengage from the pursuit goals when instructed to, as well as devote less model-based resources toward goals that were less abundant. In general, participants (n = 192) were less flexible in avoiding punishment than in pursuing reward. Thus, when instructed to pursue reward, participants often persisted in avoiding features that had previously been associated with punishment, even though at decision time these features were unambiguously benign. In a similar vein, participants showed no significant downregulation of avoidance when punishment avoidance goals were less abundant in the task. Importantly, we show preliminary evidence that individuals with chronic worry may have difficulty disengaging from punishment avoidance when instructed to seek reward. Taken together, the findings demonstrate that people avoid punishment less flexibly than they pursue reward. Future studies should test in larger samples whether a difficulty to disengage from punishment avoidance contributes to chronic worry.

The global prevalence of depression, suicide ideation, and attempts in the military forces: a systematic review and Meta-analysis of cross sectional studies

BACKGROUND

Given the wide range of depressive disorders, suicidal ideation and suicide attempts in various military studies around the world, determining the exact prevalence of these disorders in line with health planning as well as care and treatment service designing for military forces can be useful. The aim of the present meta-analysis was to determine the pooled prevalence of depressive disorders, suicide thoughts, and attempts in the military.

METHODS

The present systematic review and meta-analysis study was performed based on PRISMA criteria in 5 steps of the search strategy, screening and selection of articles, data extraction, evaluation of article quality and meta-analysis. International databases (PubMed (Medline), Scopus, Web of science, Embase (Elsevier), PsycInfo (Ovid), Cochrane CENTRAL (Ovid)) were searched using related keywords extracted from Mesh and Emtree. After screening and final selection of articles, data were extracted and qualitative evaluation was performed using the NOS checklist.

RESULTS

The results of meta-analysis showed that the prevalence of depression in active military forces and veterans was 23% (%95 CI: 20-26%) and 20% (%95 CI: 18-22%), respectively. In addition, the prevalence of suicidal ideation and attempts in the military was 11% (%95 CI: 10-13%) and 11% (%95 CI: 9-13%), respectively. The prevalence of suicide ideation and attempts in drug-using military was 18% (%95 CI: 7-33%) and 30% (%95 CI: 23-36%), respectively. The prevalence of suicidal ideation and attempts in military consuming alcohol were 9% (%95 CI: 4-13%) and 8% (%95 CI: 7-10%), respectively. In militaries with AIDS / HIV, the prevalence of suicide attempts was 5% (%95 CI: 4-8%).

CONCLUSION

Therefore, it is necessary to develop and design training and intervention programs in order to increase the awareness of the military, especially veterans, to prevent the occurrence of suicide and depression.

How Local and Global Metacognition Shape Mental Health

Metacognition is the ability to reflect on our own cognition and mental states. It is a critical aspect of human subjective experience and operates across many hierarchical levels of abstraction-encompassing local confidence in isolated decisions and global self-beliefs about our abilities and skills. Alterations in metacognition are considered foundational to neurologic and psychiatric disorders, but research has mostly focused on local metacognitive computations, missing out on the role of global aspects of metacognition. Here, we first review current behavioral and neural metrics of local metacognition that lay the foundation for this research. We then address the neurocognitive underpinnings of global metacognition uncovered by recent studies. Finally, we outline a theoretical framework in which higher hierarchical levels of metacognition may help identify the role of maladaptive metacognitive evaluation in mental health conditions, particularly when combined with transdiagnostic methods.

Psychiatric symptoms influence reward-seeking and loss-avoidance decision-making through common and distinct computational processes

AIM

Psychiatric symptoms are often accompanied by impairments in decision-making to attain rewards and avoid losses. However, due to the complex nature of mental disorders (e.g., high comorbidity), symptoms that are specifically associated with deficits in decision-making remain unidentified. Furthermore, the influence of psychiatric symptoms on computations underpinning reward-seeking and loss-avoidance decision-making remains elusive. Here, we aim to address these issues by leveraging a large-scale online experiment and computational modeling.

METHODS

In the online experiment, we recruited 1900 non-diagnostic participants from the general population. They performed either a reward-seeking or loss-avoidance decision-making task, and subsequently completed questionnaires about psychiatric symptoms.

RESULTS

We found that one trans-diagnostic dimension of psychiatric symptoms related to compulsive behavior and intrusive thought (CIT) was negatively correlated with overall decision-making performance in both the reward-seeking and loss-avoidance tasks. A deeper analysis further revealed that, in both tasks, the CIT psychiatric dimension was associated with lower preference for the options that recently led to better outcomes (i.e. reward or no-loss). On the other hand, in the reward-seeking task only, the CIT dimension was associated with lower preference for recently unchosen options.

CONCLUSION

These findings suggest that psychiatric symptoms influence the two types of decision-making, reward-seeking and loss-avoidance, through both common and distinct computational processes.

Associative learning shapes visual discrimination in a web-based classical conditioning task

Threat detection plays a vital role in adapting behavior to changing environments. A fundamental function to improve threat detection is learning to differentiate between stimuli predicting danger and safety. Accordingly, aversive learning should lead to enhanced sensory discrimination of danger and safety cues. However, studies investigating the psychophysics of visual and auditory perception after aversive learning show divergent findings, and both enhanced and impaired discrimination after aversive learning have been reported. Therefore, the aim of this web-based study is to examine the impact of aversive learning on a continuous measure of visual discrimination. To this end, 205 participants underwent a differential fear conditioning paradigm before and after completing a visual discrimination task using differently oriented grating stimuli. Participants saw either unpleasant or neutral pictures as unconditioned stimuli (US). Results demonstrated sharpened visual discrimination for the US-associated stimulus (CS+), but not for the unpaired conditioned stimuli (CS-). Importantly, this finding was irrespective of the US's valence. These findings suggest that associative learning results in increased stimulus salience, which facilitates perceptual discrimination in order to prioritize attentional deployment.

Promises and challenges of human computational ethology

The movements an organism makes provide insights into its internal states and motives. This principle is the foundation of the new field of computational ethology, which links rich automatic measurements of natural behaviors to motivational states and neural activity. Computational ethology has proven transformative for animal behavioral neuroscience. This success raises the question of whether rich automatic measurements of behavior can similarly drive progress in human neuroscience and psychology. New technologies for capturing and analyzing complex behaviors in real and virtual environments enable us to probe the human brain during naturalistic dynamic interactions with the environment that so far were beyond experimental investigation. Inspired by nonhuman computational ethology, we explore how these new tools can be used to test important questions in human neuroscience. We argue that application of this methodology will help human neuroscience and psychology extend limited behavioral measurements such as reaction time and accuracy, permit novel insights into how the human brain produces behavior, and ultimately reduce the growing measurement gap between human and animal neuroscience.

Reliability of web-based affective auditory stimulus presentation

Web-based experimental testing has seen exponential growth in psychology and cognitive neuroscience. However, paradigms involving affective auditory stimuli have yet to adapt to the online approach due to concerns about the lack of experimental control and other technical challenges. In this study, we assessed whether sounds commonly used to evoke affective responses in-lab can be used online. Using recent developments to increase sound presentation quality, we selected 15 commonly used sound stimuli and assessed their impact on valence and arousal states in a web-based experiment. Our results reveal good inter-rater and test-retest reliabilities, with results comparable to in-lab studies. Additionally, we compared a variety of previously used unpleasant stimuli, allowing us to identify the most aversive among these sounds. Our findings demonstrate that affective sounds can be reliably delivered through web-based platforms, which help facilitate the development of new auditory paradigms for affective online experiments.

Model-based aversive learning in humans is supported by preferential task state reactivation

Harm avoidance is critical for survival, yet little is known regarding the neural mechanisms supporting avoidance in the absence of trial-and-error experience. Flexible avoidance may be supported by a mental model (i.e., model-based), a process for which neural reactivation and sequential replay have emerged as candidate mechanisms. During an aversive learning task, combined with magnetoencephalography, we show prospective and retrospective reactivation during planning and learning, respectively, coupled to evidence for sequential replay. Specifically, when individuals plan in an aversive context, we find preferential reactivation of subsequently chosen goal states. Stronger reactivation is associated with greater hippocampal theta power. At outcome receipt, unchosen goal states are reactivated regardless of outcome valence. Replay of paths leading to goal states was modulated by outcome valence, with aversive outcomes associated with stronger reverse replay than safe outcomes. Our findings are suggestive of avoidance involving simulation of unexperienced states through hippocampally mediated reactivation and replay.

Safety learning and the Pavlovian conditioned inhibition of fear in humans: Current state and future directions

Safety learning occurs when an otherwise neutral stimulus comes to signal the absence of threat, allowing organisms to use safety information to inhibit fear and anxiety in nonthreatening environments. Although it continues to emerge as a topic of relevance in biological and clinical psychology, safety learning remains inconsistently defined and under-researched. Here, we analyse the Pavlovian conditioned inhibition paradigm and its application to the study of safety learning in humans. We discuss existing studies; address outstanding theoretical considerations; and identify prospects for its further application. Though Pavlovian conditioned inhibition presents a theoretically sound model of safety learning, it has been investigated infrequently, with decade-long interims between some studies, and notable methodological variability. Consequently, we argue that the full potential of conditioned inhibition as a model for human safety learning remains untapped, and propose that it could be revisited as a framework for addressing timely questions in the behavioural and clinical sciences.

Population Modelling in Affective Disorders

Purpose of Review

The prevalence of affective disorders is on the rise. This upward trajectory leads to a substantial personal and societal cost. There is growing body of literature demonstrating decision-making impairments associated with affective disorders, and more studies are using computational modelling methods to infer underlying mechanisms of these impairments from participant choice behaviour. However, lack of population modelling suggests that data resources may still be underutilised.

Recent Findings

A number of recent studies associated major depression with abnormal risky decision-making as well as impairments in temporal discounting and social decision-making. These domains capture relevant aspects of real-life decision-making. Consequently, data from these studies can be used to define behavioural phenotypes for major depression.

Summary

The manuscript describes a detailed proposal for population modelling to capture changes in the prevalence rate of major depression. The population modelling approach can also identify which decision-making domains can account for a larger part of impairments in psychosocial functioning and how behavioural interventions built on computational principles can target these to improve real-life psychosocial functioning in patient groups.

Latent cause inference during extinction learning in trauma-exposed individuals with and without PTSD

BACKGROUND

Problems in learning that sights, sounds, or situations that were once associated with danger have become safe (extinction learning) may explain why some individuals suffer prolonged psychological distress following traumatic experiences. Although simple learning models have been unable to provide a convincing account of why this learning fails, it has recently been proposed that this may be explained by individual differences in beliefs about the causal structure of the environment.

METHODS

Here, we tested two competing hypotheses as to how differences in causal inference might be related to trauma-related psychopathology, using extinction learning data collected from clinically well-characterised individuals with varying degrees of post-traumatic stress (N = 56). Model parameters describing individual differences in causal inference were related to multiple post-traumatic stress disorder (PTSD) and depression symptom dimensions via network analysis.

RESULTS

Individuals with more severe PTSD were more likely to assign observations from conditioning and extinction stages to a single underlying cause. Specifically, greater re-experiencing symptom severity was associated with a lower likelihood of inferring that multiple causes were active in the environment.

CONCLUSIONS

We interpret these results as providing evidence of a primary deficit in discriminative learning in participants with more severe PTSD. Specifically, a tendency to attribute a greater diversity of stimulus configurations to the same underlying cause resulted in greater uncertainty about stimulus-outcome associations, impeding learning both that certain stimuli were safe, and that certain stimuli were no longer dangerous. In the future, better understanding of the role of causal inference in trauma-related psychopathology may help refine cognitive therapies for these disorders.