Decision making in avoidance–reward conflict: a paradigm for non-human primates and humans

Temporal dynamics of costly avoidance in naturalistic fears: Evidence for sequential-sampling of fear and reward information

Excessive avoidance is characteristic for anxiety disorders, even when approach would lead to positive outcomes. The process of how such approach-avoidance conflicts are resolved is not sufficiently understood. We examined the temporal dynamics of approach-avoidance in intense fear of spiders. Highly fearful and non-fearful participants chose repeatedly between a fixed no spider/low reward and a spider/high reward option with varying fear (probability of spider presentation) and reward information (reward magnitude). By sequentially presenting fear and reward information, we distinguished whether decisions are dynamically driven by both information (sequential-sampling) or whether the impact of fear information is inhibited (cognitive control). Mouse movements were recorded to assess temporal decision dynamics (i.e., how strongly which information impacts decision preference at which timepoint). Highly fearful participants showed stronger avoidance despite lower gains (i.e., costly avoidance). Time-continuous multiple regression of their mouse movements yielded a stronger impact of fear compared to reward information. Importantly, presenting either information first (fear or reward) enhanced its impact during the early decision process. These findings support sequential sampling of fear and reward information, but not inhibitory control. Hence, pathological avoidance may be characterized by biased evidence accumulation rather than altered cognitive control.

Itsy bitsy spider: Fear and avoidance (generalization) in a free-exploratory virtual reality paradigm

Most experimental avoidance paradigms lack either control over the experimental situation or simplify real-life avoidance behavior to a great extent, making it difficult to generalize the results to the complex approach-avoidance situations that anxious individuals face in daily life. The current study aimed to examine the usability of our recently developed free-exploratory avoidance paradigm in Virtual Reality (VR) that allows for the assessment of subjective as well as behavioral avoidance in participants with varying levels of spider fear. In a VR escape room, participants searched for cues to decipher a code-locked door. Opening a wooden box marked with a post-it note (conditioned stimulus, CS) resulted in exposure to a virtual crawling spider (unconditioned stimulus, US). Avoidance of the original CS and other objects marked with the CS (generalization stimuli, GSs; EXPgen condition) or non-marked (CONT condition) objects was measured via questionnaires and relative manipulation times in a novel room. We expected a positive linear relationship between US aversiveness (levels of spider fear) and (generalization of) fear and avoidance behaviors. Avoidance learning and generalization was demonstrated on both a subjective and behavioral level. Higher levels of spider fear were, overall, related to more negative emotions in response to the encounter with the spider, higher US expectancies for the GSs, and more self-reported and behavioral avoidance of the original CS and the GSs. Finally, we explored relationships between trait anxiety and intolerance of uncertainty and fear and avoidance (generalization), but no robust associations were observed. In conclusion, we confirmed the expected positive linear relationship between spider fear and (generalization of) fear and avoidance behaviors. Our results suggest that our free-exploratory VR avoidance paradigm is well-suited to investigate avoidance behaviors and the generalization of avoidance.

Approach-avoidance reinforcement learning as a translational and computational model of anxiety-related avoidance

Although avoidance is a prevalent feature of anxiety-related psychopathology, differences in the measurement of avoidance between humans and non-human animals hinder our progress in its theoretical understanding and treatment. To address this, we developed a novel translational measure of anxiety-related avoidance in the form of an approach-avoidance reinforcement learning task, by adapting a paradigm from the non-human animal literature to study the same cognitive processes in human participants. We used computational modelling to probe the putative cognitive mechanisms underlying approach-avoidance behaviour in this task and investigated how they relate to subjective task-induced anxiety. In a large online study (n = 372), participants who experienced greater task-induced anxiety avoided choices associated with punishment, even when this resulted in lower overall reward. Computational modelling revealed that this effect was explained by greater individual sensitivities to punishment relative to rewards. We replicated these findings in an independent sample (n = 627) and we also found fair-to-excellent reliability of measures of task performance in a sub-sample retested 1 week later (n = 57). Our findings demonstrate the potential of approach-avoidance reinforcement learning tasks as translational and computational models of anxiety-related avoidance. Future studies should assess the predictive validity of this approach in clinical samples and experimental manipulations of anxiety.

Quantifying aberrant approach-avoidance conflict in psychopathology: A review of computational approaches

Making effective decisions during approach-avoidance conflict is critical in daily life. Aberrant decision-making during approach-avoidance conflict is evident in a range of psychological disorders, including anxiety, depression, trauma-related disorders, substance use disorders, and alcohol use disorders. To help clarify etiological pathways and reveal novel intervention targets, clinical research into decision-making is increasingly adopting a computational psychopathology approach. This approach uses mathematical models that can identify specific decision-making related processes that are altered in mental health disorders. In our review, we highlight foundational approach-avoidance conflict research, followed by more in-depth discussion of computational approaches that have been used to model behavior in these tasks. Specifically, we describe the computational models that have been applied to approach-avoidance conflict (e.g., drift-diffusion, active inference, and reinforcement learning models), and provide resources to guide clinical researchers who may be interested in applying computational modeling. Finally, we identify notable gaps in the current literature and potential future directions for computational approaches aimed at identifying mechanisms of approach-avoidance conflict in psychopathology.

Understanding clinical fear and anxiety through the lens of human fear conditioning

Fear is an adaptive emotion that mobilizes defensive resources upon confrontation with danger. However, fear becomes maladaptive and can give rise to the development of clinical anxiety when it exceeds the degree of threat, generalizes broadly across stimuli and contexts, persists after the danger is gone or promotes excessive avoidance behaviour. Pavlovian fear conditioning has been the prime research instrument that has led to substantial progress in understanding the multi-faceted psychological and neurobiological mechanisms of fear in past decades. In this Perspective, we suggest that fruitful use of Pavlovian fear conditioning as a laboratory model of clinical anxiety requires moving beyond the study of fear acquisition to associated fear conditioning phenomena: fear extinction, generalization of conditioned fear and fearful avoidance. Understanding individual differences in each of these phenomena, not only in isolation but also in how they interact, will further strengthen the external validity of the fear conditioning model as a tool with which to study maladaptive fear as it manifests in clinical anxiety.

The action dynamics of approach-avoidance conflict during decision-making

Approach-avoidance conflict is observed in the competing motivations towards the benefits and away from the costs of a decision. The current study investigates the action dynamics of response motion during such conflicts in an attempt to characterise their dynamic resolution. An approach-avoidance conflict was generated by varying the appetitive consequences of a decision (i.e., point rewards and shorter participation time) in the presence of simultaneous aversive consequences (i.e., shock probability). Across two experiments, approach-avoidance conflict differentially affected response trajectories. Approach trajectories were less complex than avoidance trajectories and, as approach and avoidance motivations neared equipotentiality, response trajectories were more deflected from the shortest route to the eventual choice. Consistency in the location of approach and avoidance response options reduced variability in performance enabling more sensitive estimates of dynamic conflict. The time course of competing influences on response trajectories including trial-to-trial effects and conflict between approach and avoidance were estimated using regression analyses. We discuss these findings in terms of a dynamic theory of approach-avoidance that we hope will lead to insights of practical relevance in the field of maladaptive avoidance.

Alike, But Not Quite: Comparing the Generalization of Pain-Related Fear and Pain-Related Avoidance

Pain-related fear and -avoidance crucially contribute to pain chronification. People with chronic pain may adopt costly avoidance strategies above and beyond what is necessary, aligning with experimental findings of excessive fear generalization to safe movements in these populations. Furthermore, recent evidence suggests that, when avoidance is costly, it can dissociate from fear. Here, we investigated whether concurrently measured pain-related fear and costly avoidance generalization correspond in one task. We also explored whether healthy participants avoid excessively despite associated costs, and if avoidance would decrease as a function of dissimilarity from a pain-associated movement. In a robotic arm-reaching task, participants could avoid a low-cost, pain-associated movement trajectory (T+), by choosing a high-cost non-painful movement trajectory (T-), at opposite ends of a movement plane. Subsequently, in the absence of pain, we introduced three movement trajectories (G1-3) between T+ and T-, and one movement trajectory on the side of T- opposite to T+ (G4), linearly increasing in costs from T+ to G4. Avoidance was operationalized as maximal deviation from T+, and as trajectory choice. Fear learning was measured using self-reported pain-expectancy, pain-related fear, and startle eye-blink electromyography. Self-reports generalized, both decreasing with increasing distance from T+. In contrast, all generalization trajectories were chosen equally, suggesting that avoidance-costs and previous pain balanced each other out. No effects emerged in the electromyography. These results add to a growing body of literature showing that (pain-related) avoidance, especially when costly, can dissociate from fear, calling for a better understanding of the factors motivating, and mitigating, disabling avoidance. PERSPECTIVE: This article presents a comparison of pain-related fear- and avoidance generalization, and an exploration of excessive avoidance in healthy participants. Our findings show that pain-related avoidance can dissociate from fear, especially when avoidance is costly, calling for a better understanding of the factors motivating and mitigating disabling avoidance.

Costly avoidance of Pavlovian fear stimuli and the temporal dynamics of its decision process

Conflicts between avoiding feared stimuli versus approaching them for competing rewards are essential for functional behavior and anxious psychopathology. Yet, little is known about the underlying decision process. We examined approach-avoidance decisions and their temporal dynamics when avoiding Pavlovian fear stimuli conflicted with gaining rewards. First, a formerly neutral stimulus (CS+) was repeatedly paired with an aversive stimulus (US) to establish Pavlovian fear. Another stimulus (CS−) was never paired with the US. A control group received neutral tones instead of aversive USs. Next, in each of 324 trials, participants chose between a CS−/low reward and a CS+/high reward option. For the latter, probability of CS+ presentation (Pavlovian fear information) and reward magnitude (reward information) varied. Computer mouse movements were tracked to capture the decision dynamics. Although no more USs occurred, pronounced and persistent costly avoidance of the Pavlovian fear CS+ was found. Time-continuous multiple regression of movement trajectories revealed a stronger and faster impact of Pavlovian fear compared to reward information during decision-making. The impact of fear information, but not reward information, modestly decreased across trials. These findings suggest a persistently stronger weighting of fear compared to reward information during approach-avoidance decisions, which may facilitate the development of pathological avoidance.

Elevated costly avoidance in anxiety disorders: Patients show little downregulation of acquired avoidance in face of competing rewards for approach

BACKGROUND

Pathological avoidance is a transdiagnostic characteristic of anxiety disorders. Avoidance conditioning re-emerged as a translational model to examine mechanisms and treatment of avoidance. However, its validity for anxiety disorders remains unclear.

METHODS

This study tested for altered avoidance in patients with anxiety disorders compared to matched controls (n = 40/group) using instrumental conditioning assessing low-cost avoidance (avoiding a single aversive outcome) and costly avoidance (avoidance conflicted with gaining rewards). Autonomic arousal and threat expectancy were assessed as indicators of conditioned fear. Associations with dimensional symptom severity were examined.

RESULTS

Patients and controls showed frequent low-cost avoidance without group differences. Controls subsequently inhibited avoidance to gain rewards, which was amplified when aversive outcomes discontinued. In contrast, patients failed to reduce avoidance when aversive and positive outcomes competed (elevated costly avoidance) and showed limited reduction when aversive outcomes discontinued (persistent costly avoidance). Interestingly, elevated costly avoidance was not linked to higher conditioned fear in patients. Moreover, individual data revealed a bimodal distribution of costly avoidance: Some patients showed persistent avoidance, others showed little to no avoidance. Persistent versus low avoiders did not differ in other task-related variables, response to gains and losses in absence of threat, sociodemographic data, or clinical characteristics.

CONCLUSIONS

Findings suggest that anxious psychopathology is associated with a deficit to inhibit avoidance in presence of competing positive outcomes. This offers novel perspectives for research on mechanisms and treatment of anxiety disorders.

Motivational competition and the paraventricular thalamus

Although significant progress has been made in understanding the behavioral and brain mechanisms for motivational systems, much less is known about competition between motivational states or motivational conflict (e.g., approach - avoidance conflict). Despite being produced under diverse conditions, behavior during motivational competition has two signatures: bistability and metastability. These signatures reveal the operation of positive feedback mechanisms in behavioral selection. Different neuronal architectures can instantiate this selection to achieve bistability and metastability in behavior, but each relies on circuit-level inhibition to achieve rapid and stable selection between competing tendencies. Paraventricular thalamus (PVT) is identified as critical to this circuit level inhibition, resolving motivational competition via its extensive projections to local inhibitory networks in the ventral striatum and extended amygdala, enabling adaptive responding under motivational conflict.

Deep brain stimulation for psychiatric disorders: From focal brain targets to cognitive networks

Deep brain stimulation (DBS) is a promising intervention for treatment-resistant psychiatric disorders, particularly major depressive disorder (MDD) and obsessive-compulsive disorder (OCD). Up to 90% of patients who have not recovered with therapy or medication have reported benefit from DBS in open-label studies. Response rates in randomized controlled trials (RCTs), however, have been much lower. This has been argued to arise from surgical variability between sites, and recent psychiatric DBS research has focused on refining targeting through personalized imaging. Much less attention has been given to the fact that psychiatric disorders arise from dysfunction in distributed brain networks, and that DBS likely acts by altering communication within those networks. This is in part because psychiatric DBS research relies on subjective rating scales that make it difficult to identify network biomarkers. Here, we overview recent DBS RCT results in OCD and MDD, as well as the follow-on imaging studies. We present evidence for a new approach to studying DBS' mechanisms of action, focused on measuring objective cognitive/emotional deficits that underpin these and many other mental disorders. Further, we suggest that a focus on cognition could lead to reliable network biomarkers at an electrophysiologic level, especially those related to inter-regional synchrony of the local field potential (LFP). Developing the network neuroscience of DBS has the potential to finally unlock the potential of this highly specific therapy.

Incentive-based, instructed, and social observational extinction of avoidance: Fear-opposite actions and their influence on fear extinction

Avoidance is a transdiagnostic symptom of clinical anxiety and its reduction a major focus of cognitive-behavioral treatments. This study examined the instrumental extinction of goal-directed avoidance by means of incentives, verbal instruction, and social observation and their influence on fear extinction. Participants acquired conditioned fear and instrumental avoidance responses (N = 160). In four randomized groups, the reduction of avoidance by incentives for non-avoidance, instructions to refrain from avoidance, and social observation of non-avoidance was compared to no intervention before removing the aversive outcome. Conditioned fear when avoidance became unavailable subsequently was tested. Incentives, instruction, and observation all reduced avoidance better than no intervention, however, with different degrees and influence on conditioned fear. Incentives and instructions strongly reduced avoidance despite high levels of fear (i.e., fear-opposite actions). This initiated fear extinction, thereby reducing conditioned fear when avoidance became unavailable. Social observation directly reduced conditioned fear, presumably because it conveyed additional information about the absence of the aversive outcome. However, observation only moderately reduced avoidance and resulted in higher fear when avoidance became unavailable. The effects of social observation may depend on the nuances of the demonstrator's behavior. The clear effects of incentive and instructions provide support for clinical interventions to reduce avoidance during exposure therapy and can serve as experimental models for their controlled investigation.

Testing a gamified Spider App to reduce spider fear and avoidance

Mobile applications are increasingly part of mental health programs and various apps have been developed for treating anxiety disorders. Typically, they aim to improve anxiety symptoms via established CBT techniques, such as exposure principles, which are considered extremely unpleasant for fearful individuals. We combined in a mobile application exposure principles with gamification elements (e.g. narrative background, level progression, points, and feedback). These elements should increase the motivation for confronting spider images and decrease the experienced distress. To evaluate the application, two groups of spider-fearful individuals played either the Spider App (experimental group) or a non-spider associated app (control group) twice a day for approximately 12 min for 7 days. After this week, participants of the experimental group showed less avoidance behavior of spiders (BAT), as well as lower anxiety of spiders (SPQ, FAS). Groups were not different in measures of depression or psychological distress. Interestingly, participants playing the Spider App reported higher anxiety, disgust and arousal ratings shortly after playing the app. However, anxiety, disgust, and arousal ratings decreased from day to day. We discuss our findings with respect to implications for the clinical practice.

Costly avoidance in anxious individuals: Elevated threat avoidance in anxious individuals under high, but not low competing rewards

BACKGROUND AND OBJECTIVES

When avoiding threat conflicts with approaching rewards, balanced responses to threat and reward information is required to guide functional behavior. Elevated threat avoidance characterizes anxious psychopathology. However, little is known about the mutual impact of threat and reward information on approach-avoidance behavior and its link to anxiety.

METHODS

High trait-anxious and low-anxious individuals (N = 74) repeatedly choose between two options. A threat/high-reward option was linked to two outcomes: a varying chance to receive an aversive stimulus and a varying high reward. A safe/low-reward option was linked to absence of the aversive stimulus and a low reward.

RESULTS

Avoidance of the threat/high-reward option increased with increasing threat. Despite threat, low-anxious individuals increasingly approached the threat/high-reward option when rewards increased. High- compared to low-anxious individuals showed elevated avoidance, but only in the presence of high competing rewards.

LIMITATIONS

Future research should examine boundary conditions by manipulating type and motivational value of appetitive and aversive outcomes (e.g., food as primary reinforcer).

CONCLUSIONS

These findings suggest that a weaker impact of rewards competing with threat contributes to elevated threat avoidance in anxious psychopathology. Costly avoidance may thus be a factor involved in anxious psychopathology.

The Neural Basis of Approach-Avoidance Conflict: A Model Based Analysis

Approach-avoidance conflict arises when the drives to pursue reward and avoid harm are incompatible. Previous neuroimaging studies of approach-avoidance conflict have shown large variability in reported neuroanatomical correlates. These prior studies have generally neglected to account for potential sources of variability, such as individual differences in choice preferences and modeling of hemodynamic response during conflict. In the present study, we controlled for these limitations using a hierarchical Bayesian model (HBM). This enabled us to measure participant-specific per-trial estimates of conflict during an approach-avoidance task. We also employed a variable epoch method to identify brain structures specifically sensitive to conflict. In a sample of 28 human participants, we found that only a limited set of brain structures [inferior frontal gyrus (IFG), right dorsolateral prefrontal cortex (dlPFC), and right pre-supplementary motor area (pre-SMA)] are specifically correlated with approach-avoidance conflict. These findings suggest that controlling for previous sources of variability increases the specificity of the neuroanatomical correlates of approach-avoidance conflict.

Social and monetary incentives counteract fear-driven avoidance: Evidence from approach-avoidance decisions

BACKGROUND AND OBJECTIVES

The reduction of avoidance behavior is a central target in the treatment of anxiety disorders, but it has rarely been studied how approach of fear-relevant stimuli may be initiated. In two studies, the impact of hypothetical monetary and symbolic social incentives on approach-avoidance behavior was examined.

METHODS

In Study 1, individuals high or low on fear of spiders (N = 84) could choose to approach a fear-relevant versus a neutral stimulus, which were equally rewarded. In a subsequent micro-intervention, approaching the fear-relevant stimulus was differentially rewarded either by monetary or social incentives. In Study 2 (N = 76), initial incentives for approach were discontinued to investigate the stability of approach.

RESULTS

Hypothetical monetary and symbolic social incentives reduced or eliminated initial avoidance, even in highly fearful individuals. Approach resulted in a decrease of self-reported aversiveness towards the fear-relevant stimulus. However, even after successful approach, fearful individuals showed significant avoidance behavior when incentives for approach were discontinued.

LIMITATIONS

Future research should investigate the long-term effects of prolonged approach incentives on multiple levels of fear (e.g., self-report, behavioral, physiological). It should also be tested if such an intervention actually improves compliance with exposure based interventions.

CONCLUSIONS

The present findings highlight that incentives are useful to initiate initial approach towards a feared stimulus. Although incentive-based approach may neither fully eliminate avoidance nor negative feelings towards the feared stimulus, such operant interventions may set the stage for more extensive extinction training.