The computational structure of consummatory anhedonia

Anhedonia is a reduction in enjoyment, motivation, or interest. It is common across mental health disorders and a harbinger of poor treatment outcomes. The enjoyment aspect, termed 'consummatory anhedonia', in particular poses fundamental questions about how the brain constructs rewards: what processes determine how intensely a reward is experienced? Here, we outline limitations of existing computational conceptualisations of consummatory anhedonia. We then suggest a richer reinforcement learning (RL) account of consummatory anhedonia with a reconceptualisation of subjective hedonic experience in terms of goal progress. This accounts qualitatively for the impact of stress, dysfunctional cognitions, and maladaptive beliefs on hedonic experience. The model also offers new views on the treatments for anhedonia.

Challenges and Solutions to the Measurement of Neurocognitive Mechanisms in Developmental Settings

Identifying early neurocognitive mechanisms that confer risk for mental health problems is one important avenue as we seek to develop successful early interventions. Currently, however, we have limited understanding of the neurocognitive mechanisms involved in shaping mental health trajectories from childhood through young adulthood, and this constrains our ability to develop effective clinical interventions. In particular, there is an urgent need to develop more sensitive, reliable, and scalable measures of individual differences for use in developmental settings. In this review, we outline methodological shortcomings that explain why widely used task-based measures of neurocognition currently tell us little about mental health risk. We discuss specific challenges that arise when studying neurocognitive mechanisms in developmental settings, and we share suggestions for overcoming them. We also propose a novel experimental approach-which we refer to as "cognitive microscopy"-that involves adaptive design optimization, temporally sensitive task administration, and multilevel modeling. This approach addresses some of the methodological shortcomings outlined above and provides measures of stability, variability, and developmental change in neurocognitive mechanisms within a multivariate framework.

The Association of Non-Drug-Related Pavlovian-to-Instrumental Transfer Effect in Nucleus Accumbens With Relapse in Alcohol Dependence: A Replication

BACKGROUND

The Pavlovian-to-instrumental transfer (PIT) paradigm measures the effects of Pavlovian conditioned cues on instrumental behavior in the laboratory. A previous study conducted by our research group observed activity in the left nucleus accumbens (NAcc) elicited by a non-drug-related PIT task across patients with alcohol dependence (AD) and healthy control subjects, and the left NAcc PIT effect differentiated patients who subsequently relapsed from those who remained abstinent. In this study, we aimed to examine whether such effects were present in a larger sample collected at a later date.

METHODS

A total of 129 recently detoxified patients with AD (21 females) and 74 healthy, age- and gender-matched control subjects (12 females) performing a PIT task during functional magnetic resonance imaging were examined. After task assessments, patients were followed for 6 months. Forty-seven patients relapsed and 37 remained abstinent.

RESULTS

We found a significant behavioral non-drug-related PIT effect and PIT-related activity in the NAcc across all participants. Moreover, subsequent relapsers showed stronger behavioral and left NAcc PIT effects than abstainers. These findings are consistent with our previous findings.

CONCLUSIONS

Behavioral non-drug-related PIT and neural PIT correlates are associated with prospective relapse risk in AD. This study replicated previous findings and provides evidence for the clinical relevance of PIT mechanisms to treatment outcome in AD. The observed difference between prospective relapsers and abstainers in the NAcc PIT effect in our study is small overall. Future studies are needed to further elucidate the mechanisms and the possible modulators of neural PIT in relapse in AD.

Susceptibility to interference between Pavlovian and instrumental control predisposes risky alcohol use developmental trajectory from ages 18 to 24

Pavlovian cues can influence ongoing instrumental behaviour via Pavlovian-to-instrumental transfer (PIT) processes. While appetitive Pavlovian cues tend to promote instrumental approach, they are detrimental when avoidance behaviour is required, and vice versa for aversive cues. We recently reported that susceptibility to interference between Pavlovian and instrumental control assessed via a PIT task was associated with risky alcohol use at age 18. We now investigated whether such susceptibility also predicts drinking trajectories until age 24, based on AUDIT (Alcohol Use Disorders Identification Test) consumption and binge drinking (gramme alcohol/drinking occasion) scores. The interference PIT effect, assessed at ages 18 and 21 during fMRI, was characterized by increased error rates (ER) and enhanced neural responses in the ventral striatum (VS), the lateral and dorsomedial prefrontal cortices (dmPFC) during conflict, that is, when an instrumental approach was required in the presence of an aversive Pavlovian cue or vice versa. We found that a stronger VS response during conflict at age 18 was associated with a higher starting point of both drinking trajectories but predicted a decrease in binge drinking. At age 21, high ER and enhanced neural responses in the dmPFC were associated with increasing AUDIT-C scores over the next 3 years until age 24. Overall, susceptibility to interference between Pavlovian and instrumental control might be viewed as a predisposing mechanism towards hazardous alcohol use during young adulthood, and the identified high-risk group may profit from targeted interventions.

Psychopathic tendency in violent offenders is associated with reduced aversive Pavlovian inhibition of behavior and associated striatal BOLD signal

Background

Violent offenders with psychopathic tendencies are characterized by instrumental, i.e., planned, callous, and unemotional (aggressive) behavior and have been shown to exhibit abnormal aversive processing. However, the consequences of abnormal aversive processing for instrumental action and associated neural mechanisms are unclear.

Materials and methods

Here we address this issue by using event-related functional magnetic resonance imaging (fMRI) in 15 violent offenders with high psychopathic tendencies and 18 matched controls during the performance of an aversive Pavlovian-to-instrumental transfer paradigm. This paradigm allowed us to assess the degree to which aversive Pavlovian cues affect instrumental action and associated neural signaling.

Results

Psychopathic tendency scores were associated with an attenuation of aversive Pavlovian inhibition of instrumental action. Moreover, exploratory analyses revealed an anomalous positive association between aversive inhibition of action and aversive inhibition of BOLD signal in the caudate nucleus of violent offenders with psychopathic tendencies. In addition, psychopathic tendency also correlated positively with amygdala reactivity during aversive versus neutral cues in Pavlovian training.

Conclusion

These findings strengthen the hypothesis that psychopathic tendencies in violent offenders are related to abnormal impact of aversive processing on instrumental behavior. The neural effects raise the possibility that this reflects deficient transfer of aversive Pavlovian inhibitory biases onto neural systems that implement instrumental action, including the caudate nucleus.

Amygdala response predicts clinical symptom reduction in patients with borderline personality disorder: A pilot fMRI study

Borderline personality disorder (BPD) is a prevalent, devastating, and heterogeneous psychiatric disorder. Treatment success is highly variable within this patient group. A cognitive neuroscientific approach to BPD might contribute to precision psychiatry by identifying neurocognitive factors that predict who will benefit from a specific treatment. Here, we build on observations that BPD is accompanied by the enhanced impact of the aversive effect on behavior and abnormal neural signaling in the amygdala. We assessed whether BPD is accompanied by abnormal aversive regulation of instrumental behavior and associated neural signaling, in a manner that is predictive of symptom reduction after therapy. We tested a clinical sample of 15 female patients with BPD, awaiting dialectical behavior therapy (DBT), and 16 matched healthy controls using fMRI and an aversive Pavlovian-to-instrumental transfer (PIT) task that assesses how instrumental behaviors are influenced by aversive Pavlovian stimuli. Patients were assessed 1 year after the start of DBT to quantify changes in BPD symptom severity. At baseline, behavioral aversive PIT and associated neural signaling did not differ between groups. However, the BOLD signal in the amygdala measured during aversive PIT was associated with symptom reduction at 1-year follow-up: higher PIT-related aversive amygdala signaling before treatment was associated with reduced clinical improvement at follow-up. Thus, within the evaluated group of BPD patients, the BOLD signal in the amygdala before treatment was related to clinical symptom reduction 1 year after the start of treatment. The results suggest that less PIT-related responsiveness of the amygdala increases the chances of treatment success. We note that the relatively small sample size is a limitation of this study and that replication is warranted.

Stronger Prejudices Are Associated With Decreased Model-Based Control

Background: Prejudices against minorities can be understood as habitually negative evaluations that are kept in spite of evidence to the contrary. Therefore, individuals with strong prejudices might be dominated by habitual or "automatic" reactions at the expense of more controlled reactions. Computational theories suggest individual differences in the balance between habitual/model-free and deliberative/model-based decision-making. Methods: 127 subjects performed the two Step task and completed the blatant and subtle prejudice scale. Results: By using analyses of choices and reaction times in combination with computational modeling, subjects with stronger blatant prejudices showed a shift away from model-based control. There was no association between these decision-making processes and subtle prejudices. Conclusion: These results support the idea that blatant prejudices toward minorities are related to a relative dominance of habitual decision-making. This finding has important implications for developing interventions that target to change prejudices across societies.

Explaining distortions in metacognition with an attractor network model of decision uncertainty

Metacognition is the ability to reflect on, and evaluate, our cognition and behaviour. Distortions in metacognition are common in mental health disorders, though the neural underpinnings of such dysfunction are unknown. One reason for this is that models of key components of metacognition, such as decision confidence, are generally specified at an algorithmic or process level. While such models can be used to relate brain function to psychopathology, they are difficult to map to a neurobiological mechanism. Here, we develop a biologically-plausible model of decision uncertainty in an attempt to bridge this gap. We first relate the model's uncertainty in perceptual decisions to standard metrics of metacognition, namely mean confidence level (bias) and the accuracy of metacognitive judgments (sensitivity). We show that dissociable shifts in metacognition are associated with isolated disturbances at higher-order levels of a circuit associated with self-monitoring, akin to neuropsychological findings that highlight the detrimental effect of prefrontal brain lesions on metacognitive performance. Notably, we are able to account for empirical confidence judgements by fitting the parameters of our biophysical model to first-order performance data, specifically choice and response times. Lastly, in a reanalysis of existing data we show that self-reported mental health symptoms relate to disturbances in an uncertainty-monitoring component of the network. By bridging a gap between a biologically-plausible model of confidence formation and observed disturbances of metacognition in mental health disorders we provide a first step towards mapping theoretical constructs of metacognition onto dynamical models of decision uncertainty. In doing so, we provide a computational framework for modelling metacognitive performance in settings where access to explicit confidence reports is not possible.

Model-Based and Model-Free Control Predicts Alcohol Consumption Developmental Trajectory in Young Adults: A 3-Year Prospective Study

BACKGROUND

A shift from goal-directed toward habitual control has been associated with alcohol dependence. Whether such a shift predisposes to risky drinking is not yet clear. We investigated how goal-directed and habitual control at age 18 predict alcohol use trajectories over the course of 3 years.

METHODS

Goal-directed and habitual control, as informed by model-based (MB) and model-free (MF) learning, were assessed with a two-step sequential decision-making task during functional magnetic resonance imaging in 146 healthy 18-year-old men. Three-year alcohol use developmental trajectories were based on either a consumption score from the self-reported Alcohol Use Disorders Identification Test (assessed every 6 months) or an interview-based binge drinking score (grams of alcohol/occasion; assessed every year). We applied a latent growth curve model to examine how MB and MF control predicted the drinking trajectory.

RESULTS

Drinking behavior was best characterized by a linear trajectory. MB behavioral control was negatively associated with the development of the binge drinking score; MF reward prediction error blood oxygen level-dependent signals in the ventromedial prefrontal cortex and the ventral striatum predicted a higher starting point and steeper increase of the Alcohol Use Disorders Identification Test consumption score over time, respectively.

CONCLUSIONS

We found that MB behavioral control was associated with the binge drinking trajectory, while the MF reward prediction error signal was closely linked to the consumption score development. These findings support the idea that unbalanced MB and MF control might be an important individual vulnerability in predisposing to risky drinking behavior.

How Representative are Neuroimaging Samples? Large-Scale Evidence for Trait Anxiety Differences Between fMRI and Behaviour-Only Research Participants

Over the past three decades, functional magnetic resonance imaging (fMRI) has become crucial to study how cognitive processes are implemented in the human brain. However, the question of whether participants recruited into fMRI studies differ from participants recruited into other study contexts has received little to no attention. This is particularly pertinent when effects fail to generalize across study contexts: for example, a behavioural effect discovered in a non-imaging context not replicating in a neuroimaging environment. Here, we tested the hypothesis, motivated by preliminary findings (N = 272), that fMRI participants differ from behaviour-only participants on one fundamental individual difference variable: trait anxiety. Analysing trait anxiety scores and possible confounding variables from healthy volunteers across multiple institutions (N = 3317), we found robust support for lower trait anxiety in fMRI study participants, consistent with a sampling or self-selection bias. The bias was larger in studies that relied on phone screening (compared with full in-person psychiatric screening), recruited at least partly from convenience samples (compared with community samples), and in pharmacology studies. Our findings highlight the need for surveying trait anxiety at recruitment and for appropriate screening procedures or sampling strategies to mitigate this bias.

Abnormal reward valuation and event-related connectivity in unmedicated major depressive disorder

BACKGROUND

Experience of emotion is closely linked to valuation. Mood can be viewed as a bias to experience positive or negative emotions and abnormally biased subjective reward valuation and cognitions are core characteristics of major depression.

METHODS

Thirty-four unmedicated subjects with major depressive disorder and controls estimated the probability that fractal stimuli were associated with reward, based on passive observations, so they could subsequently choose the higher of either their estimated fractal value or an explicitly presented reward probability. Using model-based functional magnetic resonance imaging, we estimated each subject's internal value estimation, with psychophysiological interaction analysis used to examine event-related connectivity, testing hypotheses of abnormal reward valuation and cingulate connectivity in depression.

RESULTS

Reward value encoding in the hippocampus and rostral anterior cingulate was abnormal in depression. In addition, abnormal decision-making in depression was associated with increased anterior mid-cingulate activity and a signal in this region encoded the difference between the values of the two options. This localised decision-making and its impairment to the anterior mid-cingulate cortex (aMCC) consistent with theories of cognitive control. Notably, subjects with depression had significantly decreased event-related connectivity between the aMCC and rostral cingulate regions during decision-making, implying impaired communication between the neural substrates of expected value estimation and decision-making in depression.

CONCLUSIONS

Our findings support the theory that abnormal neural reward valuation plays a central role in major depressive disorder (MDD). To the extent that emotion reflects valuation, abnormal valuation could explain abnormal emotional experience in MDD, reflect a core pathophysiological process and be a target of treatment.

A comparison of 'pruning' during multi-step planning in depressed and healthy individuals

BACKGROUND

Real-life decisions are often complex because they involve making sequential choices that constrain future options. We have previously shown that to render such multi-step decisions manageable, people 'prune' (i.e. selectively disregard) branches of decision trees that contain negative outcomes. We have theorized that sub-optimal pruning contributes to depression by promoting an oversampling of branches that result in unsavoury outcomes, which results in a negatively-biased valuation of the world. However, no study has tested this theory in depressed individuals.

METHODS

Thirty unmedicated depressed and 31 healthy participants were administered a sequential reinforcement-based decision-making task to determine pruning behaviours, and completed measures of depression and anxiety. Computational, Bayesian and frequentist analyses examined group differences in task performance and relationships between pruning and depressive symptoms.

RESULTS

Consistent with prior findings, participants robustly pruned branches of decision trees that began with large losses, regardless of the potential utility of those branches. However, there was no group difference in pruning behaviours. Further, there was no relationship between pruning and levels of depression/anxiety.

CONCLUSIONS

We found no evidence that sub-optimal pruning is evident in depression. Future research could determine whether maladaptive pruning behaviours are observable in specific sub-groups of depressed patients (e.g. in treatment-resistant individuals), or whether misuse of other heuristics may contribute to depression.

Advances in the computational understanding of mental illness

Computational psychiatry is a rapidly growing field attempting to translate advances in computational neuroscience and machine learning into improved outcomes for patients suffering from mental illness. It encompasses both data-driven and theory-driven efforts. Here, recent advances in theory-driven work are reviewed. We argue that the brain is a computational organ. As such, an understanding of the illnesses arising from it will require a computational framework. The review divides work up into three theoretical approaches that have deep mathematical connections: dynamical systems, Bayesian inference and reinforcement learning. We discuss both general and specific challenges for the field, and suggest ways forward.

The relationship between resting-state functional connectivity, antidepressant discontinuation and depression relapse

The risk of relapsing into depression after stopping antidepressants is high, but no established predictors exist. Resting-state functional magnetic resonance imaging (rsfMRI) measures may help predict relapse and identify the mechanisms by which relapses occur. rsfMRI data were acquired from healthy controls and from patients with remitted major depressive disorder on antidepressants. Patients were assessed a second time either before or after discontinuation of the antidepressant, and followed up for six months to assess relapse. A seed-based functional connectivity analysis was conducted focusing on the left subgenual anterior cingulate cortex and left posterior cingulate cortex. Seeds in the amygdala and dorsolateral prefrontal cortex were explored. 44 healthy controls (age: 33.8 (10.5), 73% female) and 84 patients (age: 34.23 (10.8), 80% female) were included in the analysis. 29 patients went on to relapse and 38 remained well. The seed-based analysis showed that discontinuation resulted in an increased functional connectivity between the right dorsolateral prefrontal cortex and the parietal cortex in non-relapsers. In an exploratory analysis, this functional connectivity predicted relapse risk with a balanced accuracy of 0.86. Further seed-based analyses, however, failed to reveal differences in functional connectivity between patients and controls, between relapsers and non-relapsers before discontinuation and changes due to discontinuation independent of relapse. In conclusion, changes in the connectivity between the dorsolateral prefrontal cortex and the posterior default mode network were associated with and predictive of relapse after open-label antidepressant discontinuation. This finding requires replication in a larger dataset.

Computational Mechanisms of Effort and Reward Decisions in Patients With Depression and Their Association With Relapse After Antidepressant Discontinuation

IMPORTANCE

Nearly 1 in 3 patients with major depressive disorder who respond to antidepressants relapse within 6 months of treatment discontinuation. No predictors of relapse exist to guide clinical decision-making in this scenario.

OBJECTIVES

To establish whether the decision to invest effort for rewards represents a persistent depression process after remission, predicts relapse after remission, and is affected by antidepressant discontinuation.

DESIGN, SETTING, AND PARTICIPANTS

This longitudinal randomized observational prognostic study in a Swiss and German university setting collected data from July 1, 2015, to January 31, 2019, from 66 healthy controls and 123 patients in remission from major depressive disorder in response to antidepressants prior to and after discontinuation. Study recruitment took place until January 2018.

EXPOSURE

Discontinuation of antidepressants.

MAIN OUTCOMES AND MEASURES

Relapse during the 6 months after discontinuation. Choice and decision times on a task requiring participants to choose how much effort to exert for various amounts of reward and the mechanisms identified through parameters of a computational model.

RESULTS

A total of 123 patients (mean [SD] age, 34.5 [11.2] years; 94 women [76%]) and 66 healthy controls (mean [SD] age, 34.6 [11.0] years; 49 women [74%]) were recruited. In the main subsample, mean (SD) decision times were slower for patients (n = 74) compared with controls (n = 34) (1.77 [0.38] seconds vs 1.61 [0.37] seconds; Cohen d = 0.52; P = .02), particularly for those who later relapsed after discontinuation of antidepressants (n = 21) compared with those who did not relapse (n = 39) (1.95 [0.40] seconds vs 1.67 [0.34] seconds; Cohen d = 0.77; P < .001). This slower decision time predicted relapse (accuracy = 0.66; P = .007). Patients invested less effort than healthy controls for rewards (F1,98 = 33.970; P < .001). Computational modeling identified a mean (SD) deviation from standard drift-diffusion models that was more prominent for patients than controls (patients, 0.67 [1.56]; controls, -0.71 [1.93]; Cohen d = 0.82; P < .001). Patients also showed higher mean (SD) effort sensitivity than controls (patients, 0.31 [0.92]; controls, -0.08 [1.03]; Cohen d = 0.51; P = .05). Relapsers differed from nonrelapsers in terms of the evidence required to make a decision for the low-effort choice (mean [SD]: relapsers, 1.36 [0.35]; nonrelapsers, 1.17 [0.26]; Cohen d = 0.65; P = .02). Group differences generally did not reach significance in the smaller replication sample (27 patients and 21 controls), but decision time prediction models from the main sample generalized to the replication sample (validation accuracy = 0.71; P = .03).

CONCLUSIONS AND RELEVANCE

This study found that the decision to invest effort was associated with prospective relapse risk after antidepressant discontinuation and may represent a persistent disease process in asymptomatic remitted major depressive disorder. Markers based on effort-related decision-making could potentially inform clinical decisions associated with antidepressant discontinuation.

Neural correlates of instrumental responding in the context of alcohol-related cues index disorder severity and relapse risk

The influence of Pavlovian conditioned stimuli on ongoing behavior may contribute to explaining how alcohol cues stimulate drug seeking and intake. Using a Pavlovian-instrumental transfer task, we investigated the effects of alcohol-related cues on approach behavior (i.e., instrumental response behavior) and its neural correlates, and related both to the relapse after detoxification in alcohol-dependent patients. Thirty-one recently detoxified alcohol-dependent patients and 24 healthy controls underwent instrumental training, where approach or non-approach towards initially neutral stimuli was reinforced by monetary incentives. Approach behavior was tested during extinction with either alcohol-related or neutral stimuli (as Pavlovian cues) presented in the background during functional magnetic resonance imaging (fMRI). Patients were subsequently followed up for 6 months. We observed that alcohol-related background stimuli inhibited the approach behavior in detoxified alcohol-dependent patients (t = - 3.86, p < .001), but not in healthy controls (t = - 0.92, p = .36). This behavioral inhibition was associated with neural activation in the nucleus accumbens (NAcc) (t₍₃₀₎ = 2.06, p < .05). Interestingly, both the effects were only present in subsequent abstainers, but not relapsers and in those with mild but not severe dependence. Our data show that alcohol-related cues can acquire inhibitory behavioral features typical of aversive stimuli despite being accompanied by a stronger NAcc activation, suggesting salience attribution. The fact that these findings are restricted to abstinence and milder illness suggests that they may be potential resilience factors.Clinical trial: LeAD study, http://www.lead-studie.de , NCT01679145.

Self-regulation of the dopaminergic reward circuit in cocaine users with mental imagery and neurofeedback

BACKGROUND

Enhanced drug-related reward sensitivity accompanied by impaired sensitivity to non-drug related rewards in the mesolimbic dopamine system are thought to underlie the broad motivational deficits and dysfunctional decision-making frequently observed in cocaine use disorder (CUD). Effective approaches to modify this imbalance and reinstate non-drug reward responsiveness are urgently needed. Here, we examined whether cocaine users (CU) can use mental imagery of non-drug rewards to self-regulate the ventral tegmental area and substantia nigra (VTA/SN). We expected that obsessive and compulsive thoughts about cocaine consumption would hamper the ability to self-regulate the VTA/SN activity and tested if real-time fMRI (rtfMRI) neurofeedback (NFB) can improve self-regulation of the VTA/SN.

METHODS

Twenty-two CU and 28 healthy controls (HC) were asked to voluntarily up-regulate VTA/SN activity with non-drug reward imagery alone, or combined with rtfMRI NFB.

RESULTS

On a group level, HC and CU were able to activate the dopaminergic midbrain and other reward regions with reward imagery. In CU, the individual ability to self-regulate the VTA/SN was reduced in those with more severe obsessive-compulsive drug use. NFB enhanced the effect of reward imagery but did not result in transfer effects at the end of the session.

CONCLUSION

CU can voluntary activate their reward system with non-drug reward imagery and improve this ability with rtfMRI NFB. Combining mental imagery and rtFMRI NFB has great potential for modifying the maladapted reward sensitivity and reinstating non-drug reward responsiveness. This motivates further work to examine the use of rtfMRI NFB in the treatment of CUD.

Major Depression Impairs the Use of Reward Values for Decision-Making

Depression is a debilitating condition with a high prevalence. Depressed patients have been shown to be diminished in their ability to integrate their reinforcement history to adjust future behaviour during instrumental reward learning tasks. Here, we tested whether such impairments could also be observed in a Pavlovian conditioning task. We recruited and analysed 32 subjects, 15 with depression and 17 healthy controls, to study behavioural group differences in learning and decision-making. Participants had to estimate the probability of some fractal stimuli to be associated with a binary reward, based on a few passive observations. They then had to make a choice between one of the observed fractals and another target for which the reward probability was explicitly given. Computational modelling was used to succinctly describe participants' behaviour. Patients performed worse than controls at the task. Computational modelling revealed that this was caused by behavioural impairments during both learning and decision phases. Depressed subjects showed lower memory of observed rewards and had an impaired ability to use internal value estimations to guide decision-making in our task.

Deficits in context-dependent adaptive coding in early psychosis and healthy individuals with schizotypal personality traits

Adaptive coding of information is a fundamental principle of brain functioning. It allows for efficient representation over a large range of inputs and thereby alleviates the limited coding range of neurons. In the present study, we investigated for the first time potential alterations in context-dependent reward adaptation and its association with symptom dimensions in the schizophrenia spectrum. We studied 27 patients with first-episode psychosis, 26 individuals with schizotypal personality traits and 25 healthy controls. We used functional MRI in combination with a variant of the monetary incentive delay task and assessed adaptive reward coding in two reward conditions with different reward ranges. Compared to healthy controls, patients with first-episode psychosis and healthy individuals with schizotypal personality traits showed a deficit in increasing the blood oxygen level-dependent response slope in the right caudate for the low reward range compared to the high reward range. In other words, the two groups showed inefficient neural adaptation to the current reward context. In addition, we found impaired adaptive coding of reward in the caudate nucleus and putamen to be associated with total symptom severity across the schizophrenia spectrum. Symptom severity was more strongly associated with neural deficits in adaptive coding than with the neural coding of absolute reward outcomes. Deficits in adaptive coding were prominent across the schizophrenia spectrum and even detectable in unmedicated (healthy) individuals with schizotypal personality traits. Furthermore, the association between total symptom severity and impaired adaptive coding in the right caudate and putamen suggests a dimensional mechanism underlying imprecise neural adaptation. Our findings support the idea that impaired adaptive coding may be a general information-processing deficit explaining disturbances within the schizophrenia spectrum over and above a simple model of blunted absolute reward signals.

Depression is associated with enhanced aversive Pavlovian control over instrumental behaviour

The dynamic modulation of instrumental behaviour by conditioned Pavlovian cues is an important process in decision-making. Patients with major depressive disorder (MDD) are known to exhibit mood-congruent biases in information processing, which may occur due to Pavlovian influences, but this hypothesis has never been tested directly in an unmedicated sample. To address this we tested unmedicated MDD patients and healthy volunteers on a computerized Pavlovian-Instrumental Transfer (PIT) task designed to separately examine instrumental approach and withdrawal actions in the context of Pavlovian appetitive and aversive cues. This design allowed us to directly measure the degree to which Pavlovian cues influence instrumental responding. Depressed patients were profoundly influenced by aversive Pavlovian stimuli, to a significantly greater degree than healthy volunteers. This was the case for instrumental behaviour both in the approach condition (in which aversive Pavlovian cues inhibited ‘go’ responses), and in the withdrawal condition (in which aversive Pavlovian cues facilitated ‘go’ responses). Exaggerated aversive PIT provides a potential cognitive mechanism for biased emotion processing in major depression. This finding also has wider significance for the understanding of disrupted motivational processing in neuropsychiatric disorders.

When Habits Are Dangerous: Alcohol Expectancies and Habitual Decision Making Predict Relapse in Alcohol Dependence

BACKGROUND

Addiction is supposedly characterized by a shift from goal-directed to habitual decision making, thus facilitating automatic drug intake. The two-step task allows distinguishing between these mechanisms by computationally modeling goal-directed and habitual behavior as model-based and model-free control. In addicted patients, decision making may also strongly depend upon drug-associated expectations. Therefore, we investigated model-based versus model-free decision making and its neural correlates as well as alcohol expectancies in alcohol-dependent patients and healthy controls and assessed treatment outcome in patients.

METHODS

Ninety detoxified, medication-free, alcohol-dependent patients and 96 age- and gender-matched control subjects underwent functional magnetic resonance imaging during the two-step task. Alcohol expectancies were measured with the Alcohol Expectancy Questionnaire. Over a follow-up period of 48 weeks, 37 patients remained abstinent and 53 patients relapsed as indicated by the Alcohol Timeline Followback method.

RESULTS

Patients who relapsed displayed reduced medial prefrontal cortex activation during model-based decision making. Furthermore, high alcohol expectancies were associated with low model-based control in relapsers, while the opposite was observed in abstainers and healthy control subjects. However, reduced model-based control per se was not associated with subsequent relapse.

CONCLUSIONS

These findings suggest that poor treatment outcome in alcohol dependence does not simply result from a shift from model-based to model-free control but is instead dependent on the interaction between high drug expectancies and low model-based decision making. Reduced model-based medial prefrontal cortex signatures in those who relapse point to a neural correlate of relapse risk. These observations suggest that therapeutic interventions should target subjective alcohol expectancies.

A Formal Valuation Framework for Emotions and Their Control

Computational psychiatry aims to apply mathematical and computational techniques to help improve psychiatric care. To achieve this, the phenomena under scrutiny should be within the scope of formal methods. As emotions play an important role across many psychiatric disorders, such computational methods must encompass emotions. Here, we consider formal valuation accounts of emotions. We focus on the fact that the flexibility of emotional responses and the nature of appraisals suggest the need for a model-based valuation framework for emotions. However, resource limitations make plain model-based valuation impossible and require metareasoning strategies to apportion cognitive resources adaptively. We argue that emotions may implement such metareasoning approximations by restricting the range of behaviors and states considered. We consider the processes that guide the deployment of the approximations, discerning between innate, model-free, heuristic, and model-based controllers. A formal valuation and metareasoning framework may thus provide a principled approach to examining emotions.

A Roadmap for the Development of Applied Computational Psychiatry

BACKGROUND

Computational psychiatry is a burgeoning field that utilizes mathematical approaches to investigate psychiatric disorders, derive quantitative predictions, and integrate data across multiple levels of description. Computational psychiatry has already led to many new insights into the neurobehavioral mechanisms that underlie several psychiatric disorders, but its usefulness from a clinical standpoint is only now starting to be considered.

METHODS

Examples of computational psychiatry are highlighted, and a phase-based pipeline for the development of clinical computational-psychiatry applications is proposed, similar to the phase-based pipeline used in drug development. It is proposed that each phase has unique endpoints and deliverables, which will be important milestones to move tasks, procedures, computational models, and algorithms from the laboratory to clinical practice.

RESULTS

Application of computational approaches should be tested on healthy volunteers in Phase I, transitioned to target populations in Phase IB and Phase IIA, and thoroughly evaluated using randomized clinical trials in Phase IIB and Phase III. Successful completion of these phases should be the basis of determining whether computational models are useful tools for prognosis, diagnosis, or treatment of psychiatric patients.

CONCLUSIONS

A new type of infrastructure will be necessary to implement the proposed pipeline. This infrastructure should consist of groups of investigators with diverse backgrounds collaborating to make computational psychiatry relevant for the clinic.

Computational psychiatry as a bridge from neuroscience to clinical applications

Translating advances in neuroscience into benefits for patients with mental illness presents enormous challenges because it involves both the most complex organ, the brain, and its interaction with a similarly complex environment. Dealing with such complexities demands powerful techniques. Computational psychiatry combines multiple levels and types of computation with multiple types of data in an effort to improve understanding, prediction and treatment of mental illness. Computational psychiatry, broadly defined, encompasses two complementary approaches: data driven and theory driven. Data-driven approaches apply machine-learning methods to high-dimensional data to improve classification of disease, predict treatment outcomes or improve treatment selection. These approaches are generally agnostic as to the underlying mechanisms. Theory-driven approaches, in contrast, use models that instantiate prior knowledge of, or explicit hypotheses about, such mechanisms, possibly at multiple levels of analysis and abstraction. We review recent advances in both approaches, with an emphasis on clinical applications, and highlight the utility of combining them.

Computational neuroimaging strategies for single patient predictions

Neuroimaging increasingly exploits machine learning techniques in an attempt to achieve clinically relevant single-subject predictions. An alternative to machine learning, which tries to establish predictive links between features of the observed data and clinical variables, is the deployment of computational models for inferring on the (patho)physiological and cognitive mechanisms that generate behavioural and neuroimaging responses. This paper discusses the rationale behind a computational approach to neuroimaging-based single-subject inference, focusing on its potential for characterising disease mechanisms in individual subjects and mapping these characterisations to clinical predictions. Following an overview of two main approaches - Bayesian model selection and generative embedding - which can link computational models to individual predictions, we review how these methods accommodate heterogeneity in psychiatric and neurological spectrum disorders, help avoid erroneous interpretations of neuroimaging data, and establish a link between a mechanistic, model-based approach and the statistical perspectives afforded by machine learning.

Pavlovian-to-instrumental transfer effects in the nucleus accumbens relate to relapse in alcohol dependence

In detoxified alcohol-dependent patients, alcohol-related stimuli can promote relapse. However, to date, the mechanisms by which contextual stimuli promote relapse have not been elucidated in detail. One hypothesis is that such contextual stimuli directly stimulate the motivation to drink via associated brain regions like the ventral striatum and thus promote alcohol seeking, intake and relapse. Pavlovian-to-Instrumental-Transfer (PIT) may be one of those behavioral phenomena contributing to relapse, capturing how Pavlovian conditioned (contextual) cues determine instrumental behavior (e.g. alcohol seeking and intake). We used a PIT paradigm during functional magnetic resonance imaging to examine the effects of classically conditioned Pavlovian stimuli on instrumental choices in n = 31 detoxified patients diagnosed with alcohol dependence and n = 24 healthy controls matched for age and gender. Patients were followed up over a period of 3 months. We observed that (1) there was a significant behavioral PIT effect for all participants, which was significantly more pronounced in alcohol-dependent patients; (2) PIT was significantly associated with blood oxygen level-dependent (BOLD) signals in the nucleus accumbens (NAcc) in subsequent relapsers only; and (3) PIT-related NAcc activation was associated with, and predictive of, critical outcomes (amount of alcohol intake and relapse during a 3 months follow-up period) in alcohol-dependent patients. These observations show for the first time that PIT-related BOLD signals, as a measure of the influence of Pavlovian cues on instrumental behavior, predict alcohol intake and relapse in alcohol dependence.

Don't Think, Just Feel the Music: Individuals with Strong Pavlovian-to-Instrumental Transfer Effects Rely Less on Model-based Reinforcement Learning

Behavioral choice can be characterized along two axes. One axis distinguishes reflexive, model-free systems that slowly accumulate values through experience and a model-based system that uses knowledge to reason prospectively. The second axis distinguishes Pavlovian valuation of stimuli from instrumental valuation of actions or stimulus-action pairs. This results in four values and many possible interactions between them, with important consequences for accounts of individual variation. We here explored whether individual variation along one axis was related to individual variation along the other. Specifically, we asked whether individuals' balance between model-based and model-free learning was related to their tendency to show Pavlovian interferences with instrumental decisions. In two independent samples with a total of 243 participants, Pavlovian-instrumental transfer effects were negatively correlated with the strength of model-based reasoning in a two-step task. This suggests a potential common underlying substrate predisposing individuals to both have strong Pavlovian interference and be less model-based and provides a framework within which to interpret the observation of both effects in addiction.

Charting the landscape of priority problems in psychiatry, part 1: classification and diagnosis

Contemporary psychiatry faces major challenges. Its syndrome-based disease classification is not based on mechanisms and does not guide treatment, which largely depends on trial and error. The development of therapies is hindered by ignorance of potential beneficiary patient subgroups. Neuroscientific and genetics research have yet to affect disease definitions or contribute to clinical decision making. In this challenging setting, what should psychiatric research focus on? In two companion papers, we present a list of problems nominated by clinicians and researchers from different disciplines as candidates for future scientific investigation of mental disorders. These problems are loosely grouped into challenges concerning nosology and diagnosis (this Personal View) and problems related to pathogenesis and aetiology (in the companion Personal View). Motivated by successful examples in other disciplines, particularly the list of Hilbert's problems in mathematics, this subjective and eclectic list of priority problems is intended for psychiatric researchers, helping to re-focus existing research and providing perspectives for future psychiatric science.

Charting the landscape of priority problems in psychiatry, part 2: pathogenesis and aetiology

This is the second of two companion papers proposing priority problems for research on mental disorders. Whereas the first paper focuses on questions of nosology and diagnosis, this Personal View concerns pathogenesis and aetiology of psychiatric diseases. We hope that this (non-exhaustive and subjective) list of problems, nominated by scientists and clinicians from different fields and institutions, provides guidance and perspectives for choosing future directions in psychiatric science.

Computational Psychiatry: towards a mathematically informed understanding of mental illness

Computational Psychiatry aims to describe the relationship between the brain's neurobiology, its environment and mental symptoms in computational terms. In so doing, it may improve psychiatric classification and the diagnosis and treatment of mental illness. It can unite many levels of description in a mechanistic and rigorous fashion, while avoiding biological reductionism and artificial categorisation. We describe how computational models of cognition can infer the current state of the environment and weigh up future actions, and how these models provide new perspectives on two example disorders, depression and schizophrenia. Reinforcement learning describes how the brain can choose and value courses of actions according to their long-term future value. Some depressive symptoms may result from aberrant valuations, which could arise from prior beliefs about the loss of agency ('helplessness'), or from an inability to inhibit the mental exploration of aversive events. Predictive coding explains how the brain might perform Bayesian inference about the state of its environment by combining sensory data with prior beliefs, each weighted according to their certainty (or precision). Several cortical abnormalities in schizophrenia might reduce precision at higher levels of the inferential hierarchy, biasing inference towards sensory data and away from prior beliefs. We discuss whether striatal hyperdopaminergia might have an adaptive function in this context, and also how reinforcement learning and incentive salience models may shed light on the disorder. Finally, we review some of Computational Psychiatry's applications to neurological disorders, such as Parkinson's disease, and some pitfalls to avoid when applying its methods.

Model-based and model-free decisions in alcohol dependence

BACKGROUND

Human and animal work suggests a shift from goal-directed to habitual decision-making in addiction. However, the evidence for this in human alcohol dependence is as yet inconclusive.

METHODS

Twenty-six healthy controls and 26 recently detoxified alcohol-dependent patients underwent behavioral testing with a 2-step task designed to disentangle goal-directed and habitual response patterns.

RESULTS

Alcohol-dependent patients showed less evidence of goal-directed choices than healthy controls, particularly after losses. There was no difference in the strength of the habitual component. The group differences did not survive controlling for performance on the Digit Symbol Substitution Task.

CONCLUSION

Chronic alcohol use appears to selectively impair goal-directed function, rather than promoting habitual responding. It appears to do so particularly after nonrewards, and this may be mediated by the effects of alcohol on more general cognitive functions subserved by the prefrontal cortex.

Pavlovian-to-instrumental transfer in alcohol dependence: a pilot study

BACKGROUND

Pavlovian processes are thought to play an important role in the development, maintenance and relapse of alcohol dependence, possibly by influencing and usurping ongoing thought and behavior. The influence of pavlovian stimuli on ongoing behavior is paradigmatically measured by pavlovian-to-instrumental transfer (PIT) tasks. These involve multiple stages and are complex. Whether increased PIT is involved in human alcohol dependence is uncertain. We therefore aimed to establish and validate a modified PIT paradigm that would be robust, consistent and tolerated by healthy controls as well as by patients suffering from alcohol dependence, and to explore whether alcohol dependence is associated with enhanced PIT.

METHODS

Thirty-two recently detoxified alcohol-dependent patients and 32 age- and gender-matched healthy controls performed a PIT task with instrumental go/no-go approach behaviors. The task involved both pavlovian stimuli associated with monetary rewards and losses, and images of drinks.

RESULTS

Both patients and healthy controls showed a robust and temporally stable PIT effect. Strengths of PIT effects to drug-related and monetary conditioned stimuli were highly correlated. Patients more frequently showed a PIT effect, and the effect was stronger in response to aversively conditioned CSs (conditioned suppression), but there was no group difference in response to appetitive CSs.

CONCLUSION

The implementation of PIT has favorably robust properties in chronic alcohol-dependent patients and in healthy controls. It shows internal consistency between monetary and drug-related cues. The findings support an association of alcohol dependence with an increased propensity towards PIT.

Depression: a decision-theoretic analysis

The manifold symptoms of depression are common and often transient features of healthy life that are likely to be adaptive in difficult circumstances. It is when these symptoms enter a seemingly self-propelling spiral that the maladaptive features of a disorder emerge. We examine this malignant transformation from the perspective of the computational neuroscience of decision making, investigating how dysfunction of the brain's mechanisms of evaluation might lie at its heart. We start by considering the behavioral implications of pessimistic evaluations of decision variables. We then provide a selective review of work suggesting how such pessimism might arise via specific failures of the mechanisms of evaluation or state estimation. Finally, we analyze ways that miscalibration between the subject and environment may be self-perpetuating. We employ the formal framework of Bayesian decision theory as a foundation for this study, showing how most of the problems arise from one of its broad algorithmic facets, namely model-based reasoning.

Chronic alcohol intake abolishes the relationship between dopamine synthesis capacity and learning signals in the ventral striatum

Drugs of abuse elicit dopamine release in the ventral striatum, possibly biasing dopamine-driven reinforcement learning towards drug-related reward at the expense of non-drug-related reward. Indeed, in alcohol-dependent patients, reactivity in dopaminergic target areas is shifted from non-drug-related stimuli towards drug-related stimuli. Such 'hijacked' dopamine signals may impair flexible learning from non-drug-related rewards, and thus promote craving for the drug of abuse. Here, we used functional magnetic resonance imaging to measure ventral striatal activation by reward prediction errors (RPEs) during a probabilistic reversal learning task in recently detoxified alcohol-dependent patients and healthy controls (N = 27). All participants also underwent 6-[(18) F]fluoro-DOPA positron emission tomography to assess ventral striatal dopamine synthesis capacity. Neither ventral striatal activation by RPEs nor striatal dopamine synthesis capacity differed between groups. However, ventral striatal coding of RPEs correlated inversely with craving in patients. Furthermore, we found a negative correlation between ventral striatal coding of RPEs and dopamine synthesis capacity in healthy controls, but not in alcohol-dependent patients. Moderator analyses showed that the magnitude of the association between dopamine synthesis capacity and RPE coding depended on the amount of chronic, habitual alcohol intake. Despite the relatively small sample size, a power analysis supports the reported results. Using a multimodal imaging approach, this study suggests that dopaminergic modulation of neural learning signals is disrupted in alcohol dependence in proportion to long-term alcohol intake of patients. Alcohol intake may perpetuate itself by interfering with dopaminergic modulation of neural learning signals in the ventral striatum, thus increasing craving for habitual drug intake.

Processing speed enhances model-based over model-free reinforcement learning in the presence of high working memory functioning

Theories of decision-making and its neural substrates have long assumed the existence of two distinct and competing valuation systems, variously described as goal-directed vs. habitual, or, more recently and based on statistical arguments, as model-free vs. model-based reinforcement-learning. Though both have been shown to control choices, the cognitive abilities associated with these systems are under ongoing investigation. Here we examine the link to cognitive abilities, and find that individual differences in processing speed covary with a shift from model-free to model-based choice control in the presence of above-average working memory function. This suggests shared cognitive and neural processes; provides a bridge between literatures on intelligence and valuation; and may guide the development of process models of different valuation components. Furthermore, it provides a rationale for individual differences in the tendency to deploy valuation systems, which may be important for understanding the manifold neuropsychiatric diseases associated with malfunctions of valuation.

Individual differences in bodily freezing predict emotional biases in decision making

Instrumental decision making has long been argued to be vulnerable to emotional responses. Literature on multiple decision making systems suggests that this emotional biasing might reflect effects of a system that regulates innately specified, evolutionarily preprogrammed responses. To test this hypothesis directly, we investigated whether effects of emotional faces on instrumental action can be predicted by effects of emotional faces on bodily freezing, an innately specified response to aversive relative to appetitive cues. We tested 43 women using a novel emotional decision making task combined with posturography, which involves a force platform to detect small oscillations of the body to accurately quantify postural control in upright stance. On the platform, participants learned whole body approach-avoidance actions based on monetary feedback, while being primed by emotional faces (angry/happy). Our data evidence an emotional biasing of instrumental action. Thus, angry relative to happy faces slowed instrumental approach relative to avoidance responses. Critically, individual differences in this emotional biasing effect were predicted by individual differences in bodily freezing. This result suggests that emotional biasing of instrumental action involves interaction with a system that controls innately specified responses. Furthermore, our findings help bridge (animal and human) decision making and emotion research to advance our mechanistic understanding of decision making anomalies in daily encounters as well as in a wide range of psychopathology.

Optimism as a prior belief about the probability of future reward

Optimists hold positive a priori beliefs about the future. In Bayesian statistical theory, a priori beliefs can be overcome by experience. However, optimistic beliefs can at times appear surprisingly resistant to evidence, suggesting that optimism might also influence how new information is selected and learned. Here, we use a novel Pavlovian conditioning task, embedded in a normative framework, to directly assess how trait optimism, as classically measured using self-report questionnaires, influences choices between visual targets, by learning about their association with reward progresses. We find that trait optimism relates to an a priori belief about the likelihood of rewards, but not losses, in our task. Critically, this positive belief behaves like a probabilistic prior, i.e. its influence reduces with increasing experience. Contrary to findings in the literature related to unrealistic optimism and self-beliefs, it does not appear to influence the iterative learning process directly.

Differential, but not opponent, effects of L -DOPA and citalopram on action learning with reward and punishment

RATIONALE

Decision-making involves two fundamental axes of control namely valence, spanning reward and punishment, and action, spanning invigoration and inhibition. We recently exploited a go/no-go task whose contingencies explicitly decouple valence and action to show that these axes are inextricably coupled during learning. This results in a disadvantage in learning to go to avoid punishment and in learning to no-go to obtain a reward. The neuromodulators dopamine and serotonin are likely to play a role in these asymmetries: Dopamine signals anticipation of future rewards and is also involved in an invigoration of motor responses leading to reward, but it also arbitrates between different forms of control. Conversely, serotonin is implicated in motor inhibition and punishment processing.

OBJECTIVE

To investigate the role of dopamine and serotonin in the interaction between action and valence during learning.Methods We combined computational modeling with pharmacological manipulation in 90 healthy human volunteers, using levodopa and citalopram to affect dopamine and serotonin, respectively.

RESULTS

We found that, after administration of levodopa,action learning was less affected by outcome valence when compared with the placebo and citalopram groups. This highlights in this context a predominant effect of levodopa in controlling the balance between different forms of control.Citalopram had distinct effects, increasing participants'tendency to perform active responses independent of outcome valence, consistent with a role in decreasing motor inhibition.

CONCLUSIONS

Our findings highlight the rich complexities of the roles played by dopamine and serotonin during instrumental learning.

Serotonin and aversive Pavlovian control of instrumental behavior in humans

Adaptive decision-making involves interaction between systems regulating Pavlovian and instrumental control of behavior. Here we investigate in humans the role of serotonin in such Pavlovian-instrumental transfer in both the aversive and the appetitive domain using acute tryptophan depletion, known to lower central serotonin levels. Acute tryptophan depletion attenuated the inhibiting effect of aversive Pavlovian cues on instrumental behavior, while leaving unaltered the activating effect of appetitive Pavlovian cues. These data suggest that serotonin is selectively involved in Pavlovian inhibition due to aversive expectations and have implications for our understanding of the mechanisms underlying a range of affective, impulsive, and aggressive neuropsychiatric disorders.

Striatal dysfunction during reversal learning in unmedicated schizophrenia patients

Subjects with schizophrenia are impaired at reinforcement-driven reversal learning from as early as their first episode. The neurobiological basis of this deficit is unknown. We obtained behavioral and fMRI data in 24 unmedicated, primarily first episode, schizophrenia patients and 24 age-, IQ- and gender-matched healthy controls during a reversal learning task. We supplemented our fMRI analysis, focusing on learning from prediction errors, with detailed computational modeling to probe task solving strategy including an ability to deploy an internal goal directed model of the task. Patients displayed reduced functional activation in the ventral striatum (VS) elicited by prediction errors. However, modeling task performance revealed that a subgroup did not adjust their behavior according to an accurate internal model of the task structure, and these were also the more severely psychotic patients. In patients who could adapt their behavior, as well as in controls, task solving was best described by cognitive strategies according to a Hidden Markov Model. When we compared patients and controls who acted according to this strategy, patients still displayed a significant reduction in VS activation elicited by informative errors that precede salient changes of behavior (reversals). Thus, our study shows that VS dysfunction in schizophrenia patients during reward-related reversal learning remains a core deficit even when controlling for task solving strategies. This result highlights VS dysfunction is tightly linked to a reward-related reversal learning deficit in early, unmedicated schizophrenia patients.

•

Probabilistic reversal learning was examined in unmedicated schizophrenia patients.

•

Computational modeling assessed subjects ability to use the latent task structure.

•

SZ patients showed lower reinforcement sensitivity and higher switch tendency.

•

Blunted striatal prediction error signal in unmedicated schizophrenia patients.

•

PFC activation during reversal errors intact in SZ patients able to do the task.

Aversive Pavlovian Control of Instrumental Behavior in Humans

Adaptive behavior involves interactions between systems regulating Pavlovian and instrumental control of actions. Here, we present the first investigation of the neural mechanisms underlying aversive Pavlovian–instrumental transfer using fMRI in humans. Recent evidence indicates that these Pavlovian influences on instrumental actions are action-specific: Instrumental approach is invigorated by appetitive Pavlovian cues but inhibited by aversive Pavlovian cues. Conversely, instrumental withdrawal is inhibited by appetitive Pavlovian cues but invigorated by aversive Pavlovian cues. We show that BOLD responses in the amygdala and the nucleus accumbens were associated with behavioral inhibition by aversive Pavlovian cues, irrespective of action context. Furthermore, BOLD responses in the ventromedial prefrontal cortex differed between approach and withdrawal actions. Aversive Pavlovian conditioned stimuli modulated connectivity between the ventromedial prefrontal cortex and the caudate nucleus. These results show that action-specific aversive control of instrumental behavior involves the modulation of fronto-striatal interactions by Pavlovian conditioned stimuli.

Pawlowsch-Instrumentelle Transfereffekte bei Alkoholabhängigkeit

Hintergrund: Alkoholabhängigkeit ist eine Verkettung ungünstiger Entscheidungen in Bezug auf Alkoholkonsum. Dieses Entscheidungsmuster scheint sich u. a wegen pawlowsch-instrumentellen Transfereffekten (PIT-Effekten) immer wieder zu reproduzieren. Ziel dieser Literaturzusammenschau ist, wichtige Befunde zum Zusammenhang zwischen PIT-Effekten und Suchterkrankungen zusammenzutragen und offene Fragen im Hinblick auf PIT bei Alkoholabhängigkeit aufzuzeigen. Methoden: Die Literaturzusammenschau nutzte keine systematische Literaturrecherche, sondern basierte auf den Recherchen im Rahmen der Forschergruppe 1617 (Learning and Habitization in Alcohol Dependence, LeAD) der Deutschen Forschungsgemeinschaft (DFG). Ergebnisse: PIT-Effekte könnten im Zusammenhang mit Alkoholabhängigkeit möglicherweise zu einem Teufelskreis führen. Dieser besteht aus der Verstärkung von PIT-Effekten durch Alkoholkonsum und verstärktem Alkoholkonsum aufgrund von verstärkten PIT-Effekten. Diskussion: PIT-Effekte bei Alkoholabhängigkeit sind bisher vorwiegend aus Tierstudien bekannt. Das PIT-Paradigma kann uns allerdings auch in der humanen Suchtforschung Aufschluss darüber geben, wie bestimmte Reizmuster Alkoholabhängige zum wiederholten Alkoholkonsum motivieren. Demnach können PIT-Experimente womöglich auch helfen Alkoholrückfälle vorherzusagen.

Go and no-go learning in reward and punishment: interactions between affect and effect

Decision-making invokes two fundamental axes of control: affect or valence, spanning reward and punishment, and effect or action, spanning invigoration and inhibition. We studied the acquisition of instrumental responding in healthy human volunteers in a task in which we orthogonalized action requirements and outcome valence. Subjects were much more successful in learning active choices in rewarded conditions, and passive choices in punished conditions. Using computational reinforcement-learning models, we teased apart contributions from putatively instrumental and Pavlovian components in the generation of the observed asymmetry during learning. Moreover, using model-based fMRI, we showed that BOLD signals in striatum and substantia nigra/ventral tegmental area (SN/VTA) correlated with instrumentally learnt action values, but with opposite signs for go and no-go choices. Finally, we showed that successful instrumental learning depends on engagement of bilateral inferior frontal gyrus. Our behavioral and computational data showed that instrumental learning is contingent on overcoming inherent and plastic Pavlovian biases, while our neuronal data showed this learning is linked to unique patterns of brain activity in regions implicated in action and inhibition respectively.

Bonsai trees in your head: how the pavlovian system sculpts goal-directed choices by pruning decision trees

When planning a series of actions, it is usually infeasible to consider all potential future sequences; instead, one must prune the decision tree. Provably optimal pruning is, however, still computationally ruinous and the specific approximations humans employ remain unknown. We designed a new sequential reinforcement-based task and showed that human subjects adopted a simple pruning strategy: during mental evaluation of a sequence of choices, they curtailed any further evaluation of a sequence as soon as they encountered a large loss. This pruning strategy was Pavlovian: it was reflexively evoked by large losses and persisted even when overwhelmingly counterproductive. It was also evident above and beyond loss aversion. We found that the tendency towards Pavlovian pruning was selectively predicted by the degree to which subjects exhibited sub-clinical mood disturbance, in accordance with theories that ascribe Pavlovian behavioural inhibition, via serotonin, a role in mood disorders. We conclude that Pavlovian behavioural inhibition shapes highly flexible, goal-directed choices in a manner that may be important for theories of decision-making in mood disorders.

Planning is tricky because choices we make now affect future choices, and future choices and outcomes should guide current choices. Because there are exponentially many combinations of future choices and actions, brute-force approaches that consider all possible combinations work only for trivially small problems. Here, we describe how humans use a simple Pavlovian strategy to cut an expanding decision tree down to a computationally manageable size. We find that humans use this strategy even when it is disadvantageous, and that the tendency to use it is related to mild depressive symptoms. The findings, we suggest, can be interpreted within a theoretical framework which relates Pavlovian behavioural inhibition to serotonin and mood disorders.