Model-Based Control in Dimensional Psychiatry

Resting-state functional connectivity of goal-directed and habitual-learning systems: The efficacy of cognitive-behavioral therapy for obsessive-compulsive disorder

BACKGROUND

There is an imbalance between goal-directed and habitual-learning system in patients with obsessive-compulsive disorder (OCD). At present, the relationship between cognitive behavior therapy (CBT) as a first-line therapy and goal-directed and habitual-learning disorder is still unclear. We attempted to discuss the effect of CBT treatment in patients with OCD, using abnormalities in goal-directed and habitual-learning-related brain regions at baseline as predictive factors.

METHODS

A total of 71 subjects, including 35 OCD patients and 36 healthy controls, were recruited. The OCD patients underwent 8 weeks of CBT. These patients were divided into two groups based on treatment response (Nresponders = 18, Nnonresponders = 17). Further subgroup analysis was conducted based on disease duration (Nshort = 17, Nlong = 18) and age of onset (Nearly = 14, Nlate = 21). We collected resting-state ROI-ROI functional connectivity data and apply repeated-measures linear mixed-effects models to investigate the differences of different subgroups.

RESULTS

CBT led to symptom improvement in OCD patients, with varying degrees of effectiveness across subgroups. The orbitofrontal cortex (OFC) and insula, key regions for goal-directed behavior and habitual-learning, respectively, showed significant impacts on CBT efficacy in subgroups with different disease durations and ages of onset.

CONCLUSION

The findings suggest that the goal-directed system may influence the efficacy of CBT through goal selection, maintenance, and emotion regulation. Furthermore, we found that disease duration and age of onset may affect treatment outcomes by modulating functional connectivity between goal-directed and habitual-learning brain regions.

Associations between somatomotor-putamen resting state connectivity and obsessive-compulsive symptoms vary as a function of stress during early adolescence: Data from the ABCD study

Obsessive-compulsive symptoms (OCS) are relatively common during adolescence although most individuals do not meet diagnostic criteria for obsessive-compulsive disorder (OCD). Nonetheless, OCS during adolescence are associated with comorbid psychopathologies and behavioral problems. Heightened levels of environmental stress and greater functional connectivity between the somatomotor network and putamen have been previously associated with elevated OCS in OCD patients relative to healthy controls. However, the interaction of these factors within the same sample of individuals has been understudied. This study examined somatomotor-putamen resting state connectivity, stress, and their interaction on OCS in adolescents from 9-12 years of age. Participants (n = 6386) were drawn from the ABCD Study 4.0 release. Multilevel modeling was used to account for nesting in the data and to assess changes in OCS in this age range. Stress moderated the association between somatomotor-putamen connectivity and OCS (β = 0.35, S.E. = 0.13, p = 0.006). Participants who reported more stress than their average and had greater somatomotor-left putamen connectivity reported more OCS, whereas participants who reported less stress than their average and had greater somatomotor-left putamen connectivity reported less OCS. These data suggest that stress differentially affects the direction of association between somatomotor-putamen connectivity and OCS. Individual differences in the experience or perception of stress may contribute to more OCS in adolescents with greater somatomotor-putamen connectivity.

Risk-promoting effects of reward-paired cues in human sign- and goal-trackers

Animal research suggests trait-like individual variation in the degree of incentive salience attribution to reward-predictive cues, defined phenotypically as sign-tracking (high) and goal-tracking (low incentive salience attribution). While these phenotypes have been linked to addiction features in rodents, their translational validity is less clear. Here, we examined whether sign- and goal-tracking in healthy human volunteers modulates the effects of reward-paired cues on decision making. Sign-tracking was measured in a Pavlovian conditioning paradigm as the amount of eye gaze fixation on the reward-predictive cue versus the location of impending reward delivery. In Study 1 (Cherkasova et al., 2018), participants were randomly assigned to perform a binary choice task in which rewards were either accompanied (cued, n = 63) or unaccompanied (uncued, n = 68) by money images and casino jingles. In Study 2, participants (n = 58) performed cued and uncued versions of the task in a within-subjects design. Across both studies, cues promoted riskier choice. Sign-tracking was not associated with risky choice in either study. Goal-tracking rather than sign-tracking was significantly associated with greater risk-promoting effects of cues in Study 1 but not in Study 2, although the direction of findings was consistent across both studies. These findings are at odds with the notion of sign-trackers being preferentially susceptible to the influence of reward cues on behavior and point to the role of mechanisms besides incentive salience in mediating such influences.

Cognitive Control in Schizophrenia: Advances in Computational Approaches

Psychiatric research is undergoing significant advances in an emerging subspeciality of computational psychiatry, building upon cognitive neuroscience research by expanding to neurocomputational modeling. Here, we illustrate some research trends in this domain using work on proactive cognitive control deficits in schizophrenia as an example. We provide a selective review of formal modeling approaches to understanding cognitive control deficits in psychopathology, focusing primarily on biologically plausible connectionist-level models as well as mathematical models that generate parameter estimates of putatively dissociable psychological or neural processes. We illustrate some of the advantages of these models in terms of understanding both cognitive control deficits in schizophrenia and the potential roles of effort and motivation. Further, we highlight critical future directions for this work, including a focus on establishing psychometric properties, additional work modeling psychotic symptoms and their interaction with cognitive control, and the need to expand both behavioral and neural modeling to samples that include individuals with different mental health conditions, allowing for the examination of dissociable neural or psychological substrates for seemingly similar cognitive impairments across disorders.

Model-based and model-free mechanisms in methamphetamine use disorder

People with methamphetamine use disorder (MUD) struggle to shift their behaviour from methamphetamine-orientated habits to goal-oriented choices. The model-based/model-free framework is well suited to understand this difficulty by unpacking the computational mechanisms that support experienced-based (model-free) and goal-directed (model-based) choices. We aimed to examine whether 1) participants with MUD differed from controls on behavioural proxies and/or computational mechanisms of model-based/model-free choices; 2) model-based/model-free decision-making correlated with MUD symptoms; and 3) model-based/model-free deficits improved over six weeks in the group with MUD. Participants with MUD and controls with similar age, IQ and socioeconomic status completed the Two-Step Task at treatment commencement (MUD n = 30, Controls n = 31) and six weeks later (MUD n = 23, Controls n = 26). We examined behavioural proxies of model-based/model-free decisions using mixed logistic regression, and their underlying mechanisms using computational modelling. At a behavioural level, participants with MUD were more likely to switch their choices following rewarded actions, although this pattern improved at follow up. At a computational level, groups were similar in their use of model-based mechanisms, but participants with MUD were less likely to apply model-free mechanisms and less likely to repeat rewarded actions. We did not find evidence that individual differences in model-based or model-free parameters were associated with greater severity of methamphetamine dependence, nor did we find that group differences in computational parameters changed between baseline and follow-up assessment. Decision-making challenges in people with MUD are likely related to difficulties in pursuing choices previously associated with positive outcomes.

Closing the loop in psychiatric deep brain stimulation: physiology, psychometrics, and plasticity

Deep brain stimulation (DBS) is an invasive approach to precise modulation of psychiatrically relevant circuits. Although it has impressive results in open-label psychiatric trials, DBS has also struggled to scale to and pass through multi-center randomized trials. This contrasts with Parkinson disease, where DBS is an established therapy treating thousands of patients annually. The core difference between these clinical applications is the difficulty of proving target engagement, and of leveraging the wide range of possible settings (parameters) that can be programmed in a given patient's DBS. In Parkinson's, patients' symptoms change rapidly and visibly when the stimulator is tuned to the correct parameters. In psychiatry, those same changes take days to weeks, limiting a clinician's ability to explore parameter space and identify patient-specific optimal settings. I review new approaches to psychiatric target engagement, with an emphasis on major depressive disorder (MDD). Specifically, I argue that better engagement may come by focusing on the root causes of psychiatric illness: dysfunction in specific, measurable cognitive functions and in the connectivity and synchrony of distributed brain circuits. I overview recent progress in both those domains, and how it may relate to other technologies discussed in companion articles in this issue.

Listening to the Data: Computational Approaches to Addiction and Learning

Computational approaches hold great promise for identifying novel treatment targets and creating translational therapeutics for substance use disorders. From circuitries underlying decision-making to computationally derived neural markers of drug-cue reactivity, this review is a summary of the approaches to data presented at our 2023 Society for Neuroscience Mini-Symposium. Here, we highlight data- and hypothesis-driven computational approaches that recently afforded advancements in addiction and learning neuroscience. First, we discuss the value of hypothesis-driven algorithmic modeling approaches, which integrate behavioral, neural, and cognitive outputs to refine hypothesis testing. Then, we review the advantages of data-driven dimensionality reduction and machine learning methods for uncovering novel predictor variables and elucidating relationships in high-dimensional data. Overall, this review highlights recent breakthroughs in cognitive mapping, model-based analysis of behavior/risky decision-making, patterns of drug taking, relapse, and neuromarker discovery, and showcases the benefits of novel modeling techniques, across both preclinical and clinical data.

Neuronal activity and reward processing in relation to binge eating

PURPOSE OF REVIEW

Studies increasingly show the importance of reward processing in binge eating and provide evidence of associated changes in the neurobiological reward system. This review gives an up-to-date overview of the neurobiological substrates of reward processing subconstructs in binge eating. Neural findings are linked to different behavioral theories and the clinical relevance is discussed.

RECENT FINDINGS

Increased neural responses in the orbitofrontal cortex, anterior cingulate cortex as well as striatum during anticipation and receipt of food rewards are found in association to binge eating. Increased model-free learning is also found and associated with altered brain reward reactivity. Data in rest report reduced striatal dopamine release and lower frontostriatal connectivity. Mechanisms of onset of binge eating are less clear, but specific personality traits, related to frontostriatal dysconnectivity, probably increase the risk of binge eating onset.

SUMMARY

Both structural and task-based imaging studies show differences in the neurobiological reward system in binge eating. These changes are linked to specific reward processing, such as altered reward responsiveness to food cues, reinforcement learning, and habitual behavior. Findings are lined with different behavioral theories of binge eating, and a staging model is described, from onset to full illness development. Understanding the specific underlying aberrant reward mechanism in binge eating, associated with different stages of the illness, enables caregivers to focus their treatment more precisely.

Impairment of arbitration between model-based and model-free reinforcement learning in obsessive-compulsive disorder

Introduction

Obsessive-compulsive disorder (OCD) is characterized by an imbalance between goal-directed and habitual learning systems in behavioral control, but it is unclear whether these impairments are due to a single system abnormality of the goal-directed system or due to an impairment in a separate arbitration mechanism that selects which system controls behavior at each point in time.

Methods

A total of 30 OCD patients and 120 healthy controls performed a 2-choice, 3-stage Markov decision-making paradigm. Reinforcement learning models were used to estimate goal-directed learning (as model-based reinforcement learning) and habitual learning (as model-free reinforcement learning). In general, 29 high Obsessive-Compulsive Inventory-Revised (OCI-R) score controls, 31 low OCI-R score controls, and all 30 OCD patients were selected for the analysis.

Results

Obsessive-compulsive disorder (OCD) patients showed less appropriate strategy choices than controls regardless of whether the OCI-R scores in the control subjects were high (p = 0.012) or low (p < 0.001), specifically showing a greater model-free strategy use in task conditions where the model-based strategy was optimal. Furthermore, OCD patients (p = 0.001) and control subjects with high OCI-R scores (H-OCI-R; p = 0.009) both showed greater system switching rather than consistent strategy use in task conditions where model-free use was optimal.

Conclusion

These findings indicated an impaired arbitration mechanism for flexible adaptation to environmental demands in both OCD patients and healthy individuals reporting high OCI-R scores.

Goal-directed and habitual decision making under stress in gambling disorder: An fMRI study

The development of addictive behaviors has been suggested to be related to a transition from goal-directed to habitual decision making. Stress is a factor known to prompt habitual behavior and to increase the risk for addiction and relapse. In the current study, we therefore used functional MRI to investigate the balance between goal-directed 'model-based' and habitual 'model-free' control systems and whether acute stress would differentially shift this balance in gambling disorder (GD) patients compared to healthy controls (HCs). Using a within-subject design, 22 patients with GD and 20 HCs underwent stress induction or a control condition before performing a multistep decision-making task during fMRI. Salivary cortisol levels showed that the stress induction was successful. Contrary to our hypothesis, GD patients did not show impaired goal-directed 'model-based' decision making, which remained similar to HCs after stress induction. Bayes factors provided three times more evidence against a difference between the groups or a group-by-stress interaction on the balance between model-based and model-free decision making. Similarly, no differences were found between groups and conditions on the neural estimates of model-based or model-free decision making. These results challenge the notion that GD is related to an increased reliance on habitual (or decreased goal-directed) control, even during stress.

Spatiotemporal Precision of Neuroimaging in Psychiatry

Aberrant patterns of cognition, perception, and behavior seen in psychiatric disorders are thought to be driven by a complex interplay of neural processes that evolve at a rapid temporal scale. Understanding these dynamic processes in vivo in humans has been hampered by a trade-off between spatial and temporal resolutions inherent to current neuroimaging technology. A recent trend in psychiatric research has been the use of high temporal resolution imaging, particularly magnetoencephalography, often in conjunction with sophisticated machine learning decoding techniques. Developments here promise novel insights into the spatiotemporal dynamics of cognitive phenomena, including domains relevant to psychiatric illnesses such as reward and avoidance learning, memory, and planning. This review considers recent advances afforded by exploiting this increased spatiotemporal precision, with specific reference to applications that seek to drive a mechanistic understanding of psychopathology and the realization of preclinical translation.

Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective

A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.

Belief updating in psychosis, depression and anxiety disorders: A systematic review across computational modelling approaches

Alterations in belief updating are proposed to underpin symptoms of psychiatric illness, including psychosis, depression, and anxiety. Key parameters underlying belief updating can be captured using computational modelling techniques, aiding the identification of unique and shared deficits, and improving diagnosis and treatment. We systematically reviewed research that applied computational modelling to probabilistic tasks measuring belief updating in stable and volatile (changing) environments, across clinical and subclinical psychosis (n = 17), anxiety (n = 9), depression (n = 9) and transdiagnostic samples (n = 9). Depression disorders related to abnormal belief updating in response to the valence of rewards, evidenced in both stable and volatile environments. Whereas psychosis and anxiety disorders were associated with difficulties adapting to changing contingencies specifically, indicating an inflexibility and/or insensitivity to environmental volatility. Higher-order learning models revealed additional difficulties in the estimation of overall environmental volatility across psychosis disorders, showing increased updating to irrelevant information. These findings stress the importance of investigating belief updating in transdiagnostic samples, using homogeneous experimental and computational modelling approaches.

Striatopallidal adenosine A2A receptor modulation of goal-directed behavior: Homeostatic control with cognitive flexibility

Dysfunction of goal-directed behaviors under stressful or pathological conditions results in impaired decision-making and loss of flexibility of thoughts and behaviors, which underlie behavioral deficits ranging from depression, obsessive-compulsive disorders and drug addiction. Tackling the neuromodulators fine-tuning this core behavioral element may facilitate the development of effective strategies to control these deficits present in multiple psychiatric disorders. The current investigation of goal-directed behaviors has concentrated on dopamine and glutamate signaling in the corticostriatal pathway. In accordance with the beneficial effects of caffeine intake on mood and cognitive dysfunction, we now propose that caffeine's main site of action - adenosine A2A receptors (A2AR) - represent a novel target to homeostatically control goal-directed behavior and cognitive flexibility. A2AR are abundantly expressed in striatopallidal neurons and colocalize and interact with dopamine D2, NMDA and metabotropic glutamate 5 receptors to integrate dopamine and glutamate signaling. Specifically, striatopallidal A2AR (i) exert an overall "break" control of a variety of cognitive processes, making A2AR antagonists a novel strategy for improving goal-directed behavior; (ii) confer homeostatic control of goal-directed behavior by acting at multiple sites with often opposite effects, to enhance cognitive flexibility; (iii) integrate dopamine and adenosine signaling through multimeric A2AR-D2R heterocomplexes allowing a temporally precise fine-tuning in response to local signaling changes. As the U.S. Food and Drug Administration recently approved the A2AR antagonist Nourianz® (istradefylline) to treat Parkinson's disease, striatal A2AR-mediated control of goal-directed behavior may offer a new and real opportunity for improving deficits of goal-directed behavior and enhance cognitive flexibility under various neuropsychiatric conditions. This article is part of the Special Issue on "Purinergic Signaling: 50 years".

A mechanistic staging model of reward processing alterations in individuals with binge-type eating disorders

Altered reward processing is thought to characterize binge-type eating disorders, but the exact nature of these alterations is unclear. A more fine-grained understanding of whether specific aspects of reward processing contribute to the development or maintenance of binge eating may point to new therapeutic targets and personalized treatments. The incentive sensitization theory of addiction proposes that repeated use of a substance increases the desire to approach a reward ('wanting') but not pleasure when consuming the reward ('liking'), suggesting that reward processes driving addiction change over time. We hypothesize that the same may be true for binge eating. Further, consistent with the maladaptive scaling hypothesis, reward processing may be heightened for multiple reinforcers in at-risk individuals but become tuned toward food once binge eating is initiated. In this article, we propose a mechanistic staging model of reward processing in binge-type eating disorders that synthesizes existing data and posits that alterations of reward processing depend on illness stage and reward type. We outline translational methods for testing key hypotheses and discuss clinical implications. Considering reward processing alterations in relation to illness stage has the potential to improve treatment outcomes by ensuring that the mechanisms targeted are personalized to the individual patient. PUBLIC SIGNIFICANCE: Individuals with binge-type eating disorders experience alterations in their desire for, and pleasure from, food. We believe that the exact nature of these alterations in reward processing change over the course of illness-from the at-risk state to an established illness. If true, treatments for binge-type eating disorders that target reward processing should be personalized to the illness stage of the patient.

An integrative evolutionary framework for psychopathology

The field of psychopathology is in a transformative phase, and is witnessing a renewed surge of interest in theoretical models of mental disorders. While many interesting proposals are competing for attention in the literature, they tend to focus narrowly on the proximate level of analysis and lack a broader understanding of biological function. In this paper, we present an integrative framework for mental disorders built on concepts from life history theory, and describe a taxonomy of mental disorders based on its principles, the fast-slow-defense model (FSD). The FSD integrates psychopathology with normative individual differences in personality and behavior, and allows researchers to draw principled distinctions between broad clusters of disorders, as well as identify functional subtypes within current diagnostic categories. Simulation work demonstrates that the model can explain the large-scale structure of comorbidity, including the apparent emergence of a general "p factor" of psychopathology. A life history approach also provides novel integrative insights into the role of environmental risk/protective factors and the developmental trajectories of various disorders.

Cardiac sensitivity to rewards in cognitively inflexible nonclinical participants

Background

In psychopathologies characterized by compulsive decision-making, core impairments include cognitive inflexibility and excessive sensitivity to rewards. It has been posited that traits shared by nonclinical individuals and psychiatric patients could help explain the pathogenesis of compulsive decision-making.

Methods

To investigate whether cognitive inflexibility predisposes nonclinical individuals to poor choices and hyper-reactivity to reward, we recruited people with high and low scores for cognitive persistence and used the Iowa Gambling Task to assess decision-making and cardiac reactivity to monetary gains/losses.

Results

As is frequently observed in psychophysiological research, the data indicated discrepancies among self-reports, behavior, and physiology. Cognitive inflexibility was not related to worse performance; however, monetary gains, in line with the literature, prompted marked cardiac accelerations. Consistent with our research goal, only inflexible participants showed large cardiac accelerations during the largest monetary wins.

Discussion

Taken together, the data confirm an association between cognitive persistence and physiological reward sensitivity in a nonclinical population. The findings are in line with recent theories on the development of compulsive behaviors that consider cognitive inflexibility as a transdiagnostic impairment and predisposing factor for excessive reactivity to rewards, and might act both as a preexisting individual trait and drug-induced deficit.

Mesial Prefrontal Cortex and Alcohol Misuse: Dissociating Cross-sectional and Longitudinal Relationships in UK Biobank

BACKGROUND

Alcohol misuse is a major global public health issue. The disorder is characterized by aberrant neural networks interacting with environment and genetics. Dissecting the neural substrates and functional networks that relate to longitudinal changes in alcohol use from those that relate to alcohol misuse cross-sectionally is important to elucidate therapeutic approaches.

METHODS

To assess how neuroimaging data, including T1, resting-state functional magnetic resonance imaging, and diffusion-weighted imaging, relate to alcohol misuse cross-sectionally and longitudinally in the UK Biobank, this study analyzed range of alcohol misuse in a population-based normative sample of 24,784 participants, ages 45 to 81 years old, in a cross-sectional analysis and a sample of 3070 participants in a longitudinal analysis 2 years later.

RESULTS

Cross-sectional analysis showed that alcohol use is associated with a reduction in dorsal anterior cingulate cortex and dorsomedial prefrontal cortex gray matter concentration and functional resting-state connectivity (nodal degree: t_24,422 = -12.99, p < 1 × 10^-17). Reduced dorsal anterior cingulate cortex/dorsomedial prefrontal cortex functional connections to the ventrolateral prefrontal cortex, amygdala, and striatum relate to greater alcohol use. In a longitudinal analysis, higher resting-state nodal degree (t₃₀₃₆ = -3.27, p = .0011) and T1 gray matter concentration in the ventromedial prefrontal cortex relate to reduced alcohol intake frequency 2 years later. Higher ventromedial prefrontal cortex and frontoparietal executive network functional connectivity is associated with lower subsequent drinking longitudinally.

CONCLUSIONS

Dorsal versus ventromedial prefrontal regions are differentially related to alcohol misuse cross-sectionally or longitudinally in a large UK Biobank normative dataset. Our study provides a comprehensive understanding of the neurobiological substrates of alcohol use as a state or prospectively, thereby providing potential targets for clinical treatment.

The role of reinforcement learning and value-based decision-making frameworks in understanding food choice and eating behaviors

The obesogenic food environment includes easy access to highly-palatable, energy-dense, "ultra-processed" foods that are heavily marketed to consumers; therefore, it is critical to understand the neurocognitive processes the underlie overeating in response to environmental food-cues (e.g., food images, food branding/advertisements). Eating habits are learned through reinforcement, which is the process through which environmental food cues become valued and influence behavior. This process is supported by multiple behavioral control systems (e.g., Pavlovian, Habitual, Goal-Directed). Therefore, using neurocognitive frameworks for reinforcement learning and value-based decision-making can improve our understanding of food-choice and eating behaviors. Specifically, the role of reinforcement learning in eating behaviors was considered using the frameworks of (1) Sign-versus Goal-Tracking Phenotypes; (2) Model-Free versus Model-Based; and (3) the Utility or Value-Based Model. The sign-and goal-tracking phenotypes may contribute a mechanistic insight on the role of food-cue incentive salience in two prevailing models of overconsumption-the Extended Behavioral Susceptibility Theory and the Reactivity to Embedded Food Cues in Advertising Model. Similarly, the model-free versus model-based framework may contribute insight to the Extended Behavioral Susceptibility Theory and the Healthy Food Promotion Model. Finally, the value-based model provides a framework for understanding how all three learning systems are integrated to influence food choice. Together, these frameworks can provide mechanistic insight to existing models of food choice and overconsumption and may contribute to the development of future prevention and treatment efforts.

Increased Habit Frequency in the Daily Lives of Patients with Acute Anorexia Nervosa

Strict eating routines and frequent rigid behavior patterns are commonly observed in patients with anorexia nervosa (AN). A recent theory proposes that while these behaviors may have been reinforced initially, they later become habitual. To date, however, research has been overly focused on eating-disorder (ED)-related habits. Over the course of seven days, we applied an ecological momentary assessment (EMA) to investigate the habit frequency and strength of ED-specific (food intake) and ED-unspecific (hygiene) habits in the daily lives of a sample of n = 57 AN and n = 57 healthy controls (HC). The results of the hierarchical models revealed that habits were significantly more likely in patients compared with HC for both categories, independently. Furthermore, a lower body mass index (BMI) was associated with increased habit frequency in AN. Our study strengthens the habit theory of AN by showing the relevance of habits beyond ED-specific behavioral domains. This also supports the development of innovative therapeutic interventions targeting habitual behavior in EDs.

Gambling Environment Exposure Increases Temporal Discounting but Improves Model-Based Control in Regular Slot-Machine Gamblers

Gambling disorder is a behavioral addiction that negatively impacts personal finances, work, relationships and mental health. In this pre-registered study (https://osf.io/5ptz9/) we investigated the impact of real-life gambling environments on two computational markers of addiction, temporal discounting and model-based reinforcement learning. Gambling disorder is associated with increased temporal discounting and reduced model-based learning. Regular gamblers (n = 30, DSM-5 score range 3-9) performed both tasks in a neutral (café) and a gambling-related environment (slot-machine venue) in counterbalanced order. Data were modeled using drift diffusion models for temporal discounting and reinforcement learning via hierarchical Bayesian estimation. Replicating previous findings, gamblers discounted rewards more steeply in the gambling-related context. This effect was positively correlated with gambling related cognitive distortions (pre-registered analysis). In contrast to our pre-registered hypothesis, model-based reinforcement learning was improved in the gambling context. Here we show that temporal discounting and model-based reinforcement learning are modulated in opposite ways by real-life gambling cue exposure. Results challenge aspects of habit theories of addiction, and reveal that laboratory-based computational markers of psychopathology are under substantial contextual control.

Need for cognition does not account for individual differences in metacontrol of decision making

Humans show metacontrol of decision making, that is they adapt their reliance on decision-making strategies toward situational differences such as differences in reward magnitude. Specifically, when higher rewards are at stake, individuals increase reliance on a more accurate but cognitively effortful strategy. We investigated whether the personality trait Need for Cognition (NFC) explains individual differences in metacontrol. Based on findings of cognitive effort expenditure in executive functions, we expected more metacontrol in individuals low in NFC. In two independent studies, metacontrol was assessed by means of a decision-making task that dissociates different reinforcement-learning strategies and in which reward magnitude was manipulated across trials. In contrast to our expectations, NFC did not account for individual differences in metacontrol of decision making. In fact, a Bayesian analysis provided moderate to strong evidence against a relationship between NFC and metacontrol. Beyond this, there was no consistent evidence for relationship between NFC and overall model-based decision making. These findings show that the effect of rewards on the engagement of effortful decision-making strategies is largely independent of the intrinsic motivation for engaging in cognitively effortful tasks and suggest a differential role of NFC for the regulation of cognitive effort in decision making and executive functions.

Alcohol dependence decreases functional activation of the caudate nucleus during model-based decision processes

BACKGROUND

Impaired decision making, a key characteristic of alcohol dependence (AD), manifests in continuous alcohol consumption despite severe negative consequences. The neural basis of this impairment in individuals with AD and differences with known neural decision mechanisms among healthy subjects are not fully understood. In particular, it is unclear whether the choice behavior among individuals with AD is based on a general impairment of decision mechanisms or is mainly explained by altered value attribution, with an overly high subjective value attributed to alcohol-related stimuli.

METHODS

Here, we use a functional magnetic resonance imaging (fMRI) monetary reward task to compare the neural processes of model-based decision making and value computation between AD individuals (n = 32) and healthy controls (n = 32). During fMRI, participants evaluated monetary offers with respect to dynamically changing constraints and different levels of uncertainty.

RESULTS

Individuals with AD showed lower activation associated with model-based decision processes in the caudate nucleus than controls, but there were no group differences in value-related neural activity or task performance.

CONCLUSIONS

Our findings highlight the role of the caudate nucleus in impaired model-based decisions of alcohol-dependent individuals.

Reinforcement learning detuned in addiction: integrative and translational approaches

Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.

Stronger Prejudices Are Associated With Decreased Model-Based Control

Background: Prejudices against minorities can be understood as habitually negative evaluations that are kept in spite of evidence to the contrary. Therefore, individuals with strong prejudices might be dominated by habitual or "automatic" reactions at the expense of more controlled reactions. Computational theories suggest individual differences in the balance between habitual/model-free and deliberative/model-based decision-making. Methods: 127 subjects performed the two Step task and completed the blatant and subtle prejudice scale. Results: By using analyses of choices and reaction times in combination with computational modeling, subjects with stronger blatant prejudices showed a shift away from model-based control. There was no association between these decision-making processes and subtle prejudices. Conclusion: These results support the idea that blatant prejudices toward minorities are related to a relative dominance of habitual decision-making. This finding has important implications for developing interventions that target to change prejudices across societies.

Volume and Connectivity Differences in Brain Networks Associated with Cognitive Constructs of Binge Eating

Bulimia nervosa (BN) and binge eating disorder (BED) are characterized by episodes of eating large amounts of food while experiencing a loss of control. Recent studies suggest that the underlying causes of BN/BED consist of a complex system of environmental cues, atypical processing of food stimuli, altered behavioral responding, and structural/functional brain differences compared with healthy controls (HC). In this narrative review, we provide an integrative account of the brain networks associated with the three cognitive constructs most integral to BN and BED, namely increased reward sensitivity, decreased cognitive control, and altered negative affect and stress responding. We show altered activity in BED/BN within several brain networks, specifically in the striatum, insula, prefrontal cortex (PFC) and orbitofrontal cortex (OFC), and cingulate gyrus. Numerous key nodes in these networks also differ in volume and connectivity compared with HC. We provide suggestions for how this integration may guide future research into these brain networks and cognitive constructs.

Effects of dopamine D2/3 and opioid receptor antagonism on the trade-off between model-based and model-free behaviour in healthy volunteers

Human behaviour requires flexible arbitration between actions we do out of habit and actions that are directed towards a specific goal. Drugs that target opioid and dopamine receptors are notorious for inducing maladaptive habitual drug consumption; yet, how the opioidergic and dopaminergic neurotransmitter systems contribute to the arbitration between habitual and goal-directed behaviour is poorly understood. By combining pharmacological challenges with a well-established decision-making task and a novel computational model, we show that the administration of the dopamine D2/3 receptor antagonist amisulpride led to an increase in goal-directed or 'model-based' relative to habitual or 'model-free' behaviour, whereas the non-selective opioid receptor antagonist naltrexone had no appreciable effect. The effect of amisulpride on model-based/model-free behaviour did not scale with drug serum levels in the blood. Furthermore, participants with higher amisulpride serum levels showed higher explorative behaviour. These findings highlight the distinct functional contributions of dopamine and opioid receptors to goal-directed and habitual behaviour and support the notion that even small doses of amisulpride promote flexible application of cognitive control.

Beyond Description and Deficits: How Computational Psychiatry Can Enhance an Understanding of Decision-Making in Anorexia Nervosa

PURPOSE OF REVIEW

Despite decades of research, knowledge of the mechanisms maintaining anorexia nervosa (AN) remains incomplete and clearly effective treatments elusive. Novel theoretical frameworks are needed to advance mechanistic and treatment research for this disorder. Here, we argue the utility of engaging a novel lens that differs from existing perspectives in psychiatry. Specifically, we argue the necessity of expanding beyond two historically common perspectives: (1) the descriptive perspective: the tendency to define mechanisms on the basis of surface characteristics and (2) the deficit perspective: the tendency to search for mechanisms associated with under-functioning of decision-making abilities and related circuity, rather than problems of over-functioning, in psychiatric disorders.

RECENT FINDINGS

Computational psychiatry can provide a novel framework for understanding AN because this approach emphasizes the role of computational misalignments (rather than absolute deficits or excesses) between decision-making strategies and environmental demands as the key factors promoting psychiatric illnesses. Informed by this approach, we argue that AN can be understood as a disorder of excess goal pursuit, maintained by over-engagement, rather than disengagement, of executive functioning strategies and circuits. Emerging evidence suggests that this same computational imbalance may constitute an under-investigated phenotype presenting transdiagnostically across psychiatric disorders. A variety of computational models can be used to further elucidate excess goal pursuit in AN. Most traditional psychiatric treatments do not target excess goal pursuit or associated neurocognitive mechanisms. Thus, targeting at the level of computational dysfunction may provide a new avenue for enhancing treatment for AN and related disorders.

Imbalance in Functional and Structural Connectivity Underlying Goal-Directed and Habitual Learning Systems in Obsessive-Compulsive Disorder

An imbalance between the goal-directed and habitual learning systems has been proposed to underlie compulsivity in obsessive-compulsive disorder (OCD). In addition, the overall balance between these systems may be influenced by stress hormones. We examined the multimodal networks underlying these dual learning systems. Both functional and structural measures indicated reduced connectivity within the goal-directed subnetwork (FC: P = 0.042; SC-FN: P = 0.014) and reduced connectivity between the goal-directed and habitual subnetworks (FC: P = 0.014; SC-FA: P = 0.052), but no differences within the habitual subnetwork in patients with OCD compared with controls. Path modeling indicated that anatomical connectivity in the goal-directed subnetwork influenced compulsive symptoms (R2 = 0.41), whereas functional connectivity within the habit subnetwork and between goal-directed and habitual subnetworks influenced obsessive symptoms (R2 = 0.63). In addition, the relationship between anatomical connectivity in the goal-directed subnetwork and compulsion was moderated by the stress hormone ACTH (adrenocorticotropic hormone), such that at low levels of ACTH greater connectivity resulted in lower compulsion, but at high levels of ACTH this relationship was reversed. These results provide new insights into the neural correlates of the imbalance between dual learning systems, and their relationship with symptom dimensions in patients with OCD. It may further support the reconceptualization of OCD as "compulsive-obsessive disorder," with a greater focus on the transdiagnostic dimension of compulsivity.

Examining the effect of depressive symptoms on habit formation and habit-breaking

BACKGROUND AND OBJECTIVES

Dysfunction in reward processing is a hallmark feature of depression. In the context of reinforcement learning, previous research has linked depression with reliance on simple habit-driven ('model-free') learning strategies over more complex, goal-directed ('model-based') strategies. However, the relationship between depression and habit-breaking remains an under-explored research area. The current study sought to bridge this gap by investigating the effect of depressive symptoms on habit formation and habit-breaking under monetary and social feedback conditions. Additionally, we examined whether spontaneous eyeblink rate (EBR), an indirect marker for striatal dopamine levels, would modulate such effects.

METHODS

Depressive symptoms were operationalized using self-report measures. To examine differences in habit formation and habit breaking, undergraduate participants (N = 156) completed a two-stage reinforcement learning task with a devaluation procedure using either monetary or social feedback.

RESULTS

Regression results showed that in the monetary feedback condition, spontaneous EBR moderated the relationship between depressive symptoms and model-free strategies; individuals with more depressive symptomatology and high EBR (higher dopamine levels) exhibited increased reliance on model-free strategies. Depressive symptoms negatively predicted devaluation sensitivity, indicative of difficulty in habit-breaking, in both monetary and social feedback contexts.

LIMITATIONS

Social feedback relied on fixed feedback rather than real-time peer evaluations; depressive symptoms were measured using self-report rather than diagnostic criteria for Major Depressive Disorder; dopaminergic functioning was measured using EBR rather than PET imaging; potential confounds were not controlled for.

CONCLUSIONS

These findings have implications for identifying altered patterns of habit formation and deficits in habit-breaking among those experiencing depressive symptoms.

Costly habitual avoidance is reduced by concurrent goal-directed approach in a modified devaluation paradigm

Avoidance habits potentially contribute to maintaining maladaptive, costly avoidance behaviors that persist in the absence of threat. However, experimental evidence about costly habitual avoidance is scarce. In two experiments, we tested whether extensively trained avoidance impairs the subsequent goal-directed approach of rewards. Healthy participants were extensively trained to avoid an aversive outcome by performing simple responses to distinct full-screen color stimuli. After the subsequent devaluation of the aversive outcome, participants received monetary rewards for correct responses to neutral object pictures, which were presented on top of the same full-screen colors. These approach responses were either compatible or incompatible with habitual avoidance responses. Notably, the full-screen colors were not relevant to inform approach responses. In Experiment 1, participants were not instructed about post-devaluation stimulus-response-reward contingencies. Accuracy was lower in habit-incompatible than in habit-compatible trials, indicating costly avoidance, whereas reaction times did not differ. In Experiment 2, contingencies were explicitly instructed. Accuracy differences disappeared, but reaction times were slower in habit-incompatible than in habit-compatible trials, indicating low-cost habitual avoidance tendencies. These findings suggest a small but consistent impact of habitual avoidance tendencies on subsequent goal-directed approach. Costly habitual responding could, however, be inhibited when competing goal-directed approach was easily realizable.

Fronto-striatal structures related with model-based control as an endophenotype for obsessive-compulsive disorder

Recent theories suggest a shift from model-based goal-directed to model-free habitual decision-making in obsessive–compulsive disorder (OCD). However, it is yet unclear, whether this shift in the decision process is heritable. We investigated 32 patients with OCD, 27 unaffected siblings (SIBs) and 31 healthy controls (HCs) using the two-step task. We computed behavioral and reaction time analyses and fitted a computational model to assess the balance between model-based and model-free control. 80 subjects also underwent structural imaging. We observed a significant ordered effect for the shift towards model-free control in the direction OCD > SIB > HC in our computational parameter of interest. However less directed analyses revealed no shift towards model-free control in OCDs. Nonetheless, we found evidence for reduced model-based control in OCDs compared to HCs and SIBs via 2nd stage reaction time analyses. In this measure SIBs also showed higher levels of model-based control than HCs. Across all subjects these effects were associated with the surface area of the left medial/right dorsolateral prefrontal cortex. Moreover, correlations between bilateral putamen/right caudate volumes and these effects varied as a function of group: they were negative in SIBs and OCDs, but positive in HCs. Associations between fronto-striatal regions and model-based reaction time effects point to a potential endophenotype for OCD.

The effects of fatty acid amide hydrolase inhibition and monoacylglycerol lipase inhibition on habit formation in mice

Emerging data indicate that endocannabinoid signaling is critical to the formation of habitual behavior. Previous work demonstrated that antagonism of cannabinoid receptor type 1 (CB1R) with AM251 during operant training impairs habit formation, but it is not known if this behavioral effect is specific to disrupted signaling of the endocannabinoid ligands anandamide or 2-arachidonoyl glycerol (2-AG). Here, we used selective pharmacological compounds during operant training to determine the impact of fatty acid amide hydrolase (FAAH) inhibition to increase anandamide (and other n-acylethanolamines) or monoacylglycerol lipase (MAGL) inhibition to increase 2-AG levels on the formation of habitual behaviors in mice using a food-reinforced contingency degradation procedure. We found, contrary to our hypothesis, that inhibition of FAAH and of MAGL disrupted the formation of habits. Next, AM251 was administered during training to verify that impaired habit formation could be assessed using contingency degradation. AM251-exposed mice responded at lower rates during training and at higher rates in the test. To understand the inconsistency with published data, we performed a proof-of-principle dose-response experiment to compare AM251 in our vehicle-solution to the published vehicle-suspension on response rates. We found consistent reductions in response rate with increasing doses of AM251 in solution and an inconsistent dose-response relationship with AM251 in suspension. Together, our data suggest that further characterization of the role of CB1R signaling in the formation of habitual responding is warranted and that augmenting endocannabinoids may have clinical utility for prophylactically preventing aberrant habit formation such as that hypothesized to occur in substance use disorders.

Anterior limb of the internal capsule tractography: relationship with capsulotomy outcomes in obsessive-compulsive disorder

OBJECTIVES

Surgical procedures targeting the anterior limb of the internal capsule (aLIC) can be effective in patients with selected treatment-refractory obsessive-compulsive disorder (OCD). The aLIC consists of white-matter tracts connecting cortical and subcortical structures and show a topographical organisation. Here we assess how aLIC streamlines are affected in OCD compared with healthy controls (HCs) and which streamlines are related with post-capsulotomy improvement.

METHODS

Diffusion-weighted MRI was used to compare white-matter microstructure via the aLIC between patients with OCD (n=100, 40 women, mean of age 31.8 years) and HCs (n=88, 39 women, mean of age 29.6 years). For each individual, the fractional anisotropy (FA) and streamline counts were calculated for each white-matter fibre bundle connecting a functionally defined prefrontal and subcortical region. Correlations between tractography measures and pre-capsulotomy and post-capsulotomy clinical outcomes (in obsessive-compulsive, anxiety and depression scores 6 months after surgery) were assessed in 41 patients with OCD.

RESULTS

Hierarchical clustering dendrograms show an aLIC organisation clustering lateral and dissociating ventral and dorsal prefrontal-thalamic streamlines, findings highly relevant to surgical targeting. Compared with HCs, patients with OCD had lower aLIC FA across multiple prefrontal cortical-subcortical regions (p<0.0073, false discovery rate-adjusted). Greater streamline counts of the dorsolateral prefrontal-thalamic tracts in patients with OCD predicted greater post-capsulotomy obsessive-compulsive improvement (p=0.016). In contrast, greater counts of the dorsal cingulate-thalamic streamlines predicted surgical outcomes mediated by depressive and anxiety improvements.

CONCLUSIONS

These findings shed light on the critical role of the aLIC in OCD and may potentially contribute towards precision targeting to optimise outcomes in OCD.

Habitual Routines and Automatic Tendencies Differential Roles in Alcohol Misuse Among Undergraduates

There is a debate over whether actions that resist devaluation (i.e., compulsive alcohol consumption) are primarily habit- or goal-directed. The incentive habit account of compulsive actions has received support from behavioral paradigms and brain imaging. In addition, the self-reported Creature of Habit Scale (COHS) has been proposed to capture inter-individual differences in habitual tendencies. It is subdivided into two dimensions: routine and automaticity. We first considered a French version of this questionnaire for validation, based on a sample of 386 undergraduates. The relationship between two dimensions of habit and the risk of substance use disorder and impulsive personality traits was also investigated. COHS has good psychometric properties with both features of habits positively associated with an Obsessive-Compulsive Inventory score. Besides, the propensity to rely more on routines was associated with lower levels of alcohol abuse and nicotine use, suggesting that some degree of routine might act as a protective factor against substance use. In contrast, a high automaticity score was associated with an increased risk of harmful alcohol use. These results demonstrate that the COHS is a valid measure of habitual tendencies and represents a useful tool for capturing inter-individual variations in drug use problems in undergraduates.

Computational Psychiatry for Computers

Computational psychiatry is a nascent field that attempts to use multi-level analyses of the underlying computational problems that we face in navigating a complex, uncertain and changing world to illuminate mental dysfunction and disease. Two particular foci of the field are the costs and benefits of environmental adaptivity and the danger and necessity of heuristics. Here, we examine the extent to which these foci and others can be used to study the actual and potential flaws of the artificial computational devices that we are increasingly inventing and empowering to navigate this very same environment on our behalf.

Heritability of overlapping impulsivity and compulsivity dimensional phenotypes

Impulsivity and compulsivity are traits relevant to a range of mental health problems and have traditionally been conceptualised as distinct constructs. Here, we reconceptualised impulsivity and compulsivity as partially overlapping phenotypes using a bifactor modelling approach and estimated heritability for their shared and unique phenotypic variance within a classical twin design. Adult twin pairs (N = 173) completed self-report questionnaires measuring psychological processes related to impulsivity and compulsivity. We fitted variance components models to three uncorrelated phenotypic dimensions: a general impulsive-compulsive dimension; and two narrower phenotypes related to impulsivity and obsessiveness.There was evidence of moderate heritability for impulsivity (A² = 0.33), modest additive genetic or common environmental effects for obsessiveness (A² = 0.25; C² = 0.23), and moderate effects of common environment (C² = 0.36) for the general dimension, This general impulsive-compulsive phenotype may reflect a quantitative liability to related mental health disorders that indexes exposure to potentially modifiable environmental risk factors.

Beyond dichotomies in reinforcement learning

Reinforcement learning (RL) is a framework of particular importance to psychology, neuroscience and machine learning. Interactions between these fields, as promoted through the common hub of RL, has facilitated paradigm shifts that relate multiple levels of analysis in a singular framework (for example, relating dopamine function to a computationally defined RL signal). Recently, more sophisticated RL algorithms have been proposed to better account for human learning, and in particular its oft-documented reliance on two separable systems: a model-based (MB) system and a model-free (MF) system. However, along with many benefits, this dichotomous lens can distort questions, and may contribute to an unnecessarily narrow perspective on learning and decision-making. Here, we outline some of the consequences that come from overconfidently mapping algorithms, such as MB versus MF RL, with putative cognitive processes. We argue that the field is well positioned to move beyond simplistic dichotomies, and we propose a means of refocusing research questions towards the rich and complex components that comprise learning and decision-making.

A Translational Paradigm to Study the Effects of Uncontrollable Stress in Humans

Theories on the aetiology of depression in humans are intimately linked to animal research on stressor controllability effects. However, explicit translations of established animal designs are lacking. In two consecutive studies, we developed a translational paradigm to study stressor controllability effects in humans. In the first study, we compared three groups of participants, one exposed to escapable stress, one yoked inescapable stress group, and a control group not exposed to stress. Although group differences indicated successful stress induction, the manipulation failed to differentiate groups according to controllability. In the second study, we employed an improved paradigm and contrasted only an escapable stress group to a yoked inescapable stress group. The final design successfully induced differential effects on self-reported perceived control, exhaustion, helplessness, and behavioural indices of adaptation to stress. The latter were examined in a new escape behaviour test which was modelled after the classic shuttle box animal paradigm. Contrary to the learned helplessness literature, exposure to uncontrollable stress led to more activity and exploration; however, these behaviours were ultimately not adaptive. We discuss the results and possible applications in light of the findings on learning and agency beliefs, inter-individual differences, and interventions aimed at improving resilience to stress-induced mental dysfunction.

Computational models of drug use and addiction: A review

In this brief review, we describe current computational models of drug-use and addiction that fall into 2 broad categories: mathematically based models that rely on computational theories, and brain-based models that link computations to brain areas or circuits. Across categories, many are models of learning and decision-making, which may be compromised in addiction. Several mathematical models take predictive coding approaches, focusing on Bayesian prediction error. Other models focus on learning processes and (traditional) prediction error. Brain-based models have incorporated prefrontal cortex, basal ganglia, and the dopamine system, based on the effects of drugs on dopamine, motivation, and executive control circuits. Several models specifically describe how behavioral control may transition from habitual to goal-directed systems, consistent with computational accounts of compromised "model-based" control. Some brain-based models have linked this to the transition of behavioral control from ventral to dorsal striatum. Overall, we propose that while computational models capture some aspects of addiction and have advanced our thinking, most have focused on the effects of drug use rather than addiction per se, most have not been tested on and/or supported by human data, and few capture multiple stages and symptoms of addiction. We conclude by suggesting a path forward for computational models of addiction. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Hunger and BMI modulate neural responses to sweet stimuli: fMRI meta-analysis

OBJECTIVE

Consuming sweet foods, even when sated, can lead to unwanted weight gain. Contextual factors, such as longer time fasting, subjective hunger, and body mass index (BMI), may increase the likelihood of overeating. Nevertheless, the neural mechanisms underlying these moderating influences on energy intake are poorly understood.

METHODS

We conducted both categorical meta-analysis and meta-regression of factors modulating neural responses to sweet stimuli, using data from 30 functional magnetic resonance imaging (fMRI) articles incorporating 39 experiments (N = 995) carried out between 2006 and 2019.

RESULTS

Responses to sweet stimuli were associated with increased activity in regions associated with taste, sensory integration, and reward processing. These taste-evoked responses were modulated by context. Longer fasts were associated with higher posterior cerebellar, thalamic, and striatal activity. Greater self-reported hunger was associated with higher medial orbitofrontal cortex (OFC), dorsal striatum, and amygdala activity and lower posterior cerebellar activity. Higher BMI was associated with higher posterior cerebellar and insular activity.

CONCLUSIONS

Variations in fasting time, self-reported hunger, and BMI are contexts associated with differential sweet stimulus responses in regions associated with reward processing and homeostatic regulation. These results are broadly consistent with a hierarchical model of taste processing. Hunger, but not fasting or BMI, was associated with sweet stimulus-related OFC activity. Our findings extend existing models of taste processing to include posterior cerebellar regions that are associated with moderating effects of both state (fast length and self-reported hunger) and trait (BMI) variables.

Addictions NeuroImaging Assessment (ANIA): Towards an integrative framework for alcohol use disorder

Alcohol misuse and addiction are major international public health issues. Addiction can be characterized as a disorder of aberrant neurocircuitry interacting with environmental, genetic and social factors. Neuroimaging in alcohol misuse can thus provide a critical window into underlying neural mechanisms, highlighting possible treatment targets and acting as clinical biomarkers for predicting risk and treatment outcomes. This neuroimaging review on alcohol misuse in humans follows the Addictions Neuroclinical Assessment (ANA) that proposes incorporating three functional neuroscience domains integral to the neurocircuitry of addiction: incentive salience and habits, negative emotional states, and executive function within the context of the addiction cycle. Here we review and integrate multiple imaging modalities focusing on underlying cognitive processes such as reward anticipation, negative emotionality, cue reactivity, impulsivity, compulsivity and executive function. We highlight limitations in the literature and propose a model forward in the use of neuroimaging as a tool to understanding underlying mechanisms and potential clinical applicability for phenotyping of heterogeneity and predicting risk and treatment outcomes.

Adaptive learning under expected and unexpected uncertainty

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

The neurochemical substrates of habitual and goal-directed control

Our daily decisions are governed by the arbitration between goal-directed and habitual strategies. However, the neurochemical basis of this arbitration is unclear. We assessed the contribution of dopaminergic, serotonergic, and opioidergic systems to this balance across reward and loss domains. Thirty-nine participants (17 healthy controls, 15 patients with pathological gambling, and 7 with binge eating disorder) underwent positron emission tomography (PET) imaging with [¹⁸F]FDOPA, [¹¹C]MADAM and [¹¹C]carfentanil to assess presynaptic dopamine, and serotonin transporter and mu-opioid receptor binding potential. Separately, participants completed a modified two-step task, which quantifies the degree to which decision-making is influenced by goal-directed or habitual strategies. All participants completed a version with reward outcomes; healthy controls additionally completed a version with loss outcomes. In the context of rewarding outcomes, we found that greater serotonin transporter binding potential in prefrontal regions was associated with habitual control, while greater serotonin transporter binding potential in the putamen was marginally associated with goal-directed control; however, the findings were no longer significant when controlling for the opposing valence (loss). In blocks with loss outcomes, we found that the opioidergic system, specifically greater [¹¹C]carfentanil binding potential, was positively associated with goal-directed control and negatively associated with habit-directed control. Our findings illuminate the complex neurochemical basis of goal-directed and habitual behavior, implicating differential roles for prefrontal and subcortical serotonin in decision-making across healthy and pathological populations.