The Impact of Cannabis Use on Cognition in People with HIV: Evidence of Function-Dependent Effects and Mechanisms from Clinical and Preclinical Studies

PURPOSE OF REVIEW

Cannabis may have beneficial anti-inflammatory effects in people with HIV (PWH); however, given this population's high burden of persisting neurocognitive impairment (NCI), clinicians are concerned they may be particularly vulnerable to the deleterious effects of cannabis on cognition. Here, we present a systematic scoping review of clinical and preclinical studies evaluating the effects of cannabinoid exposure on cognition in HIV.

RECENT FINDINGS

Results revealed little evidence to support a harmful impact of cannabis use on cognition in HIV, with few eligible preclinical data existing. Furthermore, the beneficial/harmful effects of cannabis use observed on cognition were function-dependent and confounded by several factors (e.g., age, frequency of use). Results are discussed alongside potential mechanisms of cannabis effects on cognition in HIV (e.g., anti-inflammatory), and considerations are outlined for screening PWH that may benefit from cannabis interventions. We further highlight the value of accelerating research discoveries in this area by utilizing translatable cross-species tasks to facilitate comparisons across human and animal work.

Reviewing explore/exploit decision-making as a transdiagnostic target for psychosis, depression, and anxiety

In many everyday decisions, individuals choose between trialling something novel or something they know well. Deciding when to try a new option or stick with an option that is already known to you, known as the "explore/exploit" dilemma, is an important feature of cognition that characterises a range of decision-making contexts encountered by humans. Recent evidence has suggested preferences in explore/exploit biases are associated with psychopathology, although this has typically been examined within individual disorders. The current review examined whether explore/exploit decision-making represents a promising transdiagnostic target for psychosis, depression, and anxiety. A systematic search of academic databases was conducted, yielding a total of 29 studies. Studies examining psychosis were mostly consistent in showing that individuals with psychosis explored more compared with individuals without psychosis. The literature on anxiety and depression was more heterogenous; some studies found that anxiety and depression were associated with more exploration, whereas other studies demonstrated reduced exploration in anxiety and depression. However, examining a subset of studies that employed case-control methods, there was some evidence that both anxiety and depression also were associated with increased exploration. Due to the heterogeneity across the literature, we suggest that there is insufficient evidence to conclude whether explore/exploit decision-making is a transdiagnostic target for psychosis, depression, and anxiety. However, alongside our advisory groups of lived experience advisors, we suggest that this context of decision-making is a promising candidate that merits further investigation using well-powered, longitudinal designs. Such work also should examine whether biases in explore/exploit choices are amenable to intervention.

Ketamine produces no detectable long-term positive or negative effects on cognitive flexibility or reinforcement learning of male rats

RATIONALE

Patients with major depressive disorder (MDD) often experience abnormalities in behavioral adaptation following environmental changes (i.e., cognitive flexibility) and tend to undervalue positive outcomes but overvalue negative outcomes. The probabilistic reversal learning task (PRL) is used to study these deficits across species and to explore drugs that may have therapeutic value. Selective serotonin-reuptake inhibitors (SSRIs) have limited effectiveness in treating MDD and produce inconsistent effects in non-human versions of the PRL. As such, ketamine, a novel and potentially rapid-acting therapeutic, has begun to be examined using the PRL. Two previous studies examining the effects of ketamine in the PRL have shown conflicting results and only examined short-term effects of ketamine.

OBJECTIVE

This experiment examined PRL performance across a 2-week period following a single exposure to a ketamine dose that varied across groups.

METHODS

After five sessions of PRL training, groups of rats received an injection of either 0, 10, 20 or 30 mg/kg ketamine. One-hour post-injection, rats engaged in the PRL, and subsequently sessions continued daily for 2 weeks. Traditional behavioral and computational reinforcement learning-derived measures were examined.

RESULTS

Results showed that ketamine had acute effects 1-h post-injection, including a significant decrease in the value of the punishment learning rate. Beyond 1 h, ketamine produced no detectable improvements nor decrements in performance across 2 weeks.

CONCLUSION

Overall, the present results suggest that the range of ketamine doses examined do not have long-term positive or negative effects on cognitive flexibility or reward processing in healthy rats as measured by the PRL.

How much data do we need to estimate computational models of decision-making? The COMPASS toolbox

How much data are needed to obtain useful parameter estimations from a computational model? The standard approach to address this question is to carry out a goodness-of-recovery study. Here, the correlation between individual-participant true and estimated parameter values determines when a sample size is large enough. However, depending on one's research question, this approach may be suboptimal, potentially leading to sample sizes that are either too small (underpowered) or too large (overcostly or unfeasible). In this paper, we formulate a generalized concept of statistical power and use this to propose a novel approach toward determining how much data is needed to obtain useful parameter estimates from a computational model. We describe a Python-based toolbox (COMPASS) that allows one to determine how many participants are needed to fit one specific computational model, namely the Rescorla-Wagner model of learning and decision-making. Simulations revealed that a high number of trials per person (more than the number of persons) are a prerequisite for high-powered studies in this particular setting.

Comparable roles for serotonin in rats and humans for computations underlying flexible decision-making

Serotonin is critical for adapting behavior flexibly to meet changing environmental demands. Cognitive flexibility is important for successful attainment of goals, as well as for social interactions, and is frequently impaired in neuropsychiatric disorders, including obsessive-compulsive disorder. However, a unifying mechanistic framework accounting for the role of serotonin in behavioral flexibility has remained elusive. Here, we demonstrate common effects of manipulating serotonin function across two species (rats and humans) on latent processes supporting choice behavior during probabilistic reversal learning, using computational modelling. The findings support a role of serotonin in behavioral flexibility and plasticity, indicated, respectively, by increases or decreases in choice repetition ('stickiness') or reinforcement learning rates following manipulations intended to increase or decrease serotonin function. More specifically, the rate at which expected value increased following reward and decreased following punishment (reward and punishment 'learning rates') was greatest after sub-chronic administration of the selective serotonin reuptake inhibitor (SSRI) citalopram (5 mg/kg for 7 days followed by 10 mg/kg twice a day for 5 days) in rats. Conversely, humans given a single dose of an SSRI (20 mg escitalopram), which can decrease post-synaptic serotonin signalling, and rats that received the neurotoxin 5,7-dihydroxytryptamine (5,7-DHT), which destroys forebrain serotonergic neurons, exhibited decreased reward learning rates. A basic perseverative tendency ('stickiness'), or choice repetition irrespective of the outcome produced, was likewise increased in rats after the 12-day SSRI regimen and decreased after single dose SSRI in humans and 5,7-DHT in rats. These common effects of serotonergic manipulations on rats and humans-identified via computational modelling-suggest an evolutionarily conserved role for serotonin in plasticity and behavioral flexibility and have clinical relevance transdiagnostically for neuropsychiatric disorders.

Factual and counterfactual learning in major adolescent depressive disorder, evidence from an instrumental learning study

BACKGROUND

The incidence of adolescent depressive disorder is globally skyrocketing in recent decades, albeit the causes and the decision deficits depression incurs has yet to be well-examined. With an instrumental learning task, the aim of the current study is to investigate the extent to which learning behavior deviates from that observed in healthy adolescent controls and track the underlying mechanistic channel for such a deviation.

METHODS

We recruited a group of adolescents with major depression and age-matched healthy control subjects to carry out the learning task with either gain or loss outcome and applied a reinforcement learning model that dissociates valence (positive v. negative) of reward prediction error and selection (chosen v. unchosen).

RESULTS

The results demonstrated that adolescent depressive patients performed significantly less well than the control group. Learning rates suggested that the optimistic bias that overall characterizes healthy adolescent subjects was absent for the depressive adolescent patients. Moreover, depressed adolescents exhibited an increased pessimistic bias for the counterfactual outcome. Lastly, individual difference analysis suggested that these observed biases, which significantly deviated from that observed in normal controls, were linked with the severity of depressive symoptoms as measured by HAMD scores.

CONCLUSIONS

By leveraging an incentivized instrumental learning task with computational modeling within a reinforcement learning framework, the current study reveals a mechanistic decision-making deficit in adolescent depressive disorder. These findings, which have implications for the identification of behavioral markers in depression, could support the clinical evaluation, including both diagnosis and prognosis of this disorder.

Neuropsychological Effects of Antidepressants: Translational Studies

Pharmacological treatments that improve mood were first identified serendipitously, but more than half a century later, how these drugs induce their antidepressant effects remains largely unknown. With the help of animal models, a detailed understanding of their pharmacological targets and acute and chronic effects on brain chemistry and neuronal function has been achieved, but it remains to be elucidated how these effects translate to clinical efficacy. Whilst the field has been dominated by the monoamine and neurotrophic hypotheses, the idea that the maladaptive cognitive process plays a critical role in the development and perpetuation of mood disorders has been discussed since the 1950s. Recently, studies using objective methods to quantify changes in emotional processing found acute effects with conventional antidepressants in both healthy volunteers and patients. These positive effects on emotional processing and cognition occur without a change in the subjective ratings of mood. Building from these studies, behavioural methods for animals that quantify similar cognitive affective processes have been developed. Integrating these behavioural approaches with pharmacology and targeted brain manipulations, a picture is beginning to emerge of the underlying mechanisms that may link the pharmacology of antidepressants, these neuropsychological constructs and clinical efficacy. In this chapter, we discuss findings from animal studies, experimental medicine and patients investigating the neuropsychological effects of antidepressant drugs. We discuss the possible neural circuits that contribute to these effects and discuss whether a neuropsychological model of antidepressant effects could explain the temporal differences in clinical benefits observed with conventional delayed-onset antidepressants versus rapid-acting antidepressants.

Leveraging Decision Science to Characterize Depression

This brief review examines the potential to use decision science to objectively characterize depression. We provide a brief overview of the existing literature examining different domains of decision-making in depression. Because this overview highlights the specific role of reinforcement learning as an important decision process affected in the disorder, we then introduce reinforcement learning modeling and explain how this approach has identified specific reinforcement learning deficits in depression. We conclude with ideas for future research at the intersection of decision science and depression, emphasizing the potential for decision science to help uncover underlying mechanisms and targets for the treatment of depression.

The right superior temporal gyrus plays a role in semantic-rule learning: Evidence supporting a reinforcement learning model

In real-life communication, individuals use language that carries evident rewarding and punishing elements, such as praise and criticism. A common trend is to seek more praise while avoiding criticism. Furthermore, semantics is crucial for conveying information, but such semantic access to native and foreign languages is subtly distinct. To investigate how rule learning occurs in different languages and to highlight the importance of semantics in this process, we investigated both verbal and non-verbal rule learning in first (L1) and second (L2) languages using a reinforcement learning framework, including a semantic rule and a color rule. Our computational modeling on behavioral and brain imaging data revealed that individuals may be more motivated to learn and adhere to rules in an L1 compared to L2, with greater striatum activation during the outcome phase in the L1. Additionally, results on the learning rates and inverse temperature in the two rule learning tasks showed that individuals tend to be conservative and are reluctant to change their judgments regarding rule learning of semantic information. Moreover, the greater the prediction errors, the greater activation of the right superior temporal gyrus in the semantic-rule learning condition, demonstrating that such learning has differential neural correlates than symbolic rule learning. Overall, the findings provide insight into the neural mechanisms underlying rule learning in different languages, and indicate that rule learning involving verbal semantics is not a general symbolic learning that resembles a conditioned stimulus-response, but rather has its own specific characteristics.

"The wrong tools for the right job": a critical meta-analysis of traditional tests to assess behavioural impacts of maternal separation

RATIONALE

Unconditioned tasks in rodents have been the mainstay of behavioural assessment for decades, but their validity and sensitivity to detect the behavioural consequences of early life stress (ELS) remains contentious and highly variable.

OBJECTIVES

In the present study, we carried out a meta-analysis to investigate whether persistent behavioural effects, as assessed using unconditioned procedures in rats, are a reliable consequence of early repeated maternal separation, a commonly used procedure in rodents to study ELS.

METHODS

A literature search identified 100 studies involving maternally separated rats and the following unconditioned procedures: the elevated plus maze (EPM); open field test (OFT); sucrose preference test (SPT) and forced swim task (FST). Studies were included for analysis if the separation of offspring from the dam was at least 60 min every day during the pre-weaning period prior to the start of adolescence.

RESULTS

Our findings show that unconditioned tasks are generally poor at consistently demonstrating differences between control and separated groups with pooled effect sizes that were either small or non-existent (EPM: Hedge's g =  - 0.35, p = 0.01, OFT: Hedge's g =  - 0.32, p = 0.05, SPT: Hedge's g =  - 0.33, p = 0.21, FST: Hedge's g = 0.99, p = 0.0001). Despite considerable procedural variability between studies, heterogeneity statistics were low; indicating the lack of standardization in the maternal separation protocol was the not the cause of these inconsistent effects.

CONCLUSIONS

Our findings indicate that in general, unconditioned tests of depression and anxiety are not sufficient to reveal the full behavioural repertoire of maternal separation stress should not be relied upon in isolation. We argue that more objective tasks that sensitively detect specific cognitive processes are better suited for translational research on stress-related disorders such as depression.

Inattentive responding can induce spurious associations between task behaviour and symptom measures

Although online samples have many advantages for psychiatric research, some potential pitfalls of this approach are not widely understood. Here we detail circumstances in which spurious correlations may arise between task behaviour and symptom scores. The problem arises because many psychiatric symptom surveys have asymmetric score distributions in the general population, meaning that careless responders on these surveys will show apparently elevated symptom levels. If these participants are similarly careless in their task performance, this may result in a spurious association between symptom scores and task behaviour. We demonstrate this pattern of results in two samples of participants recruited online (total N = 779) who performed one of two common cognitive tasks. False-positive rates for these spurious correlations increase with sample size, contrary to common assumptions. Excluding participants flagged for careless responding on surveys abolished the spurious correlations, but exclusion based on task performance alone was less effective.

Importance of additional behavioral observation in psychopharmacology: a case study on agomelatine's effects on feedback sensitivity in probabilistic reversal learning in rats

Since the second half of the twentieth century, many important discoveries in the field of behavioral psychopharmacology have been made using operant conditioning cages. These cages provide objective data collection and have revolutionized behavioral research. Unfortunately, in the rush towards automation, many mistakes may have been made that could have been avoided by observing experimental animals. The study described in this paper is an excellent example of how important additional behavioral observation can be for interpreting instrumental data. In this study, we evaluated the effects of single injections of 3 different doses of agomelatine (5, 10, and 40 mg/kg) on feedback sensitivity in rats. To this end, we tested 40 animals in the instrumental probabilistic reversal learning task in a Latin square design. The highest applied dose of agomelatine, prima facie, reduced the sensitivity of rats to negative feedback - an effect that can be considered antidepressant. However, additional behavioral observation dramatically changed the interpretation of the results and revealed that the perceived effect of agomelatine on sensitivity to negative feedback can actually be attributed to drug-induced drowsiness.

Explaining reversal learning deficits in anxiety with electrophysiological evidence

Reversal learning is a crucial aspect of behavioral flexibility that plays a significant role in environmental adaptation and development. While previous studies have established a link between anxiety and impaired reversal learning ability, the underlying mechanisms behind this association remain unclear. This study employed a probabilistic reversal learning task with electroencephalographic recording to investigate these mechanisms. Participants were divided into two groups based on their scores on Spielberger's State-Trait Anxiety Inventory: high trait-anxiety (HTA) and low trait-anxiety (LTA), consisting of 50 individuals in each group. The results showed that the HTA group had poorer reversal learning performance than the LTA group, including a lower tendency to shift to the new optimal option after rule reversals (reversal-shift). The study also examined event-related potentials elicited by reversals and found that although the N1 (related to attention allocation), feedback-related negativity (FRN: related to belief updating), and P3 (related to response inhibition) were all sensitive to the grouping factor, only the FRN elicited by reversal-shift mediated the relationship between anxiety and the number/reaction time of reversal-shift. From these findings, we suggest that abnormalities in belief updating may contribute to the impaired reversal learning performance observed in anxious individuals. In our opinion, this study sheds light on potential targets for interventions aimed at improving behavioral flexibility in anxious individuals.

History of major depressive disorder is associated with differences in implicit learning of emotional faces

Major depressive disorder is often associated with worsened reward learning, with blunted reward response persisting after remission. In this study, we developed a probabilistic learning task with social rewards as a learning signal. We examined the impacts of depression on social rewards (facial affect displays) as an implicit learning signal. Fifty-seven participants without a history of depression and sixty-two participants with a history of depression (current or remitted) completed a structured clinical interview and an implicit learning task with social reward. Participants underwent an open-ended interview to evaluate whether they knew the rule consciously. Linear mixed effects models revealed that participants without a history of depression learned faster and showed a stronger preference towards the positive than the negative stimulus when compared to the participants with a history of depression. In contrast, those with a history depression learned slower on average and displayed greater variability in stimulus preference. We did not detect any differences in learning between those with current and remitted depression. The results indicate that on a probabilistic social reward task, people with a history of depression exhibit slower reward learning and greater variability in their learning behavior. Improving our understanding of alterations in social reward learning and their associations with depression and anhedonia may help to develop translatable psychotherapeutic approaches for modification of maladaptive emotion regulation.

Reduced sensitivity but intact motivation to monetary rewards and reversal learning in obesity

BACKGROUND

Obesity has been linked to altered reward processing but little is known about which components of reward processing including motivation, sensitivity and learning are impaired in obesity. We examined whether obesity compared to healthy weight controls is associated with differences in distinct subdomains of reward processing. To this end, we used two established paradigms, namely the Effort Expenditure for Rewards task (EEfRT) and the Probabilistic Reversal Learning Task (PRLT).

METHODS

30 individuals with obesity (OBS) and 30 healthy weight control subjects (HC) were included in the study. Generalized estimating equation models were used to analyze EEfRT choice behavior. PRLT data was analyzed using both conventional behavioral variables of choices and computational models.

RESULTS

Our findings from the different tasks speak in favor of a hyposensitivity to non-food rewards in obesity. OBS did not make fewer overall hard task selections compared to HC in the EEfRT suggesting generally intact non-food reward motivation. However, in highly rewarding trials (i.e.,trials with high reward magnitude and high reward probability),OBSmadefewer hard task selections compared to normal weight subjects suggesting decreased sensitivity to highly rewarding non-food reinforcers. Hyposensitivity to non-food rewards was also evident in OBS in the PRLT as evidenced by lower win-stay probability compared to HC. Our computational modelling analyses revealed decreased stochasticity but intact reward and punishment learning rates in OBS.

CONCLUSIONS

Our findings provide evidence for intact reward motivation and learning in OBS but lower reward sensitivity which is linked to stochasticity of choices in a non-food context. These findings might provide further insight into the mechanism underlying dysfunctional choices in obesity.

Early-life stress biases responding to negative feedback and increases amygdala volume and vulnerability to later-life stress

Early-life stress (ELS) or adversity, particularly in the form of childhood neglect and abuse, is associated with poor mental and physical health outcomes in adulthood. However, whether these relationships are mediated by the consequences of ELS itself or by other exposures that frequently co-occur with ELS is unclear. To address this question, we carried out a longitudinal study in rats to isolate the effects of ELS on regional brain volumes and behavioral phenotypes relevant to anxiety and depression. We used the repeated maternal separation (RMS) model of chronic ELS, and conducted behavioral measurements throughout adulthood, including of probabilistic reversal learning (PRL), responding on a progressive ratio task, sucrose preference, novelty preference, novelty reactivity, and putative anxiety-like behavior on the elevated plus maze. Our behavioral assessment was combined with magnetic resonance imaging (MRI) for quantitation of regional brain volumes at three time points: immediately following RMS, young adulthood without further stress, and late adulthood with further stress. We found that RMS caused long-lasting, sexually dimorphic biased responding to negative feedback on the PRL task. RMS also slowed response time on the PRL task, but without this directly impacting task performance. RMS animals were also uniquely sensitive to a second stressor, which disproportionately impaired their performance and slowed their responding on the PRL task. MRI at the time of the adult stress revealed a larger amygdala volume in RMS animals compared with controls. These behavioral and neurobiological effects persisted well into adulthood despite a lack of effects on conventional tests of 'depression-like' and 'anxiety-like' behavior, and a lack of any evidence of anhedonia. Our findings indicate that ELS has long-lasting cognitive and neurobehavioral effects that interact with stress in adulthood and may have relevance for understanding the etiology of anxiety and depression in humans.

Development of cross-species translational paradigms for psychiatric research in the Research Domain Criteria era

The past 30 years of IBNS has included research attempting to treat the cognitive and behavioral deficits observed in people with psychiatric conditions. Early work utilized drugs identified from tests thought to be cognition-relevant, however the high failure rate crossing the translational-species barrier led to focus on developing valid cross-species translational tests. The face, predictive, and neurobiological validities used to assess animal models of psychiatry can be used to validate these tests. Clinical sensitivity is another important aspect however, for if the clinical population targeted for treatment does not exhibit task deficits, then why develop treatments? This review covers some work validating cross-species translational tests and suggests future directions. Also covered is the contribution IBNS made to fostering such research and my role in IBNS, making it more available to all including fostering mentor/mentee programs plus spearheading diversity and inclusivity initiatives. All science needs support and IBNS has supported research recreating the behavioral abnormalities that define psychiatric conditions with the aim to improve the lives of people with such conditions.

Anhedonia and sensitivity to punishment in schizophrenia, depression and opiate use disorder

BACKGROUND

From a behavioural perspective anhedonia is defined as diminished interest in the engagement of pleasurable activities. Despite its presence across a range of psychiatric disorders, the cognitive processes that give rise to anhedonia remain unclear.

METHODS

Here we examine whether anhedonia is associated with learning from positive and negative outcomes in patients diagnosed with major depression, schizophrenia and opiate use disorder alongside a healthy control group. Responses in the Wisconsin Card Sorting Test - a task associated with healthy prefrontal cortex function - were fitted to the Attentional Learning Model (ALM) which separates learning from positive and negative feedback.

RESULTS

Learning from punishment, but not from reward, was negatively associated with anhedonia beyond other socio-demographic, cognitive and clinical variables. This impairment in punishment sensitivity was also associated with faster responses following negative feedback, independently of the degree of surprise.

LIMITATIONS

Future studies should test the longitudinal association between punishment sensitivity and anhedonia also in other clinical populations controlling for the effect of specific medications.

CONCLUSIONS

Together the results reveal that anhedonic subjects, because of their negative expectations, are less sensitive to negative feedbacks; this might lead them to persist in actions leading to negative outcomes.

Individuals with problem gambling and obsessive-compulsive disorder learn through distinct reinforcement mechanisms

Obsessive-compulsive disorder (OCD) and pathological gambling (PG) are accompanied by deficits in behavioural flexibility. In reinforcement learning, this inflexibility can reflect asymmetric learning from outcomes above and below expectations. In alternative frameworks, it reflects perseveration independent of learning. Here, we examine evidence for asymmetric reward-learning in OCD and PG by leveraging model-based functional magnetic resonance imaging (fMRI). Compared with healthy controls (HC), OCD patients exhibited a lower learning rate for worse-than-expected outcomes, which was associated with the attenuated encoding of negative reward prediction errors in the dorsomedial prefrontal cortex and the dorsal striatum. PG patients showed higher and lower learning rates for better- and worse-than-expected outcomes, respectively, accompanied by higher encoding of positive reward prediction errors in the anterior insula than HC. Perseveration did not differ considerably between the patient groups and HC. These findings elucidate the neural computations of reward-learning that are altered in OCD and PG, providing a potential account of behavioural inflexibility in those mental disorders.

Modelling mood updating: a proof of principle study

BACKGROUND

Recent developments in computational psychiatry have led to the hypothesis that mood represents an expectation (prior belief) on the likely interoceptive consequences of action (i.e. emotion). This stems from ideas about how the brain navigates its external world by minimising an upper bound on surprisal (free energy) of sensory information and echoes developments in other perceptual domains.

AIMS

In this paper we aim to present a simple partial observable Markov decision process that models mood updating in response to stressful or non-stressful environmental fluctuations while seeking to minimise surprisal in relation to prior beliefs about the likely interoceptive signals experienced with specific actions (attenuating or amplifying stress and pleasure signals).

METHOD

We examine how, by altering these prior beliefs we can model mood updating in depression, mania and anxiety.

RESULTS

We discuss how these models provide a computational account of mood and its related psychopathology and relate it to previous research in reward processing.

CONCLUSIONS

Models such as this can provide hypotheses for experimental work and also open up the potential modelling of predicted disease trajectories in individual patients.

"Don't [ruminate], be happy": A cognitive perspective linking depression and anhedonia

Anhedonia, a lack of pleasure in things an individual once enjoyed, and rumination, the process of perseverative and repetitive attention to specific thoughts, are hallmark features of depression. Though these both contribute to the same debilitating disorder, they have often been studied independently and through different theoretical lenses (e.g., biological vs. cognitive). Cognitive theories and research on rumination have largely focused on understanding negative affect in depression with much less focus on the etiology and maintenance of anhedonia. In this paper, we argue that by examining the relation between cognitive constructs and deficits in positive affect, we may better understand anhedonia in depression thereby improving prevention and intervention efforts. We review the extant literature on cognitive deficits in depression and discuss how these dysfunctions may not only lead to sustained negative affect but, importantly, interfere with an ability to attend to social and environmental cues that could restore positive affect. Specifically, we discuss how rumination is associated to deficits in working memory and propose that these deficits in working memory may contribute to anhedonia in depression. We further argue that analytical approaches such as computational modeling are needed to study these questions and, finally, discuss implications for treatment.

Reinforcement-based responsiveness, depression, and anhedonia: A multi-method investigation of intergenerational risk

Offspring of depressed parents are at an increased risk for depression. Reward- and punishment-based systems might be mechanisms linking maternal outcomes to offspring depression and anhedonia. The current study was designed to investigate the intergenerational relations between maternal markers of reward and punishment responsiveness and their offspring's depression and anhedonia in a community sample of 40 mother (mean age = 44.5; SD = 6.82) and adolescent (mean age = 14.73; SD = 1.25; 52.5% female) dyads. Maternal markers of reward and punishment responsiveness were captured using self-report, behavioral, and neurophysiological methods, and self-reported depression and anhedonia symptoms were used as outcomes among the adolescent offspring. Maternal self-reported reward responsiveness and punishment learning rates were differentially associated with depression across male and female offspring. Regarding anhedonia, maternal punishment learning rate was positively related to adolescent anhedonia regardless of offspring biological sex. Maternal reward learning rate was also positively associated with anhedonia among male offspring. In general, low concurrence across self-report, behavioral, and neurophysiological markers of reward and punishment responsiveness was found. The results from the current study suggest that learning-rates on reinforcement-based behavioral tasks may be important objective markers to consider when evaluating intergenerational risk.

Sex-dependent effects of early life stress on reinforcement learning and limbic cortico-striatal functional connectivity

Major depressive disorder (MDD) is a stress-related condition hypothesized to involve aberrant reinforcement learning (RL) with positive and negative stimuli. The present study investigated whether repeated early maternal separation (REMS) stress, a procedure widely recognized to cause depression-like behaviour, affects how subjects learn from positive and negative feedback. The REMS procedure was implemented by separating male and female rats from their dam for 6 h each day from post-natal day 5-19. Control rat offspring were left undisturbed during this period. Rats were tested as adults for behavioral flexibility and feedback sensitivity on a probabilistic reversal learning task. A computational approach based on RL theory was used to derive latent behavioral variables related to reward learning and flexibility. To assess underlying brain substrates, a seed-based functional MRI connectivity analysis was applied both before and after an additional adulthood stressor in control and REMS rats. Female but not male rats exposed to REMS stress showed increased response 'stickiness' (repeated responses regardless of reward outcome). Following repeated adulthood stress, reduced functional connectivity from the basolateral amygdala (BLA) to the dorsolateral striatum (DLS), cingulate cortex (Cg), and anterior insula (AI) cortex was observed in females. By contrast, control male rats exposed to the second stressor showed impaired learning from negative feedback (i.e., non-reward) and reduced functional connectivity from the BLA to the DLS and AI compared to maternally separated males. RL in male rats exposed to REMS was unaffected. The fMRI data further revealed that connectivity between the mOFC and other prefrontal cortical and subcortical structures was positively correlated with response 'stickiness'. These findings reveal differences in how females and males respond to early life adversity and subsequent stress. These effects may be mediated by functional divergence in resting-state connectivity between the basolateral amygdala and fronto-striatal brain regions.

Decision value signals in the ventromedial prefrontal cortex and motivational and hedonic symptoms across mood and psychotic disorders

Deficits in motivation and pleasure are common across many psychiatric disorders, and manifest as symptoms of amotivation and anhedonia, which are prominent features of both mood and psychotic disorders. Here we provide evidence for an association between neural value signals and symptoms of amotivation and anhedonia across adults with major depression, bipolar disorder, schizophrenia, or no psychiatric diagnosis. We found that value signals in the ventromedial prefrontal cortex (vmPFC) during intertemporal decision-making were dampened in individuals with greater motivational and hedonic deficits, after accounting for primary diagnosis. This relationship remained significant while controlling for diagnosis-specific symptoms of mood and psychosis, such as depression as well as positive and negative symptoms. Our results demonstrate that dysfunction in the vmPFC during value-based decision-making is specifically linked to motivational and hedonic impairments. These findings provide a quantitative neural target for the potential development of novel treatments for amotivation and anhedonia.

Computational markers of experience- but not description-based decision-making are associated with future depressive symptoms in young adults

BACKGROUND

Early prediction of high depressive symptoms is crucial for selective intervention and the minimization of functional impairment. Recent cross-sectional studies indicated decision-making deficits in depression, which may be an important contributor to the disorder. Our goal was to test whether description- and experience-based decision making, two major neuroeconomic paradigms of decision-making under uncertainty, predict future depressive symptoms in young adults.

METHODS

One hundred young adults performed two decision-making tasks, one description-based, in which subjects chose between two gambling options given explicitly stated rewards and their probabilities, and the other experience-based, in which subjects were shown rewards but had to learn the probability of those rewards (or cue-outcome contingencies) via trial-and-error experience. We evaluated subjects' depressive symptoms with BDI-II at baseline (T1) and half a year later (T2).

RESULTS

Comparing subjects with low versus high levels of depressive symptoms at T2 showed that the latter performed worse on the experience- but not description-based task at T1. Computational modeling of the decision-making process suggested that subjects with high levels of depressive symptoms had a more concave utility function, indicating enhanced risk aversion. Furthermore, a more concave utility function at T1 increased the odds of high depressive symptoms at T2, even after controlling depressive symptoms at T1, perceived stress at T2, and several covariates (OR = 0.251, 95% CI [0.085, 0.741]).

CONCLUSIONS

This is the first study to demonstrate a prospective link between experience-based decision-making and depressive symptoms. Our results suggest that enhanced risk aversion in experience-based decision-making may be an important contributor to the development of depressive symptoms.

Clinical and Preclinical Assessments of Anhedonia in Psychiatric Disorders

Anhedonia is a prevalent symptom across many psychiatric disorders. The contemporary scope of anhedonia across various models includes interest, reward anticipation, motivation, effort expenditure, reward valuation, expectation, pleasure, satiation, and learning. In order to further elucidate the impact of anhedonia on treatment outcomes, quality of life, as well as brain function, validated tools to probe the various facets of anhedonia are necessary. This chapter evaluates assessment tools for anhedonia in clinical populations and in animals. Subjective clinical scales have been in use for decades, and as the construct of anhedonia evolved, contemporary scales were developed to integrate these new concepts. Clinical scales are useful for understanding the subjective experience of anhedonia but do not account for objective aspects of anhedonia, including implicit learning. Behavioral tasks that probe responses to rewarding stimuli have been useful to fill this gap and to delineate the specific brain processes underlying facets of anhedonia. Although there have been translational challenges in the assessments of anhedonia and reward deficits from preclinical to clinical (and vice versa), the multifaceted clinical scales and reward tasks provide valuable insights into the conceptualization of anhedonia and its neural basis across psychiatric disorders.

Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis

IMPORTANCE

Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent.

OBJECTIVE

To assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals.

DATA SOURCES

Web of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood).

STUDY SELECTION

Studies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models.

DATA EXTRACTION AND SYNTHESIS

Articles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected.

MAIN OUTCOMES AND MEASURES

The primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate).

RESULTS

A total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004).

CONCLUSIONS AND RELEVANCE

The simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders.

Phasic Dopamine Changes and Hebbian Mechanisms during Probabilistic Reversal Learning in Striatal Circuits: A Computational Study

Cognitive flexibility is essential to modify our behavior in a non-stationary environment and is often explored by reversal learning tasks. The basal ganglia (BG) dopaminergic system, under a top-down control of the pre-frontal cortex, is known to be involved in flexible action selection through reinforcement learning. However, how adaptive dopamine changes regulate this process and learning mechanisms for training the striatal synapses remain open questions. The current study uses a neurocomputational model of the BG, based on dopamine-dependent direct (Go) and indirect (NoGo) pathways, to investigate reinforcement learning in a probabilistic environment through a task that associates different stimuli to different actions. Here, we investigated: the efficacy of several versions of the Hebb rule, based on covariance between pre- and post-synaptic neurons, as well as the required control in phasic dopamine changes crucial to achieving a proper reversal learning. Furthermore, an original mechanism for modulating the phasic dopamine changes is proposed, assuming that the expected reward probability is coded by the activity of the winner Go neuron before a reward/punishment takes place. Simulations show that this original formulation for an automatic phasic dopamine control allows the achievement of a good flexible reversal even in difficult conditions. The current outcomes may contribute to understanding the mechanisms for active control of dopamine changes during flexible behavior. In perspective, it may be applied in neuropsychiatric or neurological disorders, such as Parkinson’s or schizophrenia, in which reinforcement learning is impaired.

Transdiagnostic and functional predictors of depression severity and trajectory in the Penn state psychiatry clinical assessment and rating evaluation system (PCARES) registry

BACKGROUND

Timely, accurate diagnosis and subsequent identification of risk factors for depression that is difficult-to-treat can aid in decreasing the burden of depressive illness and reducing probability of future disability. We aimed to identify sociodemographic, clinical, and functional factors that predict depression severity over one year in a real-world, naturalistic, transdiagnostic clinical sample. A subset sample with moderate depression was examined to determine the magnitude of improvement.

METHODS

The Penn State Psychiatry Clinical Assessment and Rating System (PCARES) Registry houses data from systematically-structured patient-reported outcomes and clinical data from an Electronic Medical Record (EMR) gathered during routine clinical care of patients seeking mental health care at a mid-Atlantic clinic. Self-report symptom and functional measures were obtained, and sociodemographic features and clinical diagnoses were extracted from the EMR from 1,766 patients between 2/6/2016 to 9/30/2019. The Patient Health Questionnaire 9 (PHQ-9) depression scale was obtained at each visit. Using a discrete mixture clustering model, the study population was divided into five longitudinal trajectory groups, termed depression severity groups, based on intra-individual PHQ-9 score trajectories over one year. Multinomial logistic regression models were estimated to evaluate associations between characteristics and the likelihood of depression severity group membership. To determine the magnitude of improvement, predictors of the slope of the PHQ-9 trajectory were examined for patients with moderate depression.

RESULTS

The strongest predictors of high depression severity over one year were poor functioning, high transdiagnostic DSM-5 Level 1 crosscutting symptom score, diagnosis of Post-Traumatic Stress Disorder (PTSD), public/self-pay insurance, female gender, and non-White race. Among the subset of patients with moderate depression, strong predictors of improvement were commercial insurance and exposure to trauma; the strongest predictors of worsening were high functional impairment, high transdiagnostic Level 1 symptom score, diagnosis of PTSD, diagnosis of bipolar disorder, and marital status of single or formerly married; depression-specific symptom measures were not predictive.

LIMITATIONS

Limitations include inferring education and income status from zip code level-data, the non-random missingness of data, and the use of diagnoses collected from the electronic medical record.

CONCLUSION

Functional impairment, transdiagnostic measures of symptom burden, and insurance status are strong predictors of depression severity and poor outcome.

Computational analysis of probabilistic reversal learning deficits in male subjects with alcohol use disorder

BACKGROUND

Alcohol use disorder is characterized by perseverative alcohol use despite negative consequences. This hallmark feature of addiction potentially relates to impairments in behavioral flexibility, which can be measured by probabilistic reversal learning (PRL) paradigms. We here aimed to examine the cognitive mechanisms underlying impaired PRL task performance in patients with alcohol use disorder (AUDP) using computational models of reinforcement learning.

METHODS

Twenty-eight early abstinent AUDP and 27 healthy controls (HC) performed an extensive PRL paradigm. We compared conventional behavioral variables of choices (perseveration; correct responses) between groups. Moreover, we fitted Bayesian computational models to the task data to compare differences in latent cognitive variables including reward and punishment learning and choice consistency between groups.

RESULTS

AUDP and HC did not significantly differ with regard to direct perseveration rates after reversals. However, AUDP made overall less correct responses and specifically showed decreased win-stay behavior compared to HC. Interestingly, AUDP showed premature switching after no or little negative feedback but elevated proneness to stay when accumulation of negative feedback would make switching a more optimal option. Computational modeling revealed that AUDP compared to HC showed enhanced learning from punishment, a tendency to learn less from positive feedback and lower choice consistency.

CONCLUSION

Our data do not support the assumption that AUDP are characterized by increased perseveration behavior. Instead our findings provide evidence that enhanced negative reinforcement and decreased non-drug-related reward learning as well as diminished choice consistency underlie dysfunctional choice behavior in AUDP.

Intolerance of Uncertainty and Challenges in Decision-making in Adults with High-Functioning Autism

<p style="text-align: justify;">Individuals with high-functioning autism have difficulties in decision-making in face of incomplete or ambiguous information, particularly in the context of social interaction. Tasks demanding an immediate response or deviation from the usual behavior make them feel excessive anxiety which restricts their social and professional activity. Attempts to camouflage their conservatism to others are one of the risk factors for comorbid depression. Therefore, they avoid new and non-routine situations, thus restricting their own social activity and professional development. On the other hand, insisting on sameness and clarity may give individuals with autism an advantage in long-lasting monotonous tasks. The aim of this review is to consider these symptoms from the perspective of predictive coding. A range of experimental studies has shown that most of the subjects with autism have difficulty in predicting the outcomes based on the cumulative history of interacting with the environment, as well as updating expectations as new evidence becomes available. These peculiarities of the analysis and pragmatic weighting of information may cause the trait intolerance of uncertainty and novelty avoidance of most people with autism.</p>

Investigation of reward learning and feedback sensitivity in non-clinical participants with a history of early life stress

Early life stress (ELS) is an important risk factor for the development of depression. Impairments in reward learning and feedback sensitivity are suggested to be an intermediate phenotype in depression aetiology therefore we hypothesised that healthy adults with a history of ELS would exhibit reward processing deficits independent of any current depressive symptoms. We recruited 64 adults with high levels of ELS and no diagnosis of a current mental health disorder and 65 controls. Participants completed the probabilistic reversal learning task and probabilistic reward task followed by depression, anhedonia, social status, and stress scales. Participants with high levels of ELS showed decreased positive feedback sensitivity in the probabilistic reversal learning task compared to controls. High ELS participants also trended towards possessing a decreased model-free learning rate. This was coupled with a decreased learning ability in the acquisition phase of block 1 following the practice session. Neither group showed a reward induced response bias in the probabilistic reward task however high ELS participants exhibited decreased stimuli discrimination. Overall, these data suggest that healthy participants without a current mental health diagnosis but with high levels of ELS show deficits in positive feedback sensitivity and reward learning in the probabilistic reversal learning task that are distinct from depressed patients. These deficits may be relevant to increased depression vulnerability.

Examining the effects of psychoactive drugs on complex behavioral processes in laboratory animals

Behavioral pharmacology has been aided significantly by the development of innovative cognitive tasks designed to examine complex behavioral processes in laboratory animals. Performance outcomes under these conditions have provided key metrics of drug action which serve to supplement traditional in vivo assays of physiologic and behavioral effects of psychoactive drugs. This chapter provides a primer of cognitive tasks designed to assay different aspects of complex behavior, including learning, cognitive flexibility, memory, attention, motivation, and impulsivity. Both capstone studies and recent publications are highlighted throughout to illustrate task value for two distinct but often interconnected translational strategies. First, task performance in laboratory animals can be utilized to elucidate how drugs of abuse affect complex behavioral processes. Here, the expectation is that adverse effects on such processes will have predictive relevance to consequences that will be experienced by humans. Second, these same task outcomes can be used to evaluate candidate therapeutics. In this case, the extent to which drug doses with medicinal value perturb task performance can contribute critical information for a more complete safety profile appraisal and advance the process of medications development. Methodological and theoretical considerations are discussed and include an emphasis on determining selectivity in drug action on complex behavioral processes.

Psychiatric symptoms influence reward-seeking and loss-avoidance decision-making through common and distinct computational processes

AIM

Psychiatric symptoms are often accompanied by impairments in decision-making to attain rewards and avoid losses. However, due to the complex nature of mental disorders (e.g., high comorbidity), symptoms that are specifically associated with deficits in decision-making remain unidentified. Furthermore, the influence of psychiatric symptoms on computations underpinning reward-seeking and loss-avoidance decision-making remains elusive. Here, we aim to address these issues by leveraging a large-scale online experiment and computational modeling.

METHODS

In the online experiment, we recruited 1900 non-diagnostic participants from the general population. They performed either a reward-seeking or loss-avoidance decision-making task, and subsequently completed questionnaires about psychiatric symptoms.

RESULTS

We found that one trans-diagnostic dimension of psychiatric symptoms related to compulsive behavior and intrusive thought (CIT) was negatively correlated with overall decision-making performance in both the reward-seeking and loss-avoidance tasks. A deeper analysis further revealed that, in both tasks, the CIT psychiatric dimension was associated with lower preference for the options that recently led to better outcomes (i.e. reward or no-loss). On the other hand, in the reward-seeking task only, the CIT dimension was associated with lower preference for recently unchosen options.

CONCLUSION

These findings suggest that psychiatric symptoms influence the two types of decision-making, reward-seeking and loss-avoidance, through both common and distinct computational processes.

Functional connectivity abnormalities underlying mood disturbances in male abstinent methamphetamine abusers

Anxiety and depression are the most common withdrawal symptoms of methamphetamine (METH) abuse, which further exacerbate relapse of METH abuse. To date, no effective pharmacotherapy exists for METH abuse and its withdrawal symptoms. Therefore, understanding the neuromechanism underlying METH abuse and its withdrawal symptoms is essential for developing clinical strategies and improving patient care. The aims of this study were to investigate brain network abnormalities in METH abusers (MAs) and their associations with affective symptoms. Forty‐eight male abstinent MAs and 48 age‐gender matched healthy controls were recruited and underwent resting state functional magnetic resonance imaging (fMRI). The severity of patient anxiety and depressive symptoms were measured by Hamilton anxiety and depression rating scales, which decreased across the duration of abstinence. Independent component analysis was used to investigate the brain network functional connectivity (FC) properties. Compared with healthy controls, MAs demonstrated hypo‐intra‐network FC in the cerebellar network and hyper‐intra‐network FC in the posterior salience network. A whole‐brain regression analysis revealed that FC strength of clusters located in the right rostral anterior cingulate cortex (rACC) within the ventromedial network (VMN) was associated with affective symptoms in the patients. Importantly, the intra‐network FC strength of the rACC in VMN mediated the association between abstinence duration and the severity level of affective symptoms. Our results demonstrate alterations in brain functional networks underlying METH abuse, and that the FC of rACC within VMN serve as a neural substrate in the association between abstinence length and affective symptom severity in the MAs.