Depressive symptoms are associated with blunted reward learning in social contexts

Beta activity in human anterior cingulate cortex mediates reward biases

The rewards that we get from our choices and actions can have a major influence on our future behavior. Understanding how reward biasing of behavior is implemented in the brain is important for many reasons, including the fact that diminution in reward biasing is a hallmark of clinical depression. We hypothesized that reward biasing is mediated by the anterior cingulate cortex (ACC), a cortical hub region associated with the integration of reward and executive control and with the etiology of depression. To test this hypothesis, we recorded neural activity during a biased judgment task in patients undergoing intracranial monitoring for either epilepsy or major depressive disorder. We found that beta (12-30 Hz) oscillations in the ACC predicted both associated reward and the size of the choice bias, and also tracked reward receipt, thereby predicting bias on future trials. We found reduced magnitude of bias in depressed patients, in whom the beta-specific effects were correspondingly reduced. Our findings suggest that ACC beta oscillations may orchestrate the learning of reward information to guide adaptive choice, and, more broadly, suggest a potential biomarker for anhedonia and point to future development of interventions to enhance reward impact for therapeutic benefit.

Contextual influence of reinforcement learning performance of depression: evidence for a negativity bias?

BACKGROUNDS

Value-based decision-making impairment in depression is a complex phenomenon: while some studies did find evidence of blunted reward learning and reward-related signals in the brain, others indicate no effect. Here we test whether such reward sensitivity deficits are dependent on the overall value of the decision problem.

METHODS

We used a two-armed bandit task with two different contexts: one 'rich', one 'poor' where both options were associated with an overall positive, negative expected value, respectively. We tested patients (N = 30) undergoing a major depressive episode and age, gender and socio-economically matched controls (N = 26). Learning performance followed by a transfer phase, without feedback, were analyzed to distangle between a decision or a value-update process mechanism. Finally, we used computational model simulation and fitting to link behavioral patterns to learning biases.

RESULTS

Control subjects showed similar learning performance in the 'rich' and the 'poor' contexts, while patients displayed reduced learning in the 'poor' context. Analysis of the transfer phase showed that the context-dependent impairment in patients generalized, suggesting that the effect of depression has to be traced to the outcome encoding. Computational model-based results showed that patients displayed a higher learning rate for negative compared to positive outcomes (the opposite was true in controls).

CONCLUSIONS

Our results illustrate that reinforcement learning performances in depression depend on the value of the context. We show that depressive patients have a specific trouble in contexts with an overall negative state value, which in our task is consistent with a negativity bias at the learning rates level.

History of major depressive disorder is associated with differences in implicit learning of emotional faces

Major depressive disorder is often associated with worsened reward learning, with blunted reward response persisting after remission. In this study, we developed a probabilistic learning task with social rewards as a learning signal. We examined the impacts of depression on social rewards (facial affect displays) as an implicit learning signal. Fifty-seven participants without a history of depression and sixty-two participants with a history of depression (current or remitted) completed a structured clinical interview and an implicit learning task with social reward. Participants underwent an open-ended interview to evaluate whether they knew the rule consciously. Linear mixed effects models revealed that participants without a history of depression learned faster and showed a stronger preference towards the positive than the negative stimulus when compared to the participants with a history of depression. In contrast, those with a history depression learned slower on average and displayed greater variability in stimulus preference. We did not detect any differences in learning between those with current and remitted depression. The results indicate that on a probabilistic social reward task, people with a history of depression exhibit slower reward learning and greater variability in their learning behavior. Improving our understanding of alterations in social reward learning and their associations with depression and anhedonia may help to develop translatable psychotherapeutic approaches for modification of maladaptive emotion regulation.

Computational models of subjective feelings in psychiatry

Research in computational psychiatry is dominated by models of behavior. Subjective experience during behavioral tasks is not well understood, even though it should be relevant to understanding the symptoms of psychiatric disorders. Here, we bridge this gap and review recent progress in computational models for subjective feelings. For example, happiness reflects not how well people are doing, but whether they are doing better than expected. This dependence on recent reward prediction errors is intact in major depression, although depressive symptoms lower happiness during tasks. Uncertainty predicts subjective feelings of stress in volatile environments. Social prediction errors influence feelings of self-worth more in individuals with low self-esteem despite a reduced willingness to change beliefs due to social feedback. Measuring affective state during behavioral tasks provides a tool for understanding psychiatric symptoms that can be dissociable from behavior. When smartphone tasks are collected longitudinally, subjective feelings provide a potential means to bridge the gap between lab-based behavioral tasks and real-life behavior, emotion, and psychiatric symptoms.

Sex-dependent effects of early life stress on reinforcement learning and limbic cortico-striatal functional connectivity

Major depressive disorder (MDD) is a stress-related condition hypothesized to involve aberrant reinforcement learning (RL) with positive and negative stimuli. The present study investigated whether repeated early maternal separation (REMS) stress, a procedure widely recognized to cause depression-like behaviour, affects how subjects learn from positive and negative feedback. The REMS procedure was implemented by separating male and female rats from their dam for 6 h each day from post-natal day 5-19. Control rat offspring were left undisturbed during this period. Rats were tested as adults for behavioral flexibility and feedback sensitivity on a probabilistic reversal learning task. A computational approach based on RL theory was used to derive latent behavioral variables related to reward learning and flexibility. To assess underlying brain substrates, a seed-based functional MRI connectivity analysis was applied both before and after an additional adulthood stressor in control and REMS rats. Female but not male rats exposed to REMS stress showed increased response 'stickiness' (repeated responses regardless of reward outcome). Following repeated adulthood stress, reduced functional connectivity from the basolateral amygdala (BLA) to the dorsolateral striatum (DLS), cingulate cortex (Cg), and anterior insula (AI) cortex was observed in females. By contrast, control male rats exposed to the second stressor showed impaired learning from negative feedback (i.e., non-reward) and reduced functional connectivity from the BLA to the DLS and AI compared to maternally separated males. RL in male rats exposed to REMS was unaffected. The fMRI data further revealed that connectivity between the mOFC and other prefrontal cortical and subcortical structures was positively correlated with response 'stickiness'. These findings reveal differences in how females and males respond to early life adversity and subsequent stress. These effects may be mediated by functional divergence in resting-state connectivity between the basolateral amygdala and fronto-striatal brain regions.

Functioning of the EROS-R Scale in a Clinical Sample of Psychiatric Patients: New Psychometric Evidence from the Classical Test Theory and the Item Response Theory

Reliable and valid assessment instruments that can be applied briefly and easily in clinical and outpatient settings that provide information about the sources of reinforcement that the patient finds in his life are especially relevant in therapy. The study aimed to evaluate the psychometric properties of the Environmental Reward Observation Scale (EROS-R) in a sample of psychiatric patients. A sample of 228 psychiatric patients of both sexes (56.1% men and 43.9% women) aged between 18 and 70 years was selected. Along with the EROS-R, other instruments were administered to assess depression and anxiety. The results show that the scale fits a unidimensional model, presenting adequate fit indices (RMSEA = 0.077 (IC 90% 0.055−0.100); SRMR = 0.048; CFI = 0.98; TLI = 0.98). It was also shown that the degree of reward provided by the environment (EROS-R) correlates negatively with the level of depression (ρ = −0.54; p < 0.01) and anxiety (ρ = −0.34; p < 0.01). From the IRT perspective, all the items present adequate discrimination indices, where item 4 is the most precise indicator to measure the degree of environmental reward. All this leads us to conclude that the EROS-R is an instrument with robust psychometric guarantees from TCT and IRT’s perspectives, making it suitable for use in clinical contexts.

Measuring Integrated Novel Dimensions in Neurodevelopmental and Stress-Related Mental Disorders (MIND-SET): Protocol for a Cross-sectional Comorbidity Study From a Research Domain Criteria Perspective

BACKGROUND

It is widely acknowledged that comorbidity between psychiatric disorders is common. Shared and diverse underpinnings of psychiatric disorders cannot be systematically understood based on symptom-based categories of mental disorders, which map poorly onto pathophysiological mechanisms. In the Measuring Integrated Novel Dimensions in Neurodevelopmental and Stress-Related Mental Disorders (MIND-SET) study, we make use of current concepts of comorbidity that transcend the current diagnostic categories. We test this approach to psychiatric problems in patients with frequently occurring psychiatric disorders and their comorbidities (excluding psychosis).

OBJECTIVE

The main aim of the MIND-SET project is to determine the shared and specific mechanisms of neurodevelopmental and stress-related psychiatric disorders at different observational levels.

METHODS

This is an observational cross-sectional study. Data from different observational levels as defined in the Research Domain Criteria (genetics, physiology, neuropsychology, system-level neuroimaging, behavior, self-report, and experimental neurocognitive paradigms) are collected over four time points. Included are adult (aged ≥18 years), nonpsychotic, psychiatric patients with a clinical diagnosis of a stress-related disorder (mood disorder, anxiety disorder, or substance use disorder) or a neurodevelopmental disorder (autism spectrum disorder or attention-deficit/hyperactivity disorder). Individuals with no current or past psychiatric diagnosis are included as neurotypical controls. Data collection started in June 2016 with the aim to include a total of 650 patients and 150 neurotypical controls by 2021. The data collection procedure includes online questionnaires and three subsequent sessions with (1) standardized clinical examination, physical examination, and blood sampling; (2) psychological constructs, neuropsychological tests, and biological marker sampling; and (3) neuroimaging measures.

RESULTS

We aim to include a total of 650 patients and 150 neurotypical control participants in the time period between 2016 and 2022. In October 2021, we are at 95% of our target.

CONCLUSIONS

The MIND-SET study enables us to investigate the mechanistic underpinnings of nonpsychotic psychiatric disorders transdiagnostically. We will identify both shared and disorder-specific markers at different observational levels that can be used as targets for future diagnostic and treatment approaches.

Challenging the negative learning bias hypothesis of depression: reversal learning in a naturalistic psychiatric sample

BACKGROUND

Classic theories posit that depression is driven by a negative learning bias. Most studies supporting this proposition used small and selected samples, excluding patients with comorbidities. However, comorbidity between psychiatric disorders occurs in up to 70% of the population. Therefore, the generalizability of the negative bias hypothesis to a naturalistic psychiatric sample as well as the specificity of the bias to depression, remain unclear. In the present study, we tested the negative learning bias hypothesis in a large naturalistic sample of psychiatric patients, including depression, anxiety, addiction, attention-deficit/hyperactivity disorder, and/or autism. First, we assessed whether the negative bias hypothesis of depression generalized to a heterogeneous (and hence more naturalistic) depression sample compared with controls. Second, we assessed whether negative bias extends to other psychiatric disorders. Third, we adopted a dimensional approach, by using symptom severity as a way to assess associations across the sample.

METHODS

We administered a probabilistic reversal learning task to 217 patients and 81 healthy controls. According to the negative bias hypothesis, participants with depression should exhibit enhanced learning and flexibility based on punishment v. reward. We combined analyses of traditional measures with more sensitive computational modeling.

RESULTS

In contrast to previous findings, this sample of depressed patients with psychiatric comorbidities did not show a negative learning bias.

CONCLUSIONS

These results speak against the generalizability of the negative learning bias hypothesis to depressed patients with comorbidities. This study highlights the importance of investigating unselected samples of psychiatric patients, which represent the vast majority of the psychiatric population.

Examining the effect of depressive symptoms on habit formation and habit-breaking

BACKGROUND AND OBJECTIVES

Dysfunction in reward processing is a hallmark feature of depression. In the context of reinforcement learning, previous research has linked depression with reliance on simple habit-driven ('model-free') learning strategies over more complex, goal-directed ('model-based') strategies. However, the relationship between depression and habit-breaking remains an under-explored research area. The current study sought to bridge this gap by investigating the effect of depressive symptoms on habit formation and habit-breaking under monetary and social feedback conditions. Additionally, we examined whether spontaneous eyeblink rate (EBR), an indirect marker for striatal dopamine levels, would modulate such effects.

METHODS

Depressive symptoms were operationalized using self-report measures. To examine differences in habit formation and habit breaking, undergraduate participants (N = 156) completed a two-stage reinforcement learning task with a devaluation procedure using either monetary or social feedback.

RESULTS

Regression results showed that in the monetary feedback condition, spontaneous EBR moderated the relationship between depressive symptoms and model-free strategies; individuals with more depressive symptomatology and high EBR (higher dopamine levels) exhibited increased reliance on model-free strategies. Depressive symptoms negatively predicted devaluation sensitivity, indicative of difficulty in habit-breaking, in both monetary and social feedback contexts.

LIMITATIONS

Social feedback relied on fixed feedback rather than real-time peer evaluations; depressive symptoms were measured using self-report rather than diagnostic criteria for Major Depressive Disorder; dopaminergic functioning was measured using EBR rather than PET imaging; potential confounds were not controlled for.

CONCLUSIONS

These findings have implications for identifying altered patterns of habit formation and deficits in habit-breaking among those experiencing depressive symptoms.

Examining the Effectiveness of Gamification in Mental Health Apps for Depression: Systematic Review and Meta-analysis

BACKGROUND

Previous research showed that computerized cognitive behavioral therapy can effectively reduce depressive symptoms. Some mental health apps incorporate gamification into their app design, yet it is unclear whether features differ in their effectiveness to reduce depressive symptoms over and above mental health apps without gamification.

OBJECTIVE

The aim of this study was to determine whether mental health apps with gamification elements differ in their effectiveness to reduce depressive symptoms when compared to those that lack these elements.

METHODS

A meta-analysis of studies that examined the effect of app-based therapy, including cognitive behavioral therapy, acceptance and commitment therapy, and mindfulness, on depressive symptoms was performed. A total of 5597 articles were identified via five databases. After screening, 38 studies (n=8110 participants) remained for data extraction. From these studies, 50 total comparisons between postintervention mental health app intervention groups and control groups were included in the meta-analysis.

RESULTS

A random effects model was performed to examine the effect of mental health apps on depressive symptoms compared to controls. The number of gamification elements within the apps was included as a moderator. Results indicated a small to moderate effect size across all mental health apps in which the mental health app intervention effectively reduced depressive symptoms compared to controls (Hedges g=-0.27, 95% CI -0.36 to -0.17; P<.001). The gamification moderator was not a significant predictor of depressive symptoms (β=-0.03, SE=0.03; P=.38), demonstrating no significant difference in effectiveness between mental health apps with and without gamification features. A separate meta-regression also did not show an effect of gamification elements on intervention adherence (β=-1.93, SE=2.28; P=.40).

CONCLUSIONS

The results show that both mental health apps with and without gamification elements were effective in reducing depressive symptoms. There was no significant difference in the effectiveness of mental health apps with gamification elements on depressive symptoms or adherence. This research has important clinical implications for understanding how gamification elements influence the effectiveness of mental health apps on depressive symptoms.

Psychometric Properties of the Environmental Reward Observation Scale: Study on Its Internal Structure, Factor Invariance, and Method Effect Associated With Its Negative Items

The aim of this study was to assess the factorial structure of the scale, the method's effect associated with its negative items, its temporal invariance, and factorial invariance according to sex. For this purpose, three samples were collected, an initial sample of 200 participants, a second sample of 461 participants and a third sample of 107 participants; making a total of 768 Peruvian university students. Other instruments were applied together with the EROS scale in order to measure satisfaction with life, anxiety, stress and depression. Regarding the results, in the initial sample it was found that the original scale containing positive and negative items does adequately fit the data (RMSEA = .19; CFI = .77; TLI = .71) and also evidence was found supporting the existence of a methodological effect associated with the negative items. It was also found that version B of the scale which only has positive items data fits the data (RMSEA = .13; CFI = .96; TLI = .95). In the second sample it was found that version B still had a good fit to the data in a larger sample (RMSEA = .07; CFI = .98; TLI = .98). In addition, it was found that the scale can be considered invariant according to sex and presents validity based on other constructs. In the third sample it was found that the test-retest reliability of the scale was adequate (.70 [CI95% .593-.788]) and also evidence was found in favor of the temporal invariance of the scale. It is concluded that the scale formed only by positive items presents more robust psychometric properties and constitutes a better alternative to measure the level of reward provided by the environment.

The actions of others act as a pseudo-reward to drive imitation in the context of social reinforcement learning

While there is no doubt that social signals affect human reinforcement learning, there is still no consensus about how this process is computationally implemented. To address this issue, we compared three psychologically plausible hypotheses about the algorithmic implementation of imitation in reinforcement learning. The first hypothesis, decision biasing (DB), postulates that imitation consists in transiently biasing the learner's action selection without affecting their value function. According to the second hypothesis, model-based imitation (MB), the learner infers the demonstrator's value function through inverse reinforcement learning and uses it to bias action selection. Finally, according to the third hypothesis, value shaping (VS), the demonstrator's actions directly affect the learner's value function. We tested these three hypotheses in 2 experiments (N = 24 and N = 44) featuring a new variant of a social reinforcement learning task. We show through model comparison and model simulation that VS provides the best explanation of learner's behavior. Results replicated in a third independent experiment featuring a larger cohort and a different design (N = 302). In our experiments, we also manipulated the quality of the demonstrators' choices and found that learners were able to adapt their imitation rate, so that only skilled demonstrators were imitated. We proposed and tested an efficient meta-learning process to account for this effect, where imitation is regulated by the agreement between the learner and the demonstrator. In sum, our findings provide new insights and perspectives on the computational mechanisms underlying adaptive imitation in human reinforcement learning.

A Hierarchical Bayesian Implementation of the Experience-Weighted Attraction Model

Social and decision-making deficits are often the first symptoms of neuropsychiatric disorders. In recent years, economic games, together with computational models of strategic learning, have been increasingly applied to the characterization of individual differences in social behavior, as well as their changes across time due to disease progression, treatment, or other factors. At the same time, the high dimensionality of these data poses an important challenge to statistical estimation of these models, potentially limiting the adoption of such approaches in patients and special populations. We introduce a hierarchical Bayesian implementation of a class of strategic learning models, experience-weighted attraction (EWA), that is widely used in behavioral game theory. Importantly, this approach provides a unified framework for capturing between- and within-participant variation, including changes associated with disease progression, comorbidity, and treatment status. We show using simulated data that our hierarchical Bayesian approach outperforms representative agent and individual-level estimation methods that are commonly used in extant literature, with respect to parameter estimation and uncertainty quantification. Furthermore, using an empirical dataset, we demonstrate the value of our approach over competing methods with respect to balancing model fit and complexity. Consistent with the success of hierarchical Bayesian approaches in other areas of behavioral science, our hierarchical Bayesian EWA model represents a powerful and flexible tool to apply to a wide range of behavioral paradigms for studying the interplay between complex human behavior and biological factors.

Integration of social cues and individual experiences during instrumental avoidance learning

Learning to avoid harmful consequences can be a costly trial-and-error process. In such situations, social information can be leveraged to improve individual learning outcomes. Here, we investigated how participants used their own experiences and others' social cues to avoid harm. Participants made repeated choices between harmful and safe options, each with different probabilities of generating shocks, while also seeing the image of a social partner. Some partners made predictive gaze cues towards the harmful choice option while others cued an option at random, and did so using neutral or fearful facial expressions. We tested how learned social information about partner reliability transferred across contexts by letting participants encounter the same partner in multiple trial blocks while facing novel choice options. Participants' decisions were best explained by a reinforcement learning model that independently learned the probabilities of options being safe and of partners being reliable and combined these combined these estimates to generate choices. Advice from partners making a fearful facial expression influenced participants' decisions more than advice from partners with neutral expressions. Our results showed that participants made better decisions when facing predictive partners and that they cached and transferred partner reliability estimates into new blocks. Using simulations we show that participants' transfer of social information into novel contexts is better adapted to variable social environments where social partners may change their cuing strategy or become untrustworthy. Finally, we found no relation between autism questionnaire scores and performance in our task, but do find autism trait related differences in learning rate parameters.

Negative Learning Bias in Depression Revisited: Enhanced Neural Response to Surprising Reward Across Psychiatric Disorders

BACKGROUND

Prior work has proposed that major depressive disorder (MDD) is associated with a specific cognitive bias: patients with depression seem to learn more from punishment than from reward. This learning bias has been associated with blunting of reward-related neural responses in the striatum. A key question is whether negative learning bias is also present in patients with MDD and comorbid disorders and whether this bias is specific to depression or shared across disorders.

METHODS

We employed a transdiagnostic approach assessing a heterogeneous group of (nonpsychotic) psychiatric patients from the MIND-Set (Measuring Integrated Novel Dimensions in Neurodevelopmental and Stress-Related Mental Disorders) cohort with and without MDD but also with anxiety, attention-deficit/hyperactivity disorder, and/or autism (n = 66) and healthy control subjects (n = 24). To investigate reward and punishment learning, we employed a deterministic reversal learning task with functional magnetic resonance imaging.

RESULTS

In contrast to previous studies, patients with MDD did not exhibit impaired reward learning or reduced reward-related neural activity anywhere in the brain. Interestingly, we observed consistently increased neural responses in the bilateral lateral prefrontal cortex of patients when they received a surprising reward. This increase was not specific to MDD, but generalized to anxiety, attention-deficit/hyperactivity disorder, and autism. Critically, increased prefrontal activity to surprising reward scaled with transdiagnostic symptom severity, particularly that associated with concentration and attention, as well as the number of diagnoses; patients with more comorbidities showed a stronger prefrontal response to surprising reward.

CONCLUSIONS

Prefrontal enhancement may reflect compensatory working memory recruitment, possibly to counteract the inability to swiftly update reward expectations. This neural mechanism may provide a candidate transdiagnostic index of psychiatric severity.

Negativity-bias in forming beliefs about own abilities

During everyday interactions people constantly receive feedback on their behavior, which shapes their beliefs about themselves. While classic studies in the field of social learning suggest that people have a tendency to learn better from good news (positivity bias) when they perceive little opportunities to immediately improve their own performance, we show updating is biased towards negative information when participants perceive the opportunity to adapt their performance during learning. In three consecutive experiments we applied a computational modeling approach on the subjects' learning behavior and reveal the negativity bias was specific for learning about own compared to others' performances and was modulated by prior beliefs about the self, i.e. stronger negativity bias in individuals lower in self-esteem. Social anxiety affected self-related negativity biases only when individuals were exposed to a judging audience thereby potentially explaining the persistence of negative self-images in socially anxious individuals which commonly surfaces in social settings. Self-related belief formation is therefore surprisingly negatively biased in situations suggesting opportunities to improve and this bias is shaped by trait differences in self-esteem and social anxiety.