Effects of subclinical depression on prefrontal-striatal model-based and model-free learning

Disorders of compulsivity: Deficits in arbitrating learning strategies

While previous research has shown that compulsivity is related to an imbalance between goal-directed and habitual learning systems, very little is known about whether this effect is due to the impairment of a single system or the impairment of the arbitration mechanism that determines which system controls behaviour at any given moment; the current study aims to address this disagreement. Nineteen alcohol use disorder, 30 obsessive-compulsive disorder (OCD) and 20 major depressive disorder patients and corresponding sex- and age-matched controls performed two-choice, three-stage Markov decision-making paradigm. Model-based and mode-free reinforcement learning models were used to independently fitted their behavioural data. Alcohol use disorder and OCD patients showed less model-based strategy choice than healthy controls in task conditions where the model-based strategy was optimal. Only OCD patients showed higher behavioural control system switching in task conditions where model-free use was optimal. Major depressive disorder patients did not differ from the matched control in both. These findings suggest that dysfunction in arbitration control between dual systems may be the basis for diverse disorders involving compulsivity.

A feasibility study of goal-directed network-based real-time fMRI neurofeedback for anhedonic depression

Anhedonia is a hallmark symptom of depression that often lacks adequate interventions. The translational gap remains in clinical treatments based on neural substrates of anhedonia. Our pilot study found that depressed individuals depended less on goal-directed (GD) reward learning (RL), with reduced reward prediction error (RPE) BOLD signal. Previous studies have found that anhedonia is related to abnormal activities and/or functional connectivities of the central executive network (CEN) and salience network (SN), both of which belong to the goal-directed system. In addition, it was found that real-time functional magnetic resonance imaging (rt-fMRI) neurofeedback (NF) could improve the balance between CEN and SN in healthy individuals. Therefore, we speculate that rt-fMRI NF of the CEN and SN associated with the GD system may improve depressive and/or anhedonic symptoms. Therefore, this study (1) will examine individuals with anhedonic depression using GD-RL behavioral task, combined with functional magnetic resonance imaging and computational modeling to explore the role of CEN/SN deficits in anhedonic depression; and (2) will utilize network-based rt-fMRI NF to investigate whether it is feasible to regulate the differential signals of brain CEN/SN of GD system through rt-fMRI NF to alleviate depressive and/or anhedonic symptoms. This study highlights the need to elucidate the intervention effects of rt-fMRI NF and the underlying computational network neural mechanisms.

Impairment of arbitration between model-based and model-free reinforcement learning in obsessive-compulsive disorder

Introduction

Obsessive-compulsive disorder (OCD) is characterized by an imbalance between goal-directed and habitual learning systems in behavioral control, but it is unclear whether these impairments are due to a single system abnormality of the goal-directed system or due to an impairment in a separate arbitration mechanism that selects which system controls behavior at each point in time.

Methods

A total of 30 OCD patients and 120 healthy controls performed a 2-choice, 3-stage Markov decision-making paradigm. Reinforcement learning models were used to estimate goal-directed learning (as model-based reinforcement learning) and habitual learning (as model-free reinforcement learning). In general, 29 high Obsessive-Compulsive Inventory-Revised (OCI-R) score controls, 31 low OCI-R score controls, and all 30 OCD patients were selected for the analysis.

Results

Obsessive-compulsive disorder (OCD) patients showed less appropriate strategy choices than controls regardless of whether the OCI-R scores in the control subjects were high (p = 0.012) or low (p < 0.001), specifically showing a greater model-free strategy use in task conditions where the model-based strategy was optimal. Furthermore, OCD patients (p = 0.001) and control subjects with high OCI-R scores (H-OCI-R; p = 0.009) both showed greater system switching rather than consistent strategy use in task conditions where model-free use was optimal.

Conclusion

These findings indicated an impaired arbitration mechanism for flexible adaptation to environmental demands in both OCD patients and healthy individuals reporting high OCI-R scores.

Rethinking model-based and model-free influences on mental effort and striatal prediction errors

A standard assumption in neuroscience is that low-effort model-free learning is automatic and continuously used, whereas more complex model-based strategies are only used when the rewards they generate are worth the additional effort. We present evidence refuting this assumption. First, we demonstrate flaws in previous reports of combined model-free and model-based reward prediction errors in the ventral striatum that probably led to spurious results. More appropriate analyses yield no evidence of model-free prediction errors in this region. Second, we find that task instructions generating more correct model-based behaviour reduce rather than increase mental effort. This is inconsistent with cost-benefit arbitration between model-based and model-free strategies. Together, our data indicate that model-free learning may not be automatic. Instead, humans can reduce mental effort by using a model-based strategy alone rather than arbitrating between multiple strategies. Our results call for re-evaluation of the assumptions in influential theories of learning and decision-making.

Pupillary response in reward processing in adults with major depressive disorder in remission

OBJECTIVE

Major depressive disorder (MDD) is associated with impaired reward processing and reward learning. The literature is inconclusive regarding whether these impairments persist after remission. The current study examined reward processing during a probabilistic learning task in individuals in remission from MDD (n = 19) and never depressed healthy controls (n = 31) matched for age and sex. The outcome measures were pupil dilation (an indirect index of noradrenergic activity and arousal) and computational modeling parameters.

METHOD

Participants completed two versions (facial/nonfacial feedback) of probabilistic reward learning task with changing contingencies. Pupil dilation was measured with a corneal reflection eye tracker. The hypotheses and analysis plan were preregistered.

RESULT

Healthy controls had larger pupil dilation following losses than gains (p <.001), whereas no significant difference between outcomes was found in individuals with a history of MDD, resulting in an interaction between group and outcome (β = 0.81, SE = 0.34, t = 2.37, p = .018). The rMDD group also achieved lower mean score at the last trial (t[46.77] = 2.12, p = .040) as well as a smaller proportion of correct choices (t[46.70] = 2.09, p = .041) compared with healthy controls.

CONCLUSION

Impaired reward processing may persist after remission from MDD and could constitute a latent risk factor for relapse. Measuring pupil dilation in a reward learning task is a promising method for identifying reward processing abnormalities linked to MDD. The task is simple and noninvasive, which makes it feasible for clinical research.

Functional MRI study of feedback-based reinforcement learning in depression

Reinforcement learning depends upon the integrity of emotional circuitry to establish associations between environmental cues, decisions, and positive or negative outcomes in order to guide behavior through experience. The emotional dysregulation characteristic of major depressive disorder (MDD) may alter activity in frontal and limbic structures that are key to learning. Although reward and decision-making have been examined in MDD, the effects of depression on associative learning is less well studied. We investigated whether depressive symptoms would be related to abnormalities in learning-related brain activity as measured by functional magnetic resonance imaging (fMRI). Also, we explored whether melancholic and atypical features were associated with altered brain activity. We conducted MRI scans on a 4T Varian MRI system in 10 individuals with MDD and 10 healthy subjects. We examined event-related brain activation during feedback-based learning task using Analysis of Functional NeuroImages (AFNI) for image processing and statistical analysis. We observed that MDD patients exhibited reduced activation in visual cortex but increased activation in cingulate and insular regions compared to healthy participants. Also, in relation to features of depressive subtypes, we observed that levels of activation in striatal, thalamic, and precuneus regions were negatively correlated with atypical characteristics. These results suggest that the effects of MDD change the neural circuitry underlying associative learning, and these effects may depend upon subtype features of MDD.