Impaired Expected Value Computations Coupled With Overreliance on Stimulus-Response Learning in Schizophrenia

The challenge of learning adaptive mental behavior

Many psychotherapies aim to help people replace maladaptive mental behaviors (such as those leading to unproductive worry) with more adaptive ones (such as those leading to active problem solving). Yet, little is known empirically about how challenging it is to learn adaptive mental behaviors. Mental behaviors entail taking mental operations and thus may be more challenging to perform than motor actions; this challenge may enhance or impair learning. In particular, challenge when learning is often desirable because it improves retention. Yet, it is also plausible that the necessity of carrying out mental operations interferes with learning the expected values of mental actions by impeding credit assignment: the process of updating an action's value after reinforcement. Then, it may be more challenging not only to perform-but also to learn the consequences of-mental (vs. motor) behaviors. We designed a task to assess learning to take adaptive mental versus motor actions via matched probabilistic feedback. In two experiments (N = 300), most participants found it more difficult to learn to select optimal mental (vs. motor) actions, as evident in worse accuracy not only in a learning but also test (retention) phase. Computational modeling traced this impairment to an indicator of worse credit assignment (impaired construction and maintenance of expected values) when learning mental actions, accounting for worse accuracy in the learning and retention phases. The results suggest that people have particular difficulty learning adaptive mental behavior and pave the way for novel interventions to scaffold credit assignment and promote adaptive thinking. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Reduced learning bias towards the reward context in medication-naive first-episode schizophrenia patients

BACKGROUND

Reinforcement learning has been proposed to contribute to the development of amotivation in individuals with schizophrenia (SZ). Accumulating evidence suggests dysfunctional learning in individuals with SZ in Go/NoGo learning and expected value representation. However, previous findings might have been confounded by the effects of antipsychotic exposure. Moreover, reinforcement learning also rely on the learning context. Few studies have examined the learning performance in reward and loss-avoidance context separately in medication-naïve individuals with first-episode SZ. This study aimed to explore the behaviour profile of reinforcement learning performance in medication-naïve individuals with first-episode SZ, including the contextual performance, the Go/NoGo learning and the expected value representation performance.

METHODS

Twenty-nine medication-naïve individuals with first-episode SZ and 40 healthy controls (HCs) who have no significant difference in age and gender, completed the Gain and Loss Avoidance Task, a reinforcement learning task involving stimulus pairs presented in both the reward and loss-avoidance context. We assessed the group difference in accuracy in the reward and loss-avoidance context, the Go/NoGo learning and the expected value representation. The correlations between learning performance and the negative symptom severity were examined.

RESULTS

Individuals with SZ showed significantly lower accuracy when learning under the reward than the loss-avoidance context as compared to HCs. The accuracies under the reward context (90%win- 10%win) in the Acquisition phase was significantly and negatively correlated with the Scale for the Assessment of Negative Symptoms (SANS) avolition scores in individuals with SZ. On the other hand, individuals with SZ showed spared ability of Go/NoGo learning and expected value representation.

CONCLUSIONS

Despite our small sample size and relatively modest findings, our results suggest possible reduced learning bias towards reward context among medication-naïve individuals with first-episode SZ. The reward learning performance was correlated with amotivation symptoms. This finding may facilitate our understanding of the underlying mechanism of negative symptoms. Reinforcement learning performance under the reward context may be important to better predict and prevent the development of schizophrenia patients' negative symptom, especially amotivation.

Hot and cold cognitive disturbances in antidepressant-free patients with major depressive disorder: a NeuroPharm study

BACKGROUND

Cognitive disturbances are common and disabling features of major depressive disorder (MDD). Previous studies provide limited insight into the co-occurrence of hot (emotion-dependent) and cold (emotion-independent) cognitive disturbances in MDD. Therefore, we here map both hot and cold cognition in depressed patients compared to healthy individuals.

METHODS

We collected neuropsychological data from 92 antidepressant-free MDD patients and 103 healthy controls. All participants completed a comprehensive neuropsychological test battery assessing hot cognition including emotion processing, affective verbal memory and social cognition as well as cold cognition including verbal and working memory and reaction time.

RESULTS

The depressed patients showed small to moderate negative affective biases on emotion processing outcomes, moderate increases in ratings of guilt and shame and moderate deficits in verbal and working memory as well as moderately slowed reaction time compared to healthy controls. We observed no correlations between individual cognitive tasks and depression severity in the depressed patients. Lastly, an exploratory cluster analysis suggested the presence of three cognitive profiles in MDD: one characterised predominantly by disturbed hot cognitive functions, one characterised predominantly by disturbed cold cognitive functions and one characterised by global impairment across all cognitive domains. Notably, the three cognitive profiles differed in depression severity.

CONCLUSION

We identified a pattern of small to moderate disturbances in both hot and cold cognition in MDD. While none of the individual cognitive outcomes mapped onto depression severity, cognitive profile clusters did. Overall cognition-based stratification tools may be useful in precision medicine approaches to MDD.

Mechanisms Underlying Motivational Dysfunction in Schizophrenia

Negative symptoms are a debilitating feature of schizophrenia which are often resistant to pharmacological intervention. The mechanisms underlying them remain poorly understood, and diagnostic methods rely on phenotyping through validated questionnaires. Deeper endo-phenotyping is likely to be necessary in order to improve current understanding. In the last decade, valuable behavioural insights have been gained through the use of effort-based decision making (EBDM) tasks. These have highlighted impairments in reward-related processing in schizophrenia, particularly associated with negative symptom severity. Neuroimaging investigations have related these changes to dysfunction within specific brain networks including the ventral striatum (VS) and frontal brain regions. Here, we review the behavioural and neural evidence associated with negative symptoms, shedding light on potential underlying mechanisms and future therapeutic possibilities. Findings in the literature suggest that schizophrenia is characterised by impaired reward based learning and action selection, despite preserved hedonic responses. Associations between amotivation and reward-processing deficits have not always been clear, and may be mediated by factors including cognitive dysfunction or dysfunctional or self-defeatist beliefs. Successful endo-phenotyping of negative symptoms as a function of objective behavioural and neural measurements is crucial in advancing our understanding of this complex syndrome. Additionally, transdiagnostic research–leveraging findings from other brain disorders, including neurological ones–can shed valuable light on the possible common origins of motivation disorders across diseases and has important implications for future treatment development.

Computational approaches and machine learning for individual-level treatment predictions

RATIONALE

The impact of neuroscience-based approaches for psychiatry on pragmatic clinical decision-making has been limited. Although neuroscience has provided insights into basic mechanisms of neural function, these insights have not improved the ability to generate better assessments, prognoses, diagnoses, or treatment of psychiatric conditions.

OBJECTIVES

To integrate the emerging findings in machine learning and computational psychiatry to address the question: what measures that are not derived from the patient's self-assessment or the assessment by a trained professional can be used to make more precise predictions about the individual's current state, the individual's future disease trajectory, or the probability to respond to a particular intervention?

RESULTS

Currently, the ability to use individual differences to predict differential outcomes is very modest possibly related to the fact that the effect sizes of interventions are small. There is emerging evidence of genetic and neuroimaging-based heterogeneity of psychiatric disorders, which contributes to imprecise predictions. Although the use of machine learning tools to generate clinically actionable predictions is still in its infancy, these approaches may identify subgroups enabling more precise predictions. In addition, computational psychiatry might provide explanatory disease models based on faulty updating of internal values or beliefs.

CONCLUSIONS

There is a need for larger studies, clinical trials using machine learning, or computational psychiatry model parameters predictions as actionable outcomes, comparing alternative explanatory computational models, and using translational approaches that apply similar paradigms and models in humans and animals.

Retention of Value Representations Across Time in People With Schizophrenia and Healthy Control Subjects

BACKGROUND

The current study aimed to further etiological understanding of the psychological mechanisms underlying negative symptoms in people with schizophrenia. Specifically, we tested whether negative symptom severity is associated with reduced retention of reward-related information over time and thus a degraded ability to utilize such information to guide future action selection.

METHODS

Forty-four patients with a diagnosis of schizophrenia or schizoaffective disorder and 28 healthy control volunteers performed a probabilistic reinforcement-learning task involving stimulus pairs in which choices resulted in reward or in loss avoidance. Following training, participants indicated their valuation of learned stimuli in a test/transfer phase. The test/transfer phase was administered immediately following training and 1 week later. Percent retention was defined as accuracy at week-long delay divided by accuracy at immediate delay.

RESULTS

Healthy control subjects and people with schizophrenia showed similarly robust retention of reinforcement learning over a 1-week delay interval. However, in the schizophrenia group, negative symptom severity was associated with reduced retention of information regarding the value of actions across a week-long interval. This pattern was particularly notable for stimuli associated with reward compared with loss avoidance.

CONCLUSIONS

Our results show that although individuals with schizophrenia may initially learn about rewarding aspects of their environment, such learning decays at a more rapid rate in patients with severe negative symptoms. Thus, previously learned reward-related information may be more difficult to access to guide future decision making and to motivate action selection.

Impaired Expected Value Computations in Schizophrenia Are Associated With a Reduced Ability to Integrate Reward Probability and Magnitude of Recent Outcomes

BACKGROUND

Motivational deficits in people with schizophrenia (PSZ) are associated with an inability to integrate the magnitude and probability of previous outcomes. The mechanisms that underlie probability-magnitude integration deficits, however, are poorly understood. We hypothesized that increased reliance on "valueless" stimulus-response associations, in lieu of expected value (EV)-based learning, could drive probability-magnitude integration deficits in PSZ with motivational deficits.

METHODS

Healthy volunteers (n = 38) and PSZ (n = 49) completed a learning paradigm consisting of four stimulus pairs. Reward magnitude (3, 2, 1, 0 points) and probability (90%, 80%, 20%, 10%) determined each stimulus's EV. Following a learning phase, new and familiar stimulus pairings were presented. Participants were asked to select stimuli with the highest reward value.

RESULTS

PSZ with high motivational deficits made increasingly less optimal choices as the difference in reward value (probability × magnitude) between two competing stimuli increased. Using a previously validated computational hybrid model, PSZ relied less on EV ("Q-learning") and more on stimulus-response learning ("actor-critic"), which correlated with Scale for the Assessment of Negative Symptoms motivational deficit severity. PSZ specifically failed to represent reward magnitude, consistent with model demonstrations showing that response tendencies in the actor-critic were preferentially driven by reward probability.

CONCLUSIONS

Probability-magnitude deficits in PSZ with motivational deficits arise from underutilization of EV in favor of reliance on valueless stimulus-response associations. Confirmed by our computational hybrid framework, probability-magnitude integration deficits were driven specifically by a failure to represent reward magnitude. This work provides a first mechanistic explanation of complex EV-based learning deficits in PSZ with motivational deficits that arise from an inability to combine information from different reward modalities.