Explicit and implicit reinforcement learning across the psychosis spectrum

Modeling Decision-Making in Schizophrenia: Associations Between Computationally Derived Risk Propensity and Self-Reported Risk Perception

BACKGROUND AND HYPOTHESIS

Schizophrenia is associated with a decreased pursuit of risky rewards during uncertain-risk decision-making. However, putative mechanisms subserving this disadvantageous risky reward pursuit, such as contributions of cognition and relevant traits, remain poorly understood.

STUDY DESIGN

Participants (30 schizophrenia/schizoaffective disorder [SZ]; 30 comparison participants [CP]) completed the Balloon Analogue Risk Task (BART). Computational modeling captured subprocesses of uncertain-risk decision-making: Risk Propensity, Prior Belief of Success, Learning Rate, and Behavioral Consistency. IQ, self-reported risk-specific processes (ie, Perceived Risks and Expected Benefit of Risks), and non-risk-specific traits (ie, defeatist beliefs; hedonic tone) were examined for relationships with Risk Propensity to determine what contributed to differences in risky reward pursuit.

STUDY RESULTS

On the BART, the SZ group exhibited lower Risk Propensity, higher Prior Beliefs of Success, and comparable Learning Rates. Furthermore, Risk Propensity was positively associated with IQ across groups. Linear models predicting Risk Propensity revealed 2 interactions: 1 between group and Perceived Risk, and 1 between IQ and Perceived Risk. Specifically, in both the SZ group and individuals with below median IQ, lower Perceived Risks was related to lower Risk Propensity. Thus, lower perception of financial risks was associated with a less advantageous pursuit of uncertain-risk rewards.

CONCLUSIONS

Findings suggest consistently decreased risk-taking on the BART in SZ may reflect risk imperception, the failure to accurately perceive and leverage relevant information to guide the advantageous pursuit of risky rewards. Additionally, our results highlight the importance of cognition in uncertain-risk decision-making.

Motor precision deficits in clinical high risk for psychosis

Motor deficits appear prior to psychosis onset, provide insight into vulnerability as well as mechanisms that give rise to emerging illness, and are predictive of conversion. However, to date, the extant literature has often targeted a complex abnormality (e.g., gesture dysfunction, dyskinesia), or a single fundamental domain (e.g., accuracy) but rarely provided critical information about several of the individual components that make up more complex behaviors (or deficits). This preliminary study applies a novel implicit motor task to assess domains of motor accuracy, speed, recognition, and precision in individuals at clinical high risk for psychosis (CHR-p). Sixty participants (29 CHR-p; 31 healthy volunteers) completed clinical symptom interviews and a novel Serial Interception Sequence Learning (SISL) task that assessed implicit motor sequence accuracy, speed, precision, and explicit sequence recognition. These metrics were examined in multilevel models that enabled the examination of overall effects and changes in motor domains over blocks of trials and by positive/negative symptom severity. Implicit motor sequence accuracy, speed, and explicit sequence recognition were not detected as impacted in CHR-p. When compared to healthy controls, individuals at CHR-p were less precise in motor responses both overall (d = 0.91) and particularly in early blocks which normalized over later blocks. Within the CHR-p group, these effects were related to positive symptom levels (t = - 2.22, p = 0.036), such that individuals with higher symptom levels did not improve in motor precision over time (r's = 0.01-0.05, p's > 0.54). CHR-p individuals showed preliminary evidence of motor precision deficits but no other motor domain deficits, particularly in early performance that normalized with practice.

Social and non-social feedback stimuli lead to comparable levels of reward learning and reward responsiveness in an online probabilistic reward task

Social stimuli seem to be processed more easily and efficiently than non-social stimuli. The current study tested whether social feedback stimuli improve reward learning in a probabilistic reward task (PRT), in which one response option is usually rewarded more often than the other via presentation of non-social reward stimuli. In a pre-registered online study with 305 participants, 75 participants were presented with a non-social feedback stimulus (a star) and information about gains, which is typically used in published PRT studies. Three other groups (with 73-82 participants each) were presented with one of three social feedback stimuli: verbal praise, an attractive happy face, or a "thumbs up"-picture. The data were analysed based on classical signal detection theory, drift diffusion modelling, and Bayesian analyses of null effects. All PRT variants yielded the expected behavioural preference for the more frequently rewarded response. There was no processing advantage of social over non-social feedback stimuli. Bayesian analyses further supported the observation that social feedback stimuli neither increased nor decreased behavioural preferences in the PRT. The current findings suggest that the PRT is a robust experimental paradigm independent of the applied feedback stimuli. They also suggest that the occurrence of a processing advantage for social feedback stimuli is dependent on the experimental task and design.

How to study psychological mechanisms of mania? A systematic review on the methodology of experimental studies on manic mood dysregulation of leading theories on bipolar disorder

INTRODUCTION

Although there are several psychological theories on bipolar disorders (BD), the empirical evidence on these theories through experimental studies is still limited. The current study systematically reviews experimental methods used in studies on the main theories of BD: Reward Hypersensitivity Theory (RST) or Behavioral Activation System (BAS), Integrative Cognitive Model (ICM), Positive Emotion Persistence (PEP), Manic Defense theory (MD), and Mental Imagery (MI). The primary aim is to provide an overview of the used methods and to identify limitations and suggest areas of improvement.

METHODS

A systematic search of six databases until October 2023 was conducted. Study selection involved two independent reviewers extracting data on experimental study design and methodology.

RESULTS

A total of 84 experimental studies were reviewed. BAS and RST were the most frequently studied theories. The majority of these experimental studies focus on mechanisms of reward sensitivity. Other important elements of the reviewed theories, such as goal setting and-attainment, situation selection (avoidance or approach), activation, affective/emotional reactivity, and regulatory strategies, are understudied. Self-report and neuropsychological tasks are most often used, while mood induction and physiological measures are rarely used.

CONCLUSION

There is a need for more consensus on the operationalization of psychological theories of mania. Standardization of test batteries could improve comparability among studies and foster a more systematic approach to experimental research. Research on affective (activated) states is still underrepresented in comparison with studies on trait vulnerabilities.

Pupillary responses as a biomarker of cognitive effort and the impact of task difficulty on reward processing in schizophrenia

Negative symptoms of schizophrenia robustly predict functional outcomes but remain relatively resistant to available treatments. Better measures of negative symptoms, especially motivational deficits, are needed to better understand these symptoms and improve treatment development. Recent research shows promise in linking behavioral effort tasks to motivational negative symptoms, reward processing deficits, and defeatist attitudes, but few studies account for individual or group (patient v. control) differences in cognitive ability to perform the tasks. Individuals with poorer abilities might be less motivated to perform tasks because they find them more difficult to perform. This study used a personalized digit span task to control task difficulty while measuring task effort via pupillary responses (greater dilation indicates greater cognitive effort) at varying monetary rewards ($1 & $2). Participants with schizophrenia (N = 34) and healthy controls (N = 41) performed a digit span task with personalized max span lengths and easy (max- 2 digits) and overload (max+ 2 digits) conditions. Consistent with many studies, pupillary responses (cognitive effort) increased with greater difficulty until exceeding capacity. A similar pattern of reward responsivity was seen in both groups, such that greater reward increased dilation (effort) comparably for both groups when difficulty was within capacity. Neither patients nor controls exerted increased effort for greater reward when difficulty exceeded capacity. In patients, positive relationships were found between pupil dilation and defeatist performance beliefs if task difficulty was within capacity; a relationship that reversed if the task was too difficult. The findings demonstrate the importance of accounting for cognitive capacity and task difficulty when evaluating motivation and reward sensitivity and illustrate the utility of pupillary responses as an objective measure of effort in schizophrenia.

Differential deficits in social versus monetary reinforcement learning in schizophrenia: Associations with facial emotion recognition

Despite evidence that individuals with schizophrenia (SZ) have an intact desire for social relationships, they have small social networks and report high levels of loneliness. Difficulty with reinforcement learning (RL), the ability to update behavior based on feedback, may inhibit the formation and maintenance of social relationships in SZ. However, impaired RL in SZ has largely been demonstrated via monetary tasks. Thus, it remains unclear whether SZ are similarly impaired in social and monetary RL, or whether social-specific factors may further inhibit their ability to learn from social feedback. Thirty-one individuals with SZ and 31 healthy controls (HCs) participated in a RL paradigm to test hypotheses about social versus monetary RL. SZ exhibited impaired RL compared to HCs in both social and monetary tasks. Further, a Group × Task interaction demonstrated that SZ was more impaired when learning from social than monetary reinforcement, F(1, 59) = 5.99, p = .017. This differential deficit to social RL was not accounted for by reported pleasure from social feedback, which did not differ between groups. Instead, SZ had poorer emotion recognition than HCs, t(1, 60) = 4.80, p < .001, particularly for negative emotions, and controlling for this eliminated the differential social RL impairment. These results suggest the possibility that difficulty recognizing social cues, especially those indicating negative feedback, may relate to a reduced ability to learn from others' feedback. Thus, future research could elucidate whether targeting these emotion recognition difficulties in treatment could serve as a potential mechanism for improving social functioning in SZ. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Reinforcement learning profiles and negative symptoms across chronic and clinical high-risk phases of psychotic illness

Negative symptoms are prominent in individuals with schizophrenia (SZ) and youth at clinical high-risk for psychosis (CHR). In SZ, negative symptoms are linked to reinforcement learning (RL) dysfunction; however, previous research suggests implicit RL remains intact. It is unknown whether implicit RL is preserved in the CHR phase where negative symptom mechanisms are unclear, knowledge of which may assist in developing early identification and prevention methods. Participants from two studies completed an implicit RL task: Study 1 included 53 SZ individuals and 54 healthy controls (HC); Study 2 included 26 CHR youth and 23 HCs. Bias trajectories reflecting implicit RL were compared between groups and correlations with negative symptoms were examined. Cluster analysis investigated RL profiles across the combined samples. Implicit RL was comparable between HC and their corresponding SZ and CHR groups. However, cluster analysis was able to parse performance heterogeneity across diagnostic boundaries into two distinct RL profiles: a Positive/Early Learning cluster (65% of participants) with positive bias scores increasing from the first to second task block, and a Negative/Late Learning cluster (35% of participants) with negative bias scores increasing from the second to third block. Clusters did not differ in the proportion of CHR vs. SZ cases; however, the Negative/Late Learning cluster had more severe negative symptoms. Although implicit RL is intact in CHR similar to SZ, distinct implicit RL phenotypic profiles with elevated negative symptoms were identified trans-phasically, suggesting distinct reward-processing mechanisms can contribute to negative symptoms independent of phases of illness.

Transdiagnostic Predictors of Everyday Functioning: Examining the Relationships of Depression and Reinforcement Learning

BACKGROUND AND HYPOTHESIS

Impairments in function (ie, the ability to independently accomplish daily tasks) have been established in psychotic disorders. Identifying factors that contribute to these deficits is essential to developing effective interventions. The current study had several goals: examine potential differential relationships across domains of neurocognition, assess whether reinforcement learning is related to function, identify if predictors of function are transdiagnostic, determine whether depression and positive symptoms contribute to function, and to explore whether the modality of assessment impacts observed relationships.

STUDY DESIGN

Data from 274 participants were examined with schizophrenia/schizoaffective disorder (SZ; n = 195) and bipolar disorder (BD; n = 79). To reduce dimensionality, a PCA was completed on neurocognitive tasks which resulted in 3 components. These components and clinical interview data were used to investigate predictors of functional domains across measures of function (self- and informant-report SLOF and UPSA).

RESULTS

Two components, working memory/processing speed/episodic memory (βs = 0.18-0.42), and negative/positive reinforcement learning (β = -0.04), predicted different functional domains. Predictors of function were largely transdiagnostic with two exceptions: reinforcement learning had a positive association with self-reported interpersonal relationships for SZ and a negative association for BD (β = 0.34), and the negative association between positive symptoms and self-reported social acceptability was stronger for BD than for SZ (β = 0.93). Depression robustly predicted self-reported but not informant-reported function, and anhedonia predicted all domains of informant-reported function.

CONCLUSIONS

These findings imply that reinforcement learning may differentially relate to function across disorders, traditional domains of neurocognition can be effective transdiagnostic targets for interventions, and positive symptoms and depression play a critical role in self-perceived functional impairments.

A neuro-computational social learning framework to facilitate transdiagnostic classification and treatment across psychiatric disorders

Social deficits are among the core and most striking psychiatric symptoms, present in most psychiatric disorders. Here, we introduce a novel social learning framework, which consists of neuro-computational models that combine reinforcement learning with various types of social knowledge structures. We outline how this social learning framework can help specify and quantify social psychopathology across disorders and provide an overview of the brain regions that may be involved in this type of social learning. We highlight how this framework can specify commonalities and differences in the social psychopathology of individuals with autism spectrum disorder (ASD), personality disorders (PD), and major depressive disorder (MDD) and improve treatments on an individual basis. We conjecture that individuals with psychiatric disorders rely on rigid social knowledge representations when learning about others, albeit the nature of their rigidity and the behavioral consequences can greatly differ. While non-clinical cohorts tend to efficiently adapt social knowledge representations to relevant environmental constraints, psychiatric cohorts may rigidly stick to their preconceived notions or overly coarse knowledge representations during learning.

Negative Symptom Inventory-Self-Report (NSI-SR): Initial development and validation

Negative symptoms (i.e., anhedonia, avolition, asociality, blunted affect, alogia) are frequently observed in the schizophrenia-spectrum (SZ) and associated with functional disability. While semi-structured interviews of negative symptoms represent a gold-standard approach, they require specialized training and may be vulnerable to rater biases. Thus, brief self-report questionnaires measuring negative symptoms may be useful. Existing negative symptom questionnaires demonstrate that this approach may be promising in schizophrenia, but no measure has been devised for use across stages of psychotic illness. The present study reports initial psychometric validation of the Negative Symptom Inventory-Self-Report (NSI-SR), the self-report counterpart of the Negative Symptom Inventory-Psychosis Risk clinical interview. The NSI-SR is a novel transphasic negative symptoms measure assessing the domains of anhedonia, avolition, and asociality. The NSI-SR and related measures were administered to two samples: 1) undergraduates (n = 335), 2) community participants, including: SZ (n = 32), clinical-high risk for psychosis (CHR, n = 25), and healthy controls matched to SZ (n = 31) and CHR (n = 30). The psychometrically trimmed 11-item NSI-SR showed good internal consistency and a three-factor solution reflecting avolition, asociality, and anhedonia. The NSI-SR demonstrated convergent validity via moderate to large correlations with clinician-rated negative symptoms and related constructs in both samples. Discriminant validity was supported by lower correlations with positive symptoms in both samples; however, correlations with positive symptoms were still significant. These initial psychometric findings suggest that the NSI-SR is a reliable and valid brief questionnaire capable of measuring negative symptoms across phases of psychotic illness.

Aberrant reward processing to positive versus negative outcomes across psychotic disorders

Several studies of reward processing in schizophrenia have shown reduced sensitivity to positive, but not negative, outcomes although inconsistencies have been reported. In addition, few studies have investigated whether patients show a relative deficit to social versus nonsocial rewards, whether deficits occur across the spectrum of psychosis, or whether deficits relate to negative symptoms and functioning. This study examined probabilistic implicit learning via two visually distinctive slot machines for social and nonsocial rewards in 101 outpatients with diverse psychotic disorders and 48 community controls. The task consisted of two trial types: positive (optimal to choose a positive vs. neutral machine) and negative (optimal to choose a neutral vs. negative machine), with two reward conditions: social (faces) and nonsocial (money) reward conditions. A significant group X trial type interaction indicated that controls performed better on positive than negative trials, whereas patients showed the opposite pattern of better performance on negative than positive trials. In addition, both groups performed better for social than nonsocial stimuli, despite lower overall task performance in patients. Within patients, worse performance on negative trials showed significant, small-to-moderate correlations with motivation and pleasure-related negative symptoms and social functioning. The current findings suggest reward processing disturbances, particularly decreased sensitivity to positive outcomes, extend beyond schizophrenia to a broader spectrum of psychotic disorders and relate to important clinical outcomes.

Cognitive [Computational] Neuroscience Test Reliability and Clinical Applications for Serious Mental Illness (CNTRaCS) Consortium: Progress and Future Directions

The development of treatments for impaired cognition in schizophrenia has been characterized as the most important challenge facing psychiatry at the beginning of the twenty-first century. The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) project was designed to build on the potential benefits of using tasks and tools from cognitive neuroscience to better understanding and treat cognitive impairments in psychosis. These benefits include: (1) the use of fine-grained tasks that measure discrete cognitive processes; (2) the ability to design tasks that distinguish between specific cognitive domain deficits and poor performance due to generalized deficits resulting from sedation, low motivation, poor test taking skills, etc.; and (3) the ability to link cognitive deficits to specific neural systems, using animal models, neuropsychology, and functional imaging. CNTRICS convened a series of meetings to identify paradigms from cognitive neuroscience that maximize these benefits and identified the steps need for translation into use in clinical populations. The Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRaCS) Consortium was developed to help carry out these steps. CNTRaCS consists of investigators at five different sites across the country with diverse expertise relevant to a wide range of the cognitive systems identified as critical as part of CNTRICs. This work reports on the progress and current directions in the evaluation and optimization carried out by CNTRaCS of the tasks identified as part of the original CNTRICs process, as well as subsequent extensions into the Positive Valence systems domain of Research Domain Criteria (RDoC). We also describe the current focus of CNTRaCS, which involves taking a computational psychiatry approach to measuring cognitive and motivational function across the spectrum of psychosis. Specifically, the current iteration of CNTRaCS is using computational modeling to isolate parameters reflecting potentially more specific cognitive and visual processes that may provide greater interpretability in understanding shared and distinct impairments across psychiatric disorders.

Human Variation in Error-Based and Reinforcement Motor Learning Is Associated With Entorhinal Volume

Error-based and reward-based processes are critical for motor learning and are thought to be mediated via distinct neural pathways. However, recent behavioral work in humans suggests that both learning processes can be bolstered by the use of cognitive strategies, which may mediate individual differences in motor learning ability. It has been speculated that medial temporal lobe regions, which have been shown to support motor sequence learning, also support the use of cognitive strategies in error-based and reinforcement motor learning. However, direct evidence in support of this idea remains sparse. Here we first show that better overall learning during error-based visuomotor adaptation is associated with better overall learning during the reward-based shaping of reaching movements. Given the cognitive contribution to learning in both of these tasks, these results support the notion that strategic processes, associated with better performance, drive intersubject variation in both error-based and reinforcement motor learning. Furthermore, we show that entorhinal cortex volume is larger in better learning individuals-characterized across both motor learning tasks-compared with their poorer learning counterparts. These results suggest that individual differences in learning performance during error and reinforcement learning are related to neuroanatomical differences in entorhinal cortex.

Preserved Unconscious Processing in Schizophrenia: The Case of Motivation

BACKGROUND AND HYPOTHESIS

Motivation deficit is a hallmark of schizophrenia that has a strong impact on their daily life. An alteration of reward processing has been repeatedly highlighted in schizophrenia, but to what extent it involves a deficient amplification of reward representation through conscious processing remains unclear. Indeed, patients with schizophrenia exhibit a disruption of conscious processing, whereas unconscious processing appears to be largely preserved.

STUDY DESIGN

To further explore the nature of motivational deficit in schizophrenia and the implication of consciousness disruption in this symptom, we used a masking paradigm testing motivation both under conscious and unconscious conditions in patients with schizophrenia (n = 31) and healthy controls (n = 32). Participants were exposed to conscious or subliminal coin pictures representing money at stake and were subsequently asked to perform an effort-task by squeezing a handgrip as hard as possible to win this reward.

STUDY RESULTS

We observed a preserved effect of unconscious monetary rewards on force production in both groups, without any significant difference between them. By contrast, in the conscious condition, patients with schizophrenia were less sensitive to rewards than controls. Our results confirm that unconscious incentives have effects on exerted forces in the general population, and demonstrate that patients with schizophrenia exhibit a dissociation between an impaired conscious motivation and a preserved unconscious motivation.

CONCLUSIONS

These findings suggest the existence of several steps in motivational processes that can be differentially affected and might have implication for patient care.

Research Domain Criteria (RDoC): Progress and Potential

The National Institute of Mental Health (NIMH) addressed in its 2008 Strategic Plan an emerging concern that the current diagnostic system was hampering translational research, as accumulating data suggested that disorder categories constituted heterogeneous syndromes rather than specific diseases. However, established practices in peer review placed high priority on extant disorders in evaluating grant applications for mental illness. To provide guidelines for alternative study designs, NIMH included a goal to develop new ways of studying psychopathology based on dimensions of measurable behavior and related neurobiological measures. The Research Domain Criteria (RDoC) project is the result, intended to build a literature that informs new conceptions of mental illness and future revisions to diagnostic manuals. The framework calls for the study of empirically-derived fundamental dimensions as characterized by related behavioral/psychological and neurobiological data (e.g., reward valuation, working memory). RDoC also emphasizes full-range dimensional approaches (from typical to increasingly abnormal), neurodevelopment and environmental effects, and research designs that integrate data across behavioral, biological, and self-report measures. This commentary provides an overview of the project's first decade and its potential future directions. RDoC remains grounded in experimental psychopathology perspectives, and its progress is strongly linked to psychological measurement and integrative approaches to brain-behavior relationships.

Both unmedicated and medicated individuals with schizophrenia show impairments across a wide array of cognitive and reinforcement learning tasks

BACKGROUND

Schizophrenia is a disorder characterized by pervasive deficits in cognitive functioning. However, few well-powered studies have examined the degree to which cognitive performance is impaired even among individuals with schizophrenia not currently on antipsychotic medications using a wide range of cognitive and reinforcement learning measures derived from cognitive neuroscience. Such research is particularly needed in the domain of reinforcement learning, given the central role of dopamine in reinforcement learning, and the potential impact of antipsychotic medications on dopamine function.

METHODS

The present study sought to fill this gap by examining healthy controls (N = 75), unmedicated (N = 48) and medicated (N = 148) individuals with schizophrenia. Participants were recruited across five sites as part of the CNTRaCS Consortium to complete tasks assessing processing speed, cognitive control, working memory, verbal learning, relational encoding and retrieval, visual integration and reinforcement learning.

RESULTS

Individuals with schizophrenia who were not taking antipsychotic medications, as well as those taking antipsychotic medications, showed pervasive deficits across cognitive domains including reinforcement learning, processing speed, cognitive control, working memory, verbal learning and relational encoding and retrieval. Further, we found that chlorpromazine equivalency rates were significantly related to processing speed and working memory, while there were no significant relationships between anticholinergic load and performance on other tasks.

CONCLUSIONS

These findings add to a body of literature suggesting that cognitive deficits are an enduring aspect of schizophrenia, present in those off antipsychotic medications as well as those taking antipsychotic medications.

Psychological Dimensions Relevant to Motivation and Pleasure in Schizophrenia

Motivation and pleasure deficits are common in schizophrenia, strongly linked with poorer functioning, and may reflect underlying alterations in brain functions governing reward processing and goal pursuit. While there is extensive research examining cognitive and reward mechanisms related to these deficits in schizophrenia, less attention has been paid to psychological characteristics that contribute to resilience against, or risk for, motivation and pleasure impairment. For example, psychological tendencies involving positive future expectancies (e.g., optimism) and effective affect management (e.g., reappraisal, mindfulness) are associated with aspects of reward anticipation and evaluation that optimally guide goal-directed behavior. Conversely, maladaptive thinking patterns (e.g., defeatist performance beliefs, asocial beliefs) and tendencies that amplify negative cognitions (e.g., rumination), may divert cognitive resources away from goal pursuit or reduce willingness to exert effort. Additionally, aspects of sociality, including the propensity to experience social connection as positive reinforcement may be particularly relevant for pursuing social goals. In the current review, we discuss the roles of several psychological characteristics with respect to motivation and pleasure in schizophrenia. We argue that individual variation in these psychological dimensions is relevant to the study of motivation and reward processing in schizophrenia, including interactions between these psychological dimensions and more well-characterized cognitive and reward processing contributors to motivation. We close by emphasizing the value of considering a broad set of modulating factors when studying motivation and pleasure functions in schizophrenia.

Reduced learning bias towards the reward context in medication-naive first-episode schizophrenia patients

BACKGROUND

Reinforcement learning has been proposed to contribute to the development of amotivation in individuals with schizophrenia (SZ). Accumulating evidence suggests dysfunctional learning in individuals with SZ in Go/NoGo learning and expected value representation. However, previous findings might have been confounded by the effects of antipsychotic exposure. Moreover, reinforcement learning also rely on the learning context. Few studies have examined the learning performance in reward and loss-avoidance context separately in medication-naïve individuals with first-episode SZ. This study aimed to explore the behaviour profile of reinforcement learning performance in medication-naïve individuals with first-episode SZ, including the contextual performance, the Go/NoGo learning and the expected value representation performance.

METHODS

Twenty-nine medication-naïve individuals with first-episode SZ and 40 healthy controls (HCs) who have no significant difference in age and gender, completed the Gain and Loss Avoidance Task, a reinforcement learning task involving stimulus pairs presented in both the reward and loss-avoidance context. We assessed the group difference in accuracy in the reward and loss-avoidance context, the Go/NoGo learning and the expected value representation. The correlations between learning performance and the negative symptom severity were examined.

RESULTS

Individuals with SZ showed significantly lower accuracy when learning under the reward than the loss-avoidance context as compared to HCs. The accuracies under the reward context (90%win- 10%win) in the Acquisition phase was significantly and negatively correlated with the Scale for the Assessment of Negative Symptoms (SANS) avolition scores in individuals with SZ. On the other hand, individuals with SZ showed spared ability of Go/NoGo learning and expected value representation.

CONCLUSIONS

Despite our small sample size and relatively modest findings, our results suggest possible reduced learning bias towards reward context among medication-naïve individuals with first-episode SZ. The reward learning performance was correlated with amotivation symptoms. This finding may facilitate our understanding of the underlying mechanism of negative symptoms. Reinforcement learning performance under the reward context may be important to better predict and prevent the development of schizophrenia patients' negative symptom, especially amotivation.

The prediction-error hypothesis of schizophrenia: new data point to circuit-specific changes in dopamine activity

Schizophrenia is a severe psychiatric disorder affecting 21 million people worldwide. People with schizophrenia suffer from symptoms including psychosis and delusions, apathy, anhedonia, and cognitive deficits. Strikingly, schizophrenia is characterised by a learning paradox involving difficulties learning from rewarding events, whilst simultaneously 'overlearning' about irrelevant or neutral information. While dysfunction in dopaminergic signalling has long been linked to the pathophysiology of schizophrenia, a cohesive framework that accounts for this learning paradox remains elusive. Recently, there has been an explosion of new research investigating how dopamine contributes to reinforcement learning, which illustrates that midbrain dopamine contributes in complex ways to reinforcement learning, not previously envisioned. This new data brings new possibilities for how dopamine signalling contributes to the symptomatology of schizophrenia. Building on recent work, we present a new neural framework for how we might envision specific dopamine circuits contributing to this learning paradox in schizophrenia in the context of models of reinforcement learning. Further, we discuss avenues of preclinical research with the use of cutting-edge neuroscience techniques where aspects of this model may be tested. Ultimately, it is hoped that this review will spur to action more research utilising specific reinforcement learning paradigms in preclinical models of schizophrenia, to reconcile seemingly disparate symptomatology and develop more efficient therapeutics.

A review of risky decision-making in psychosis-spectrum disorders

The investigation of risky decision-making has a prominent place in clinical science, with sundry behavioral tasks aimed at empirically quantifying the psychological construct of risk-taking. However, use of differing behavioral tasks has resulted in lack of agreement on risky decision-making within psychosis-spectrum disorders, as findings fail to converge upon the typical, binary conceptualization of increased risk-seeking or risk-aversion. The current review synthesizes the behavioral, risky decision-making literature to elucidate how specific task parameters may contribute to differences in task performance, and their associations with psychosis symptomatology and cognitive functioning. A paring of the literature suggests that: 1) Explicit risk-taking may be characterized by risk imperception, evidenced by less discrimination between choices of varying degrees of risk, potentially secondary to cognitive deficits. 2) Ambiguous risk-taking findings are inconclusive with few published studies. 3) Uncertain risk-taking findings, consistently interpreted as more risk-averse, have not parsed risk attitudes from confounding processes that may impact decision-making (e.g. risk imperception, reward processing, motivation). Thus, overgeneralized interpretations of task-specific risk-seeking/aversion should be curtailed, as they may fail to appropriately characterize decision-making phenomena. Future research in psychosis-spectrum disorders would benefit from empirically isolating contributions of specific processes during risky decision-making, including the newly hypothesized risk imperception.

Responses to positive affect and unique resting-state connectivity in individuals at clinical high-risk for psychosis

Individuals at clinical high-risk for psychosis (CHR) report dampened positive affect, while this deficit appears to be an important clinical marker, our current understanding of underlying causes is limited. Dysfunctional regulatory strategies (i.e., abnormal use of dampening, self-focused, or emotion-focused strategies) may account for dampening affect but has not yet been examined. Participants (57 CHR and 56 healthy controls) completed the Response to Positive Affect Scale, clinical interviews, and resting-state scan examining nucleus accumbens (NAcc) connectivity. Individuals at CHR for psychosis showed greater dampening (but no differences in self/emotion-focus) in self-reported response to positive affect compared to healthy controls. In individuals at CHR, higher levels of dampening and lower levels of self-focus were associated with higher positive and lower negative symptoms. Dampening responses were related to decreased dorsal and rostral anterior cingulate cortex-NAcc resting-state connectivity in the CHR group but increased dorsal and rostral anterior cingulate cortex-NAcc resting-state connectivity in the healthy control group. Self-focused responses were related to increased dorsolateral prefrontal cortex-NAcc resting-state connectivity in the CHR group but decreased resting-state connectivity in the healthy control group. Self-reported dampening of positive affect was elevated in individuals at CHR for psychosis. Dampening and self-focused responses were associated with distinct resting-state connectivity compared to peers, suggesting unique mechanisms underlying these emotion regulation strategies. Responses to positive affect may be a useful target for cognitive treatment, but individuals at CHR show distinct neurocorrelates and may require a tailored approach.

Pleasure, Reward Value, Prediction Error and Anhedonia

In order to develop effective treatments for anhedonia we need to understand its underlying neurobiological mechanisms. Anhedonia is conceptually strongly linked to reward processing, which involves a variety of cognitive and neural operations. This chapter reviews the evidence for impairments in experiencing hedonic response (pleasure), reward valuation and reward learning based on outcomes (commonly conceptualised in terms of "reward prediction error"). Synthesising behavioural and neuroimaging findings, we examine case-control studies of patients with depression and schizophrenia, including those focusing specifically on anhedonia. Overall, there is reliable evidence that depression and schizophrenia are associated with disrupted reward processing. In contrast to the historical definition of anhedonia, there is surprisingly limited evidence for impairment in the ability to experience pleasure in depression and schizophrenia. There is some evidence that learning about reward and reward prediction error signals are impaired in depression and schizophrenia, but the literature is inconsistent. The strongest evidence is for impairments in the representation of reward value and how this is used to guide action. Future studies would benefit from focusing on impairments in reward processing specifically in anhedonic samples, including transdiagnostically, and from using designs separating different components of reward processing, formulating them in computational terms, and moving beyond cross-sectional designs to provide an assessment of causality.

Anhedonia in Schizophrenia

Anhedonia has long been considered a cardinal symptom of schizophrenia. This symptom is strongly associated with poor functional outcome, and limited treatment options are available. While originally conceptualized as an inability to experience pleasure, recent work has consistently shown that individuals with schizophrenia have an intact capacity to experience pleasure in-the-moment. Adjacent work in basic affective neuroscience has broadened the conceptualization of anhedonia to include not only the capacity to experience pleasure but highlights important temporal affective dynamics and decision-making processes that go awry in schizophrenia. Here we detail these mechanisms for emotional and motivational impairment in people with schizophrenia including: (1) initial response to reward; (2) reward anticipation; (3) reward learning; (4) effort-cost decision-making; (5) working memory and cognitive control. We will review studies that utilized various types of rewards (e.g., monetary, social), in order to draw conclusions regarding whether findings vary by reward type. We will then discuss how modern assessment methods may best incorporate each of the mechanisms, to provide a more fine-grained understanding of anhedonia in individuals with schizophrenia. We will close by providing a discussion of relevant future directions.

Probabilistic Reinforcement Learning and Anhedonia

Despite the prominence of anhedonic symptoms associated with diverse neuropsychiatric conditions, there are currently no approved therapeutics designed to attenuate the loss of responsivity to previously rewarding stimuli. However, the search for improved treatment options for anhedonia has been reinvigorated by a recent reconceptualization of the very construct of anhedonia, including within the Research Domain Criteria (RDoC) initiative. This chapter will focus on the RDoC Positive Valence Systems construct of reward learning generally and sub-construct of probabilistic reinforcement learning specifically. The general framework emphasizes objective measurement of a subject's responsivity to reward via reinforcement learning under asymmetrical probabilistic contingencies as a means to quantify reward learning. Indeed, blunted reward responsiveness and reward learning are central features of anhedonia and have been repeatedly described in major depression. Moreover, these probabilistic reinforcement techniques can also reveal neurobiological mechanisms to aid development of innovative treatment approaches. In this chapter, we describe how investigating reward learning can improve our understanding of anhedonia via the four RDoC-recommended tasks that have been used to probe sensitivity to probabilistic reinforcement contingencies and how such task performance is disrupted in various neuropsychiatric conditions. We also illustrate how reverse translational approaches of probabilistic reinforcement assays in laboratory animals can inform understanding of pharmacological and physiological mechanisms. Next, we briefly summarize the neurobiology of probabilistic reinforcement learning, with a focus on the prefrontal cortex, anterior cingulate cortex, striatum, and amygdala. Finally, we discuss treatment implications and future directions in this burgeoning area.

Smoking as a Common Modulator of Sensory Gating and Reward Learning in Individuals with Psychotic Disorders

Motivational and perceptual disturbances co-occur in psychosis and have been linked to aberrations in reward learning and sensory gating, respectively. Although traditionally studied independently, when viewed through a predictive coding framework, these processes can both be linked to dysfunction in striatal dopaminergic prediction error signaling. This study examined whether reward learning and sensory gating are correlated in individuals with psychotic disorders, and whether nicotine—a psychostimulant that amplifies phasic striatal dopamine firing—is a common modulator of these two processes. We recruited 183 patients with psychotic disorders (79 schizophrenia, 104 psychotic bipolar disorder) and 129 controls and assessed reward learning (behavioral probabilistic reward task), sensory gating (P50 event-related potential), and smoking history. Reward learning and sensory gating were correlated across the sample. Smoking influenced reward learning and sensory gating in both patient groups; however, the effects were in opposite directions. Specifically, smoking was associated with improved performance in individuals with schizophrenia but impaired performance in individuals with psychotic bipolar disorder. These findings suggest that reward learning and sensory gating are linked and modulated by smoking. However, disorder-specific associations with smoking suggest that nicotine may expose pathophysiological differences in the architecture and function of prediction error circuitry in these overlapping yet distinct psychotic disorders.

Psychiatric symptoms influence reward-seeking and loss-avoidance decision-making through common and distinct computational processes

AIM

Psychiatric symptoms are often accompanied by impairments in decision-making to attain rewards and avoid losses. However, due to the complex nature of mental disorders (e.g., high comorbidity), symptoms that are specifically associated with deficits in decision-making remain unidentified. Furthermore, the influence of psychiatric symptoms on computations underpinning reward-seeking and loss-avoidance decision-making remains elusive. Here, we aim to address these issues by leveraging a large-scale online experiment and computational modeling.

METHODS

In the online experiment, we recruited 1900 non-diagnostic participants from the general population. They performed either a reward-seeking or loss-avoidance decision-making task, and subsequently completed questionnaires about psychiatric symptoms.

RESULTS

We found that one trans-diagnostic dimension of psychiatric symptoms related to compulsive behavior and intrusive thought (CIT) was negatively correlated with overall decision-making performance in both the reward-seeking and loss-avoidance tasks. A deeper analysis further revealed that, in both tasks, the CIT psychiatric dimension was associated with lower preference for the options that recently led to better outcomes (i.e. reward or no-loss). On the other hand, in the reward-seeking task only, the CIT dimension was associated with lower preference for recently unchosen options.

CONCLUSION

These findings suggest that psychiatric symptoms influence the two types of decision-making, reward-seeking and loss-avoidance, through both common and distinct computational processes.

Extraversion but not depression predicts reward sensitivity: Revisiting the measurement of anhedonic phenotypes

Recently, increasing efforts have been made to define and measure dimensional phenotypes associated with psychiatric disorders. One example is a probabilistic reward task developed by Pizzagalli, Jahn, and O'Shea (2005) to assess anhedonia, by measuring response to a differential reinforcement schedule. This task has been used in many studies, which have connected blunted reward response in the task to depressive symptoms, across clinical groups and in the general population. The current study attempted to replicate these findings in a large community sample and also investigated possible associations with Extraversion, a personality trait linked to reward sensitivity. Participants (N = 299) completed the probabilistic reward task, as well as the Beck Depression Inventory, Personality Inventory for the DSM-5, Big Five Inventory, and Big Five Aspect Scales. Our direct replication attempts used bivariate correlations and analysis of variance models. Follow-up and extension analyses used structural equation models to assess relations among reward sensitivity, depression, Extraversion, and Neuroticism. No significant associations were found between reward sensitivity and depression, thus failing to replicate previous findings. Reward sensitivity (both modeled as response bias aggregated across blocks and as response bias controlling for baseline) showed positive associations with Extraversion, but not Neuroticism. Findings suggest reward sensitivity as measured by this task may be related primarily to Extraversion and its pathological manifestations, rather than to depression per se, consistent with existing models that conceptualize depressive symptoms as combining features of Neuroticism and low Extraversion. Findings are discussed in broader contexts of dimensional psychopathology frameworks, replicable science, and behavioral task reliability. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Relations Among Anhedonia, Reinforcement Learning, and Global Functioning in Help-seeking Youth

Dysfunction in the neural circuits underlying salience signaling is implicated in symptoms of psychosis and may predict conversion to a psychotic disorder in youth at clinical high risk (CHR) for psychosis. Additionally, negative symptom severity, including consummatory and anticipatory aspects of anhedonia, may predict functional outcome in individuals with schizophrenia-spectrum disorders. However, it is unclear whether anhedonia is related to the ability to attribute incentive salience to stimuli (through reinforcement learning [RL]) and whether measures of anhedonia and RL predict functional outcome in a younger, help-seeking population. We administered the Salience Attribution Test (SAT) to 33 participants who met criteria for either CHR or a recent-onset psychotic disorder and 29 help-seeking youth with nonpsychotic disorders. In the SAT, participants must identify relevant and irrelevant stimulus dimensions and be sensitive to different reinforcement probabilities for the 2 levels of the relevant dimension ("adaptive salience"). Adaptive salience attribution was positively related to both consummatory pleasure and functioning in the full sample. Analyses also revealed an indirect effect of adaptive salience on the relation between consummatory pleasure and both role (αβ = .22, 95% CI = 0.02, 0.48) and social functioning (αβ = .14, 95% CI = 0.02, 0.30). These findings suggest a distinct pathway to poor global functioning in help-seeking youth, via impaired reward sensitivity and RL.

Reliability and Replicability of Implicit and Explicit Reinforcement Learning Paradigms in People With Psychotic Disorders

BACKGROUND

Motivational deficits in people with psychosis may be a result of impairments in reinforcement learning (RL). Therefore, behavioral paradigms that can accurately measure these impairments and their change over time are essential.

METHODS

We examined the reliability and replicability of 2 RL paradigms (1 implicit and 1 explicit, each with positive and negative reinforcement components) given at 2 time points to healthy controls (n = 75), and people with bipolar disorder (n = 62), schizoaffective disorder (n = 60), and schizophrenia (n = 68).

RESULTS

Internal consistency was acceptable (mean α = 0.78 ± 0.15), but test-retest reliability was fair to low (mean intraclass correlation = 0.33 ± 0.25) for both implicit and explicit RL. There were no clear effects of practice for these tasks. Largely, performance on these tasks shows intact implicit and impaired explicit RL in psychosis. Symptom presentation did not relate to performance in any robust way.

CONCLUSIONS

Our findings replicate previous literature showing spared implicit RL and impaired explicit reinforcement in psychosis. This suggests typical basal ganglia dopamine release, but atypical recruitment of the orbitofrontal and dorsolateral prefrontal cortices. However, we found that these tasks have only fair to low test-retest reliability and thus may not be useful for assessing change over time in clinical trials.

Retention of Value Representations Across Time in People With Schizophrenia and Healthy Control Subjects

BACKGROUND

The current study aimed to further etiological understanding of the psychological mechanisms underlying negative symptoms in people with schizophrenia. Specifically, we tested whether negative symptom severity is associated with reduced retention of reward-related information over time and thus a degraded ability to utilize such information to guide future action selection.

METHODS

Forty-four patients with a diagnosis of schizophrenia or schizoaffective disorder and 28 healthy control volunteers performed a probabilistic reinforcement-learning task involving stimulus pairs in which choices resulted in reward or in loss avoidance. Following training, participants indicated their valuation of learned stimuli in a test/transfer phase. The test/transfer phase was administered immediately following training and 1 week later. Percent retention was defined as accuracy at week-long delay divided by accuracy at immediate delay.

RESULTS

Healthy control subjects and people with schizophrenia showed similarly robust retention of reinforcement learning over a 1-week delay interval. However, in the schizophrenia group, negative symptom severity was associated with reduced retention of information regarding the value of actions across a week-long interval. This pattern was particularly notable for stimuli associated with reward compared with loss avoidance.

CONCLUSIONS

Our results show that although individuals with schizophrenia may initially learn about rewarding aspects of their environment, such learning decays at a more rapid rate in patients with severe negative symptoms. Thus, previously learned reward-related information may be more difficult to access to guide future decision making and to motivate action selection.

Pareidolia in Schizophrenia and Bipolar Disorder

While there are many studies on pareidolia in healthy individuals and patients with schizophrenia, to our knowledge, there are no prior studies on pareidolia in patients with bipolar disorder. Accordingly, in this study, we, for the first time, measured pareidolia in patients with bipolar disorder (N = 50), and compared that to patients with schizophrenia (N = 50) and healthy controls (N = 50). We have used (a) the scene test, which consists of 10 blurred images of natural scenes that was previously found to produce illusory face responses and (b) the noise test which had 32 black and white images consisting of visual noise and 8 images depicting human faces; participants indicated whether a face was present on these images and to point to the location where they saw the face. Illusory responses were defined as answers when observers falsely identified objects that were not on the images in the scene task (maximum illusory score: 10), and the number of noise images in which they reported the presence of a face (maximum illusory score: 32). Further, we also calculated the total pareidolia score for each task (the sum number of images with illusory responses in the scene and noise tests). The responses were scored by two independent raters with an excellent congruence (kappa > 0.9). Our results show that schizophrenia patients scored higher on pareidolia measures than both healthy controls and patients with bipolar disorder. Our findings are agreement with prior findings on more impaired cognitive processes in schizophrenia than in bipolar patients.

Dissociable mechanisms underpinning effort-cost decision-making across the psychosis spectrum

Recent theoretical models propose that abnormal effort-cost decision-making (ECDM) likely has divergent underpinnings across mood and psychotic disorders. However, whether this same model applies to individuals across the psychosis spectrum, including individuals with affective psychosis, remains unclear. This study aimed to empirically test whether two component processes - working memory and reward learning - contribute to ECDM impairment across the psychosis spectrum. ECDM was assessed using the Effort Expenditure for Rewards Task in individuals with psychotic disorders (n = 190) and healthy controls (n = 52). Working memory was assessed using a Digit Sequencing Task and reward learning was assessed using a Probabilistic Reward Task. Relative to the control group, the psychosis group showed reduced willingness to expend effort for higher probability, higher value rewards. This effect was most pronounced in individuals with schizophrenia and schizoaffective disorder relative to individuals with psychotic bipolar disorder. Across the whole sample, better working memory but not reward learning predicted greater willingness to expend effort for higher probability rewards. However, the link between working memory and ECDM differed as a function of patient symptom profile. Specifically, working memory was only predictive of ECDM for individuals with less severe negative symptoms and minimal depressive symptoms. For individuals with more severe negative symptoms, poorer ECDM was instead predicted by deficits in reward learning. Although these findings reiterate the important link between working memory and ECDM in individuals with psychotic disorders, they also show that this link varies in accordance with the presence of prominent negative and depressive symptoms.

Overlapping Neurobiological Substrates for Early-Life Stress and Resilience to Psychosis

Early-life stress, such as childhood maltreatment, is a well-known etiological factor in psychopathology, including psychosis. Exposure to early-life stress disrupts the neurodevelopment of widespread brain systems, including key components of the hypothalamic-pituitary-adrenal axis stress response, such as the amygdala, hippocampus, and medial prefrontal cortex, as well as key components of the brain's reward system, such as the nucleus accumbens and orbitofrontal cortex. These disruptions have a considerable impact on the function of emotion and reward circuitry, which play a central role in the emergence and severity of psychosis. While this overlap may provide insight into the pathophysiology of psychosis, it also provides unique opportunities to elucidate neurobiological substrates that may promote resilience to psychosis. In this review, we discuss the hypothalamic-pituitary-adrenal axis stress response, discuss the disruption in the neurodevelopment of emotion and reward processing associated with early stress exposures, and examine how this circuitry may contribute to resilience to psychotic disorders.

Distinguishing patterns of impairment on inhibitory control and general cognitive ability among bipolar with and without psychosis, schizophrenia, and schizoaffective disorder

BACKGROUND

Deficits in inhibitory control on a Stop Signal Task (SST) were previously observed to be of similar magnitude across schizophrenia, schizoaffective, and bipolar disorder with psychosis, despite variation in general cognitive ability. Understanding different patterns of performance on the SST may elucidate different pathways to the impaired inhibitory control each group displayed. Comparing nonpsychotic bipolar disorder to the psychosis groups on SST may also expand our understanding of the shared neurobiology of this illness spectrum.

METHODS

We tested schizophrenia (n = 220), schizoaffective (n = 216), bipolar disorder with (n = 192) and without psychosis (n = 67), and 280 healthy comparison participants with a SST and the Brief Assessment of Cognition in Schizophrenia (BACS), a measure of general cognitive ability.

RESULTS

All patient groups had a similar degree of impaired inhibitory control over prepotent responses. However, bipolar groups differed from schizophrenia and schizoaffective groups in showing speeded responses and inhibition errors that were not accounted for by general cognitive ability. Schizophrenia and schizoaffective groups had a broader set of deficits on inhibition and greater general cognitive deficit, which fully accounted for the inhibition deficits. No differences were found between the clinically well-matched bipolar with and without psychosis groups, including for inhibitory control or general cognitive ability.

CONCLUSIONS

We conclude that 1) while impaired inhibitory control on a SST is of similar magnitude across the schizo-bipolar spectrum, including nonpsychotic bipolar, different mechanisms may underlie the impairments, and 2) history of psychosis in bipolar disorder does not differentially impact inhibitory behavioral control or general cognitive abilities.

Beyond dichotomies in reinforcement learning

Reinforcement learning (RL) is a framework of particular importance to psychology, neuroscience and machine learning. Interactions between these fields, as promoted through the common hub of RL, has facilitated paradigm shifts that relate multiple levels of analysis in a singular framework (for example, relating dopamine function to a computationally defined RL signal). Recently, more sophisticated RL algorithms have been proposed to better account for human learning, and in particular its oft-documented reliance on two separable systems: a model-based (MB) system and a model-free (MF) system. However, along with many benefits, this dichotomous lens can distort questions, and may contribute to an unnecessarily narrow perspective on learning and decision-making. Here, we outline some of the consequences that come from overconfidently mapping algorithms, such as MB versus MF RL, with putative cognitive processes. We argue that the field is well positioned to move beyond simplistic dichotomies, and we propose a means of refocusing research questions towards the rich and complex components that comprise learning and decision-making.

Abnormal approach-related motivation but spared reinforcement learning in MDD: Evidence from fronto-midline Theta oscillations and frontal Alpha asymmetry

Major depression is characterized by abnormal reward processing and reinforcement learning (RL). This impairment might stem from deficient motivation processes, in addition to reduced reward sensitivity. In this study, we recorded 64-channel EEG in a large cohort of major depressive disorder (MDD) patients and matched healthy controls (HC) while they performed a standard RL task. Participants were asked to discover, by trial and error, several hidden stimulus-response associations having different reward probabilities, as enforced using evaluative feedback. We extracted induced fronto-midline Theta (FMT) power time-locked to the response and feedback as neurophysiological index of RL. Furthermore, we assessed approach-related motivation by measuring frontal alpha asymmetry concurrently. At the behavioral level, MDD patients and HCs showed comparable RL. At the EEG level, FMT power systematically varied as a function of reward probability, with opposing effects found at the response and feedback levels. Although this global pattern was spared in MDD, at the feedback level these patients showed however a steep FMT power decrease across trials when reward probability was low. Moreover, they showed impaired approach-related motivation during task execution, as reflected by frontal Alpha asymmetry. These results suggest a dissociation between (globally spared) RL and (impaired) approach motivation in MDD.

Enhancing Psychosis Risk Prediction Through Computational Cognitive Neuroscience

Research suggests that early identification and intervention with individuals at clinical high risk (CHR) for psychosis may be able to improve the course of illness. The first generation of studies suggested that the identification of CHR through the use of specialized interviews evaluating attenuated psychosis symptoms is a promising strategy for exploring mechanisms associated with illness progression, etiology, and identifying new treatment targets. The next generation of research on psychosis risk must address two major limitations: (1) interview methods have limited specificity, as recent estimates indicate that only 15%-30% of individuals identified as CHR convert to psychosis and (2) the expertise needed to make CHR diagnosis is only accessible in a handful of academic centers. Here, we introduce a new approach to CHR assessment that has the potential to increase accessibility and positive predictive value. Recent advances in clinical and computational cognitive neuroscience have generated new behavioral measures that assay the cognitive mechanisms and neural systems that underlie the positive, negative, and disorganization symptoms that are characteristic of psychotic disorders. We hypothesize that measures tied to symptom generation will lead to enhanced sensitivity and specificity relative to interview methods and the cognitive intermediate phenotype measures that have been studied to date that are typically indicators of trait vulnerability and, therefore, have a high false positive rate for conversion to psychosis. These new behavioral measures have the potential to be implemented on the internet and at minimal expense, thereby increasing accessibility of assessments.

Dissecting the impact of depression on decision-making

BACKGROUND

Cognitive deficits in depressed adults may reflect impaired decision-making. To investigate this possibility, we analyzed data from unmedicated adults with Major Depressive Disorder (MDD) and healthy controls as they performed a probabilistic reward task. The Hierarchical Drift Diffusion Model (HDDM) was used to quantify decision-making mechanisms recruited by the task, to determine if any such mechanism was disrupted by depression.

METHODS

Data came from two samples (Study 1: 258 MDD, 36 controls; Study 2: 23 MDD, 25 controls). On each trial, participants indicated which of two similar stimuli was presented; correct identifications were rewarded. Quantile-probability plots and the HDDM quantified the impact of MDD on response times (RT), speed of evidence accumulation (drift rate), and the width of decision thresholds, among other parameters.

RESULTS

RTs were more positively skewed in depressed v. healthy adults, and the HDDM revealed that drift rates were reduced-and decision thresholds were wider-in the MDD groups. This pattern suggests that depressed adults accumulated the evidence needed to make decisions more slowly than controls did.

CONCLUSIONS

Depressed adults responded slower than controls in both studies, and poorer performance led the MDD group to receive fewer rewards than controls in Study 1. These results did not reflect a sensorimotor deficit but were instead due to sluggish evidence accumulation. Thus, slowed decision-making-not slowed perception or response execution-caused the performance deficit in MDD. If these results generalize to other tasks, they may help explain the broad cognitive deficits seen in depression.

Dynamic changes of brain networks during feedback-related processing of reinforcement learning in schizophrenia

Previous studies have reported that schizophrenia (SZ) patients showed selective reinforcement learning deficits and abnormal feedback-related event-related potential (ERP) components. However, how the brain networks and their topological properties evolve over time during transient feedback-related cognition processing in SZ patients has not been investigated so far. In this paper, using publicly available feedback-related ERP data which were recorded from SZ patients and healthy controls (HC) when they performed a reinforcement learning task, we carried out an event-related network analysis where topology of brain functional networks was characterized with some graph measures including clustering coefficient (C), global efficiency (E_global) and local efficiency (E_local) on a millisecond timescale. Our results showed that the brain functional networks displayed rapid rearrangements of topological properties during transient feedback-related cognition process for both two groups. More importantly, we found that SZ patients exhibited significantly reduced theta-band (time window of 170-350 ms after stimuli onset) brain functional connectivity strength, E_global, E_local and C in response to negative feedback stimuli compared to HC group. The network based statistic (NBS) analysis detected one significantly decreased theta-band subnetwork in SZ patients mainly involving in frontal-occipital and temporal-occipital connections compared to HC group. In addition, clozapine treatment seemed to greatly reduce theta-band power and topological measures of brain networks in SZ patients. Finally, the theta-band power, graph measures and functional connectivity were extracted to train a support vector machine classifier for classification of HC from SZ, or Cloz + SZ or Cloz- SZ, and a relatively good classification accuracy of 84.48%, 89.47% and 78.26% was obtained, respectively. The above results suggested a less optimal organization of theta-band brain network in SZ patients, and studying the topological parameters of brain networks evolve over time during transient feedback-related processing could be useful for understanding the pathophysiologic mechanisms underlying reinforcement learning deficits in SZ patients.

Grant Report on Social Reward Learning in Schizophrenia †

We report on the ongoing R21 project “Social Reward Learning in Schizophrenia”. Impairments in social cognition are a hallmark of schizophrenia. However, little work has been done on social reward learning deficits in schizophrenia. The overall goal of the project is to assess social reward learning in schizophrenia. A probabilistic reward learning (PRL) task is being used in the MRI scanner to evaluate reward learning to negative and positive social feedback. Monetary reward learning is used as a comparison to assess specificity. Behavioral outcomes and brain areas, included those involved in reward, are assessed in patients with schizophrenia or schizoaffective disorder and controls. It is also critical to determine whether decreased expected value (EV) of social stimuli and/or reward prediction error (RPE) learning underlie social reward learning deficits to inform potential treatment pathways. Our central hypothesis is that the pattern of social learning deficits is an extension of a more general reward learning impairment in schizophrenia and that social reward learning deficits critically contribute to deficits in social motivation and pleasure. We hypothesize that people with schizophrenia will show impaired behavioral social reward learning compared to controls, as well as decreased ventromedial prefrontal cortex (vmPFC) EV signaling at time of choice and decreased striatal RPE signaling at time of outcome, with potentially greater impairment to positive than negative feedback. The grant is in its second year. It is hoped that this innovative approach may lead to novel and more targeted treatment approaches for social cognitive impairments, using cognitive remediation and/or brain stimulation.

Victory is its own reward: oxytocin increases costly competitive behavior in schizophrenia

BACKGROUND

Aberrant sensitivity to social reward may be an important contributor to abnormal social behavior that is a core feature of schizophrenia. The neuropeptide oxytocin impacts the salience of social information across species, but its effect on social reward in schizophrenia is unknown.

METHODS

We used a competitive economic game and computational modeling to examine behavioral dynamics and oxytocin effects on sensitivity to social reward among 39 men with schizophrenia and 54 matched healthy controls. In a randomized, double-blind study, participants received one dose of oxytocin (40 IU) or placebo and completed a 35-trial Auction Game that quantifies preferences for monetary v. social reward. We analyzed bidding behavior using multilevel linear mixed models and reinforcement learning models.

RESULTS

Bidding was motivated by preferences for both monetary and social reward in both groups, but bidding dynamics differed: patients initially overbid less compared to controls, and across trials, controls decreased their bids while patients did not. Oxytocin administration was associated with sustained overbidding across trials, particularly in patients. This drug effect was driven by a stronger preference for winning the auction, regardless of monetary consequences. Learning rate and response variability did not differ between groups or drug condition, suggesting that differences in bidding derive primarily from differences in the subjective value of social rewards.

CONCLUSIONS

Our findings suggest that schizophrenia is associated with diminished motivation for social reward that may be increased by oxytocin administration.

Investigating real-time social interaction in pairs of adolescents with the Perceptual Crossing Experiment

The study of real-time social interaction provides ecologically valid insight into social behavior. The objective of the current research is to experimentally assess real-time social contingency detection in an adolescent population, using a shortened version of the Perceptual Crossing Experiment (PCE). Pairs of 148 adolescents aged between 12 and 19 were instructed to find each other in a virtual environment interspersed with other objects by interacting with each other using tactile feedback only. Across six rounds, participants demonstrated increasing accuracy in social contingency detection, which was associated with increasing subjective experience of the mutual interaction. Subjective experience was highest in rounds when both participants were simultaneously accurate in detecting each other's presence. The six-round version yielded comparable social contingency detection outcome measures to a ten-round version of the task. The shortened six-round version of the PCE has therefore enabled us to extend the previous findings on social contingency detection in adults to an adolescent population, enabling implementation in prospective research designs to assess the development of social contingency detection over time.

A tradeoff relationship between internal monitoring and external feedback during the dynamic process of reinforcement learning

Effective behavior monitoring, including internal monitoring/error detection and external monitoring/feedback, is very pivotal for reinforcement learning. However, less attention has been paid to internal monitoring and the dynamic learning performance in reinforcement learning, and there is still a heated debate on which kind of external feedback is relied on in the reinforcement learning. In order to address these questions, an adaption probabilistic selection task was used to examine the effect of the internal monitoring, external feedback and the relationship between them for approach learners and avoidance learners during dynamic learning process of reinforcement learning and behavior adaption. Error-related negativity (ERN), feedback-related negativity (FRN) and feedback-related P300 are three ERPs components, which can be used as the indexes of internal monitoring, external feedback and behavior adaption. For our results, the ERN effect of avoidance learners become large in block 3, which is earlier than approach learners (block 4). This phenomenon suggests that avoidance learners learned faster than approach learners. In addition, the FRN amplitude of avoidance learners in block 4 was significantly smaller than the other three blocks. The aforementioned results demonstrated a tradeoff relationship between the ERN and FRN effects.

Affective cognition in bipolar disorder: A systematic review by the ISBD targeting cognition task force

BACKGROUND

Impairments in affective cognition are part of the neurocognitive profile and possible treatment targets in bipolar disorder (BD), but the findings are heterogeneous. The International Society of Bipolar Disorder (ISBD) Targeting Cognition Task Force conducted a systematic review to (i) identify the most consistent findings in affective cognition in BD, and (ii) provide suggestions for affective cognitive domains for future study and meta-analyses.

METHODS

The review included original studies reporting behavioral measures of affective cognition in BD patients vs controls following the procedures of the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) statement. Searches were conducted on PubMed/MEDLINE, EMBASE, and PsychInfo from inception until November 2018.

RESULTS

A total of 106 articles were included (of which nine included data for several affective domains); 41 studies assessed emotional face processing; 23 studies investigated reactivity to emotional words and images; 3 investigated explicit emotion regulation; 17 assessed implicit emotion regulation; 31 assessed reward processing and affective decision making. In general, findings were inconsistent. The most consistent findings were trait-related difficulties in facial emotion recognition and implicit emotion regulation, and impairments in reward processing and affective decision making during mood episodes. Studies using eye-tracking and facial emotion analysis revealed subtle trait-related abnormalities in emotional reactivity.

CONCLUSION

The ISBD Task Force recommends facial expression recognition, implicit emotion regulation, and reward processing as domains for future research and meta-analyses. An important step to aid comparability between studies in the field would be to reach consensus on an affective cognition test battery for BD.

Reward and punishment learning in schizophrenia and bipolar disorder

Prior studies on reward learning deficits in psychiatric disorders have used probabilistic learning tasks, making it unclear whether impairment is due to the probabilistic nature of the task rather than reward processing. In this study, we tested probabilistic vs. deterministic reward and punishment learning in healthy controls and three patient groups: schizophrenia (SZ), psychotic bipolar disorder (BD), and nonpsychotic BD. Experimental results show that reward learning was impaired in patients with SZ and patients with psychotic BD in the probabilistic learning task compared to patients with nonpsychotic BD and healthy controls. In contrast, punishment learning in the probabilistic task was impaired in patients with nonpsychotic BD compared to the other patient groups and healthy controls. There were no significant differences among all groups in the deterministic learning task scores. We also found that Hamilton Depression Scale scores negatively correlated with probabilistic learning performance. Our data may suggest that reward learning impairment may be due to the nature of the task as well as subtype of BD.

Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals

Individuals from across the psychosis spectrum display impairments in reinforcement learning. In some individuals, these deficits may result from aberrations in reward prediction error (RPE) signaling, conveyed by dopaminergic projections to the ventral striatum (VS). However, there is mounting evidence that VS RPE signals are relatively intact in medicated people with schizophrenia (PSZ). We hypothesized that, in PSZ, reinforcement learning deficits often are not related to RPE signaling per se but rather their impact on learning and behavior (i.e., learning rate modulation), due to dysfunction in anterior cingulate and dorsomedial prefrontal cortex (dmPFC). Twenty-six PSZ and 23 healthy volunteers completed a probabilistic reinforcement learning paradigm with occasional, sudden, shifts in contingencies. Using computational modeling, we found evidence of an impairment in trial-wise learning rate modulation (α) in PSZ before and after a reinforcement contingency shift, expressed most in PSZ with more severe motivational deficits. In a subsample of 22 PSZ and 22 healthy volunteers, we found little evidence for between-group differences in VS RPE and dmPFC learning rate signals, as measured with fMRI. However, a follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. These findings point to an impairment in learning rate modulation in PSZ, leading to a reduced ability to adjust task behavior in response to unexpected outcomes. At the level of the brain, learning rate modulation deficits may be associated with decreased involvement of the dmPFC within a greater RL network.

From neuroimaging to daily functioning: A multimethod analysis of reward anticipation in people with schizophrenia

Negative symptoms are a core clinical feature of schizophrenia that are only marginally responsive to current treatments. Recent work suggests that deficits in reinforcement learning and anticipatory responses to reward may be two mechanisms that help explain impairments in motivation in those with schizophrenia. The present study utilized a reinforcement-learning paradigm, which allowed us to examine both reward anticipation and reinforcement learning. Twenty-eight people with schizophrenia and 30 healthy controls completed a reinforcement-learning task while undergoing functional MRI. Participants with schizophrenia also completed a weeklong ecological momentary assessment protocol reporting anticipated motivation and pleasure in their daily activities. Unexpectedly, we found no significant group differences in performance or neural response in reinforcement learning. However, we found that poorer reward learning was associated with greater clinician ratings of negative symptoms and daily reports of anticipatory motivation and pleasure negative symptoms. In regards to anticipatory responses, we found that people with schizophrenia showed blunted activation in the anterior cingulate, insula, caudate, and putamen while anticipating reward. Further, blood oxygen level-dependent (BOLD) response in reward related regions during anticipation of reward was significantly related to both clinician-rated motivation and pleasure deficits as well as daily reports of motivation and pleasure. Our results provide further evidence of deficits during reward anticipation in individuals with schizophrenia, particularly for those with severe negative symptoms, and some evidence for worse reward learning among those with greater negative symptoms. Moreover, our findings suggest that these deficits show important relationships with emotional and motivational functioning in everyday life. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Effortful goal-directed behavior in schizophrenia: Computational subtypes and associations with cognition

Schizophrenia is associated with amotivation and reduced goal-directed behavior, which have been linked to poor functional outcomes. Motivational deficits in schizophrenia are often measured using effort-based decision-making (EBDM) paradigms, revealing consistent alterations in effort expenditure relative to controls. Although these results have generally been interpreted in terms of decreased motivation, the ability to use trial-by-trial changes in reward magnitude or probability of receipt to guide effort allocation may also be affected by cognitive deficits. To date, it remains unclear whether altered performance in EBDM primarily reflects deficits in motivation, cognitive functioning, or both. We applied a newly developed computational modeling approach to the analysis of EBDM data from two previously collected samples comprising 153 patients and 105 controls to determine the extent to which individuals did or did not use available information about reward and probability to guide effort allocation. Half of the participants with schizophrenia failed to incorporate information about reward and probability when making effort-expenditure decisions. The subset of patients who exhibited difficulties using reward and probability information were characterized by greater impairments across measures of cognitive functioning. Interestingly, even within the subset of patients who successfully used reward and probability information to guide effort expenditure, higher levels of negative symptoms related to motivation and avolition were associated with greater effort aversion during the task. Taken together, these data suggest that prior reports of aberrant EBDM in schizophrenia patients are related to both cognitive function and individual differences in negative symptoms. (PsycINFO Database Record (c) 2019 APA, all rights reserved).