Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia

Basal ganglia deep brain stimulation restores cognitive flexibility and exploration-exploitation balance disrupted by NMDA-R antagonism

Learning thrives on cognitive flexibility and exploration. Subjects with schizophrenia have impaired cognitive flexibility and maladaptive exploration patterns. The basal ganglia-dorsolateral prefrontal cortex (BG-DLPFC) network plays a significant role in learning processes. However, how this network maintains cognitive flexibility and exploration patterns and what alters these patterns in schizophrenia remains elusive. Using a combination of extracellular recordings, pharmacological manipulations, macro-stimulation techniques, and mathematical modeling, we show that in the nonhuman primate (NHP), the external segment of the globus pallidus (GPe, the central nucleus of the BG network) modulates cognitive flexibility and exploration patterns (experiments were done in females only). We found that chronic, low-dose administration of N-methyl-D-aspartate receptor (NMDA-R) antagonist, phencyclidine (PCP), decreases directed exploration but increases random exploration, as seen in schizophrenia. In line with adaptive working-memory reinforcement-learning models of the BG-DLPFC network, low-frequency GPe macro-stimulation restores the balance of both exploration types. Our findings suggest that exploration-exploitation imbalance reflects abnormal BG-DLPFC activity and that GPe stimulation may restore it.

An Examination of Intolerance of Uncertainty in Schizophrenia

INTRODUCTION

Schizophrenia is associated with multiple comorbidities and symptoms, suggestive of common transdiagnostic processes. Elevated intolerance of uncertainty (IU) is one such transdiagnostic process, but little research has been conducted on IU in schizophrenia.

METHODS

This study assessed the associations between IU, schizophrenia diagnosis and schizophrenia symptoms using a between-group cross-sectional design. The sample comprised 113 participants, 72 people with a schizophrenia diagnosis and 41 control participants without (73 male, 40 female, age range 19-69 [M = 42.1, SD = 13.0]). Measures of schizophrenia symptoms, IU, depression and anxiety symptoms, rumination, executive functioning and rumination were taken.

RESULTS

Schizophrenia diagnosis was predicted by lower prospective IU and higher levels of inhibitory IU. Specifically, the prospective subscale was uniquely associated with general schizophrenia symptoms. Higher levels of positive symptoms were associated with higher prospective IU and lower inhibitory IU, however, not when anxiety and depressive symptoms, rumination and verbal fluency were controlled for. No unique associations were found with negative symptoms.

CONCLUSIONS

Different directions of association between the subtypes of IU and schizophrenia diagnosis as well as distinct relationships between IU and symptom subtypes suggest that prospective and inhibitory IU have distinct associations with the disorder.

A review on exploration-exploitation trade-off in psychiatric disorders

Balancing exploration and exploitation is a crucial aspect of adaptive decision-making, but psychiatric disorders can disrupt this balance in various ways, shedding light on their neurocognitive roots and guiding targeted interventions. In this systematic review, we aimed to delineate potential exploration-exploitation impairments across psychiatric disorders. Through a thorough search on PubMed, we identified forty-six relevant studies employing tasks probing exploration-exploitation balances, which we synthesized to reveal distinct patterns. These disorders are clustered into three categories: addictive patterns, emotional/cognitive disturbances, and neurological (neurodevelopmental and neurodegenerative) disorders. Our findings show that anxiety and mood disorders often enhance exploratory behaviors, while depression impact decision stability and reward sensitivity. In contrast, schizophrenia, OCD (Obsessive-Compulsive Disorder), and ADHD (Attention-Deficit/Hyperactivity Disorder) are characterized by excessive switching and difficulties in balancing exploration and exploitation, leading to impaired learning and adaptability. Additionally, disorders with addictive-like features disrupt optimal decision-making strategies by either heightening exploration or causing maladaptive persistence, thus skewing the balance away from effective decision-making. Individuals exhibiting addiction-like or compulsive behaviors often demonstrate imbalances in the explore-exploit trade-off, resulting in suboptimal decision-making characterized by reduced exploration, flawed foraging strategies, and impulsive or perseverative choices despite adverse outcomes. This suggests that such disorders may originate from dysfunctional foraging processes applied to decision-making. In sum, different patterns of exploration-exploitation balance in different disorders are crucial in understanding the difficulties in learning and decision making of neuropsychiatric disorders. This suggests that such disorders may stem from dysregulated decision-making processes, where uncertainty plays a central role. Dysfunctions in dopaminergic and noradrenergic pathways appear to disrupt the brain's representation of uncertainty, thereby altering exploratory behavior. In sum, the varying patterns of exploration-exploitation balance across different disorders are critical for understanding the challenges in learning and decision-making associated with neuropsychiatric conditions.

Probabalistic reinforcement learning impairments predict negative symptom severity and risk for conversion in youth at clinical high-risk for psychosis

BACKGROUND

Elucidation of transphasic mechanisms (i.e., mechanisms that occur across illness phases) underlying negative symptoms could inform early intervention and prevention efforts and additionally identify treatment targets that could be effective regardless of illness stage. This study examined whether a key reinforcement learning behavioral pattern characterized by reduced difficulty learning from rewards that have been found to underlie negative symptoms in those with a schizophrenia diagnosis also contributes to negative symptoms in those at clinical high-risk (CHR) for psychosis.

METHODS

CHR youth (n = 46) and 51 healthy controls (CN) completed an explicit reinforcement learning task with two phases. During the acquisition phase, participants learned to select between pairs of stimuli probabilistically reinforced with feedback indicating receipt of monetary gains or avoidance of losses. Following training, the transfer phase required participants to select between pairs of previously presented stimuli during the acquisition phase and novel stimuli without receiving feedback. These test phase pairings allowed for inferences about the contributions of prediction error and value representation mechanisms to reinforcement learning deficits.

RESULTS

In acquisition, CHR participants displayed impaired learning from gains specifically that were associated with greater negative symptom severity. Transfer performance indicated these acquisition deficits were largely driven by value representation deficits. In addition to negative symptoms, this profile of deficits was associated with a greater risk of conversion to psychosis and lower functioning.

CONCLUSIONS

Impairments in positive reinforcement learning, specifically effectively representing reward value, may be an important transphasic mechanism of negative symptoms and a marker of psychosis liability.

Corticolimbic circuitry as a druggable target in schizophrenia spectrum disorders: a narrative review

Schizophrenia spectrum disorders (SSD) involve disturbances in the integration of perception, emotion and cognition. The corticolimbic system is an interacting set of cortical and subcortical brain regions critically involved in this process. Understanding how neural circuitry and molecular mechanisms within this corticolimbic system may contribute to the development of not only positive symptoms but also negative and cognitive deficits in SSD has been a recent focus of intense research, as the latter are not adequately treated by current antipsychotic medications and are more strongly associated with poorer functioning and long-term outcomes. This review synthesises recent developments examining corticolimbic dysfunction in the pathophysiology of SSD, with a focus on neuroimaging advances and related novel methodologies that enable the integration of data across different scales. We then integrate how these findings may inform the identification of novel therapeutic and preventive targets for SSD symptomatology. A range of pharmacological interventions have shown initial promise in correcting corticolimbic dysfunction and improving negative, cognitive and treatment-resistant symptoms. We discuss current challenges and opportunities for improving the still limited translation of these research findings into clinical practice. We argue how our knowledge of the role of corticolimbic dysfunction can be improved by combining multiple research modalities to examine hypotheses across different spatial and temporal scales, combining neuroimaging with experimental interventions and utilising large-scale consortia to advance biomarker identification. Translation of these findings into clinical practice will be aided by consideration of optimal intervention timings, biomarker-led patient stratification, and the development of more selective medications.

Modeling Decision-Making in Schizophrenia: Associations Between Computationally Derived Risk Propensity and Self-Reported Risk Perception

BACKGROUND AND HYPOTHESIS

Schizophrenia is associated with a decreased pursuit of risky rewards during uncertain-risk decision-making. However, putative mechanisms subserving this disadvantageous risky reward pursuit, such as contributions of cognition and relevant traits, remain poorly understood.

STUDY DESIGN

Participants (30 schizophrenia/schizoaffective disorder [SZ]; 30 comparison participants [CP]) completed the Balloon Analogue Risk Task (BART). Computational modeling captured subprocesses of uncertain-risk decision-making: Risk Propensity, Prior Belief of Success, Learning Rate, and Behavioral Consistency. IQ, self-reported risk-specific processes (ie, Perceived Risks and Expected Benefit of Risks), and non-risk-specific traits (ie, defeatist beliefs; hedonic tone) were examined for relationships with Risk Propensity to determine what contributed to differences in risky reward pursuit.

STUDY RESULTS

On the BART, the SZ group exhibited lower Risk Propensity, higher Prior Beliefs of Success, and comparable Learning Rates. Furthermore, Risk Propensity was positively associated with IQ across groups. Linear models predicting Risk Propensity revealed 2 interactions: 1 between group and Perceived Risk, and 1 between IQ and Perceived Risk. Specifically, in both the SZ group and individuals with below median IQ, lower Perceived Risks was related to lower Risk Propensity. Thus, lower perception of financial risks was associated with a less advantageous pursuit of uncertain-risk rewards.

CONCLUSIONS

Findings suggest consistently decreased risk-taking on the BART in SZ may reflect risk imperception, the failure to accurately perceive and leverage relevant information to guide the advantageous pursuit of risky rewards. Additionally, our results highlight the importance of cognition in uncertain-risk decision-making.

A causal role of the right dorsolateral prefrontal cortex in random exploration

Decision to explore new options with uncertain outcomes or exploit familiar options with known outcomes is a fundamental challenge that the brain faces in almost all real-life decisions. Previous studies have shown that humans use two main explorative strategies to negotiate this explore-exploit tradeoff. Exploring for the sake of information is called directed exploration, and exploration driven by behavioral variability is known as random exploration. While previous neuroimaging studies have shown different neural correlates for these explorative strategies, including right frontopolar cortex (FPC), right dorsolateral prefrontal cortex (DLPFC), and dorsal anterior cingulate cortex (dACC), there is still a lack of causal evidence for most of these brain regions. Here, we focused on the right DLPFC, which was previously supported to be involved in exploration. Using the continuous theta burst stimulation (cTBS) and Horizon task on twenty-five healthy right-handed adult participants, we showed that inhibiting rDLPFC did not change directed exploration but selectively reduced random exploration, by increasing reward sensitivity over the average reward of each option. This suggests a causal role for rDLPFC in random exploration, and further supports dissociable neural implementations for these two explorative strategies.

Reviewing explore/exploit decision-making as a transdiagnostic target for psychosis, depression, and anxiety

In many everyday decisions, individuals choose between trialling something novel or something they know well. Deciding when to try a new option or stick with an option that is already known to you, known as the "explore/exploit" dilemma, is an important feature of cognition that characterises a range of decision-making contexts encountered by humans. Recent evidence has suggested preferences in explore/exploit biases are associated with psychopathology, although this has typically been examined within individual disorders. The current review examined whether explore/exploit decision-making represents a promising transdiagnostic target for psychosis, depression, and anxiety. A systematic search of academic databases was conducted, yielding a total of 29 studies. Studies examining psychosis were mostly consistent in showing that individuals with psychosis explored more compared with individuals without psychosis. The literature on anxiety and depression was more heterogenous; some studies found that anxiety and depression were associated with more exploration, whereas other studies demonstrated reduced exploration in anxiety and depression. However, examining a subset of studies that employed case-control methods, there was some evidence that both anxiety and depression also were associated with increased exploration. Due to the heterogeneity across the literature, we suggest that there is insufficient evidence to conclude whether explore/exploit decision-making is a transdiagnostic target for psychosis, depression, and anxiety. However, alongside our advisory groups of lived experience advisors, we suggest that this context of decision-making is a promising candidate that merits further investigation using well-powered, longitudinal designs. Such work also should examine whether biases in explore/exploit choices are amenable to intervention.

Exploration-Exploitation and Suicidal Behavior in Borderline Personality Disorder and Depression

Importance

Clinical theory and behavioral studies suggest that people experiencing suicidal crisis are often unable to find constructive solutions or incorporate useful information into their decisions, resulting in premature convergence on suicide and neglect of better alternatives. However, prior studies of suicidal behavior have not formally examined how individuals resolve the tradeoffs between exploiting familiar options and exploring potentially superior alternatives.

Objective

To investigate exploration and exploitation in suicidal behavior from the formal perspective of reinforcement learning.

Design, Setting, and Participants

Two case-control behavioral studies of exploration-exploitation of a large 1-dimensional continuous space and a 21-day prospective ambulatory study of suicidal ideation were conducted between April 2016 and March 2022. Participants were recruited from inpatient psychiatric units, outpatient clinics, and the community in Pittsburgh, Pennsylvania, and underwent laboratory and ambulatory assessments. Adults diagnosed with borderline personality disorder (BPD) and midlife and late-life major depressive disorder (MDD) were included, with each sample including demographically equated groups with a history of high-lethality suicide attempts, low-lethality suicide attempts, individuals with BPD or MDD but no suicide attempts, and control individuals without psychiatric disorders. The MDD sample also included a subgroup with serious suicidal ideation.

Main Outcomes and Measures

Behavioral (model-free and model-derived) indices of exploration and exploitation, suicide attempt lethality (Beck Lethality Scale), and prospectively assessed suicidal ideation.

Results

The BPD group included 171 adults (mean [SD] age, 30.55 [9.13] years; 135 [79%] female). The MDD group included 143 adults (mean [SD] age, 62.03 [6.82] years; 81 [57%] female). Across the BPD (χ23 = 50.68; P < .001) and MDD (χ24 = 36.34; P < .001) samples, individuals with high-lethality suicide attempts discovered fewer options than other groups as they were unable to shift away from unrewarded options. In contrast, those with low-lethality attempts were prone to excessive behavioral shifts after rewarded and unrewarded actions. No differences were seen in strategic early exploration or in exploitation. Among 84 participants with BPD in the ambulatory study, 56 reported suicidal ideation. Underexploration also predicted incident suicidal ideation (χ21 = 30.16; P < .001), validating the case-control results prospectively. The findings were robust to confounds, including medication exposure, affective state, and behavioral heterogeneity.

Conclusions and Relevance

The findings suggest that narrow exploration and inability to abandon inferior options are associated with serious suicidal behavior and chronic suicidal thoughts. By contrast, individuals in this study who engaged in low-lethality suicidal behavior displayed a low threshold for taking potentially disadvantageous actions.

You are How You Eat: Foraging Behavior as a Potential Novel Marker of Rat Affective State

Effective and safe foraging requires animals to behave according to the expectations they have about the rewards, threats, and costs in their environment. Since these factors are thought to be reflected in the animals' affective states, we can use foraging behavior as a window into those states. In this study, rats completed a foraging task in which they had repeatedly to decide whether to continue to harvest a food source despite increasing time costs, or to forgo food to switch to a different food source. Rats completed this task across two experiments using manipulations designed to induce both positive and negative, and shorter- and longer- term changes in affective state: removal and return of enrichment (Experiment 1), implementation and reversal of an unpredictable housing treatment (Experiment 1), and delivery of rewards (tickling or sucrose) and punishers (air-puff or back-handling) immediately prior to testing (Experiment 2). In Experiment 1, rats completed fewer trials and were more prone to switching between troughs when housed in standard, compared to enriched, housing conditions. In Experiment 2, rats completed more trials following pre-test tickling compared to pre-test sucrose delivery. However, we also found that they were prone to disengaging from the task, suggesting they were really choosing between three options: 'harvest', 'switch', or 'not work'. This limits the straightforward interpretation of the results. At present, foraging behavior within the context of this task cannot reliably be used as an indicator of an affective state in animals.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-024-00242-4.

Enhanced Punishment Responses in Patients With Schizophrenia: An Event-Related Potential Study

It is well known that abnormal reward processing is a characteristic feature of various psychopathologies including schizophrenia. Reduced reward anticipation has been suggested as a core symptom of schizophrenia. The Monetary Incentive Delay Task (MID) is frequently used to detect reward anticipation. The present study aims to evaluate the amplitude and latency of event-related potential (ERP) P300 in patients with schizophrenia (SCH) compared to healthy controls during the MID task. Twenty patients with SCH and 21 demographically matched healthy controls (HC) were included in the study. ERP P300 amplitude and latency values were compared between groups using an MID task in which reward and loss cues were presented. Relations between P300 and clinical facets were investigated in the patient group. SCH group had enhanced mean P300 amplitudes and delayed peak latency in the punishment condition compared with HC. These higher responses were also associated with negative symptoms. SCH group showed altered reward processing as being more sensitive to loss of reward conditions as firstly evidenced by electrophysiological methods, possibly due to abnormality in various systems including social withdrawal, social defeat, and behavioral inhibition system.

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Reduced Reward Processing in Schizophrenia: A Comprehensive EEG Event-Related Oscillation Study

It is well known that abnormal reward processing is a characteristic feature of various psychopathologies including schizophrenia (SZ). Reduced reward anticipation has been suggested as a core symptom of SZ. The present study aims to evaluate the event-related oscillations (EROs) delta, theta, alpha, beta, and gamma in patients with SZ during the Monetary Incentive Delay (MID) task, which elicits the neural activity of reward processing. Twenty-one patients with SZ and twenty-two demographically matched healthy controls were included in the study. EROs were compared between groups and correlation analyses were conducted to determine a possible relationship between clinical scores and ERO values. Compared with healthy controls, the SZ group had reduced (1) delta and theta amplitudes in the reward condition (2) total beta and non-incentive cue-related beta amplitudes, and (3) incentive cue-related frontal gamma amplitudes. These reductions can be interpreted as impaired dopaminergic neurotransmission and disrupted cognitive functioning in the reward processing of SZ. In contrast, SZ patients showed higher incentive cue-related theta and occipital gamma amplitudes compared to controls. These increments may reflect negative symptoms in SZ. Moreover, theta amplitudes showed a negative correlation with Calgary Depression Scale for Schizophrenia scores and a positive correlation with attentional impulsivity. This is the first study showing the impairments of SZ patients in EROs from delta to gamma frequency bands compared with healthy controls during reward anticipation. Being the first comprehensive study, our results can be interpreted as providing evidence for disrupted brain dynamics in the reward processing of SZ studied by EROs. It may become possible to help patients' wellness by improving our understanding of reward processing in schizophrenia and developing innovative rehabilitation treatments based on these findings.

Schizophrenia: from neurochemistry to circuits, symptoms and treatments

Schizophrenia is a leading cause of global disability. Current pharmacotherapy for the disease predominantly uses one mechanism - dopamine D2 receptor blockade - but often shows limited efficacy and poor tolerability. These limitations highlight the need to better understand the aetiology of the disease to aid the development of alternative therapeutic approaches. Here, we review the latest meta-analyses and other findings on the neurobiology of prodromal, first-episode and chronic schizophrenia, and the link to psychotic symptoms, focusing on imaging evidence from people with the disorder. This evidence demonstrates regionally specific neurotransmitter alterations, including higher glutamate and dopamine measures in the basal ganglia, and lower glutamate, dopamine and γ-aminobutyric acid (GABA) levels in cortical regions, particularly the frontal cortex, relative to healthy individuals. We consider how dysfunction in cortico-thalamo-striatal-midbrain circuits might alter brain information processing to underlie psychotic symptoms. Finally, we discuss the implications of these findings for developing new, mechanistically based treatments and precision medicine for psychotic symptoms, as well as negative and cognitive symptoms.

Reinforcement learning profiles and negative symptoms across chronic and clinical high-risk phases of psychotic illness

Negative symptoms are prominent in individuals with schizophrenia (SZ) and youth at clinical high-risk for psychosis (CHR). In SZ, negative symptoms are linked to reinforcement learning (RL) dysfunction; however, previous research suggests implicit RL remains intact. It is unknown whether implicit RL is preserved in the CHR phase where negative symptom mechanisms are unclear, knowledge of which may assist in developing early identification and prevention methods. Participants from two studies completed an implicit RL task: Study 1 included 53 SZ individuals and 54 healthy controls (HC); Study 2 included 26 CHR youth and 23 HCs. Bias trajectories reflecting implicit RL were compared between groups and correlations with negative symptoms were examined. Cluster analysis investigated RL profiles across the combined samples. Implicit RL was comparable between HC and their corresponding SZ and CHR groups. However, cluster analysis was able to parse performance heterogeneity across diagnostic boundaries into two distinct RL profiles: a Positive/Early Learning cluster (65% of participants) with positive bias scores increasing from the first to second task block, and a Negative/Late Learning cluster (35% of participants) with negative bias scores increasing from the second to third block. Clusters did not differ in the proportion of CHR vs. SZ cases; however, the Negative/Late Learning cluster had more severe negative symptoms. Although implicit RL is intact in CHR similar to SZ, distinct implicit RL phenotypic profiles with elevated negative symptoms were identified trans-phasically, suggesting distinct reward-processing mechanisms can contribute to negative symptoms independent of phases of illness.

Interindividual Brain and Behavior Differences in Adaptation to Unexpected Uncertainty

To adapt to a new environment, individuals must alternate between exploiting previously learned "action-consequence" combinations and exploring new actions for which the consequences are unknown: they face an exploration/exploitation trade-off. The neural substrates of these behaviors and the factors that may relate to the interindividual variability in their expression remain overlooked, in particular when considering neural connectivity patterns. Here, to trigger environmental uncertainty, false feedbacks were introduced in the second phase of an associative learning task. Indices reflecting exploitation and cost of uncertainty were computed. Changes in the intrinsic connectivity were determined using resting-state functional connectivity (rFC) analyses before and after performing the "cheated" phase of the task in the MRI. We explored their links with behavioral and psychological factors. Dispersion in the participants' cost of uncertainty was used to categorize two groups. These groups showed different patterns of rFC changes. Moreover, in the overall sample, exploitation was correlated with rFC changes between (1) the anterior cingulate cortex and the cerebellum region 3, and (2) the left frontal inferior gyrus (orbital part) and the right frontal inferior gyrus (triangular part). Anxiety and doubt about action propensity were weakly correlated with some rFC changes. These results demonstrate that the exploration/exploitation trade-off involves the modulation of cortico-cerebellar intrinsic connectivity.

A computational neuroimaging study of reinforcement learning and goal-directed exploration in schizophrenia spectrum disorders

BACKGROUND

Prior evidence indicates that negative symptom severity and cognitive deficits, in people with schizophrenia (PSZ), relate to measures of reward-seeking and loss-avoidance behavior (implicating the ventral striatum/VS), as well as uncertainty-driven exploration (reliant on rostrolateral prefrontal cortex/rlPFC). While neural correlates of reward-seeking and loss-avoidance have been examined in PSZ, neural correlates of uncertainty-driven exploration have not. Understanding neural correlates of uncertainty-driven exploration is an important next step that could reveal insights to how this mechanism of cognitive and negative symptoms manifest at a neural level.

METHODS

We acquired fMRI data from 29 PSZ and 36 controls performing the Temporal Utility Integration decision-making task. Computational analyses estimated parameters corresponding to learning rates for both positive and negative reward prediction errors (RPEs) and the degree to which participates relied on representations of relative uncertainty. Trial-wise estimates of expected value, certainty, and RPEs were generated to model fMRI data.

RESULTS

Behaviorally, PSZ demonstrated reduced reward-seeking behavior compared to controls, and negative symptoms were positively correlated with loss-avoidance behavior. This finding of a bias toward loss avoidance learning in PSZ is consistent with previous work. Surprisingly, neither behavioral measures of exploration nor neural correlates of uncertainty in the rlPFC differed significantly between groups. However, we showed that trial-wise estimates of relative uncertainty in the rlPFC distinguished participants who engaged in exploratory behavior from those who did not. rlPFC activation was positively associated with intellectual function.

CONCLUSIONS

These results further elucidate the nature of reinforcement learning and decision-making in PSZ and healthy volunteers.

Trust and psychosis: a systematic review and meta-analysis

BACKGROUND

Impaired trust in other humans is commonly seen in psychosis and it leads to poor societal functioning. However, examining trust behavior in an experimental setting is challenging. Investigators have used the trust game, a neuro-economic game to assess trust behavior in psychosis. However, the findings are inconsistent. Hence, we systematically reviewed the existing literature and conducted a meta-analysis to examine trust behavior in patients with psychosis, their relatives, and those at high risk for psychosis.

METHODS

We searched electronic databases for studies that have examined trust game in patients with psychosis, published up to November 2021. The primary outcome measure was the baseline trust in a trust game by patients and controls. The meta-analysis was performed if at least three data sets of control and patient groups were available for that measure/design. We conducted meta-analyses with a random-effects model. The results were described narratively wherever meta-analysis was not possible due to paucity of studies.

RESULTS

The searches across the databases including cross-references yielded 465 publications of which 10 studies were included in the final analysis. Baseline trust in the trust game was significantly lower in patients with psychosis compared to controls (SMD 0.39, 95% CI -0.14 to 0.64, p -0.002). However, a similar decrease in baseline trust was not present in relatives of patients (SMD 0.08, 95% CI -0.20 to 0.36, p -0.58).

CONCLUSIONS

The current meta-analysis suggests significant trust deficits in patients with psychosis. Future studies with a bigger sample size are required to understand the nature of trust deficits and factors affecting this impairment.

Integrative Brain Network and Salience Models of Psychopathology and Cognitive Dysfunction in Schizophrenia

Brain network models of cognitive control are central to advancing our understanding of psychopathology and cognitive dysfunction in schizophrenia. This review examines the role of large-scale brain organization in schizophrenia, with a particular focus on a triple-network model of cognitive control and its role in aberrant salience processing. First, we provide an overview of the triple network involving the salience, frontoparietal, and default mode networks and highlight the central role of the insula-anchored salience network in the aberrant mapping of salient external and internal events in schizophrenia. We summarize the extensive evidence that has emerged from structural, neurochemical, and functional brain imaging studies for aberrancies in these networks and their dynamic temporal interactions in schizophrenia. Next, we consider the hypothesis that atypical striatal dopamine release results in misattribution of salience to irrelevant external stimuli and self-referential mental events. We propose an integrated triple-network salience-based model incorporating striatal dysfunction and sensitivity to perceptual and cognitive prediction errors in the insula node of the salience network and postulate that dysregulated dopamine modulation of salience network-centered processes contributes to the core clinical phenotype of schizophrenia. Thus, a powerful paradigm to characterize the neurobiology of schizophrenia emerges when we combine conceptual models of salience with large-scale cognitive control networks in a unified manner. We conclude by discussing potential therapeutic leads on restoring brain network dysfunction in schizophrenia.

A low dimensional manifold of human exploratory behavior reveals opposing roles for apathy and anxiety

Exploration-exploitation decision-making is a feature of daily life that is altered in a number of neuropsychiatric conditions. Humans display a range of exploration and exploitation behaviors, which can be affected by apathy and anxiety. It remains unknown how factors underlying decision-making generate the spectrum of observed exploration-exploitation behavior and how they relate to states of anxiety and apathy. Here, we report a latent structure underlying sequential exploration and exploitation decisions that explains variation in anxiety and apathy. 1001 participants in a gender-balanced sample completed a three-armed restless bandit task along with psychiatric symptom surveys. Using dimensionality reduction methods, we found that decision sequences reduced to a low-dimensional manifold. The axes of this manifold explained individual differences in the balance between states of exploration and exploitation and the stability of those states, as determined by a statistical mechanics model of decision-making. Position along the balance axis was correlated with opposing symptoms of behavioral apathy and anxiety, while position along the stability axis correlated with the level of emotional apathy. This result resolves a paradox over how these symptoms can be correlated in samples but have opposite effects on behavior. Furthermore, this work provides a basis for using behavioral manifolds to reveal relationships between behavioral dynamics and affective states, with important implications for behavioral measurement approaches to neuropsychiatric conditions.

The reliability and validity of the revised Green et al. paranoid thoughts scale in individuals at clinical high-risk for psychosis

INTRODUCTION

Paranoia is a common and impairing psychosis symptom, which exists along a severity continuum that extends into the general population. Individuals at clinical high-risk for psychosis (CHR) frequently experience paranoia and this may elevate their risk for developing full psychosis. Nonetheless, limited work has examined the efficient measurement of paranoia in CHR individuals. The present study aimed to validate an often-used self-report measure, the revised green paranoid thoughts scale (RGPTS), in this critical population.

METHOD

Participants were CHR individuals (n = 103), mixed clinical controls (n = 80), and healthy controls (n = 71) who completed self-report and interview measures. Confirmatory factor analysis (CFA), psychometric indices, group differences, and relations to external measures were used to evaluate the reliability and validity of the RGPTS.

RESULTS

CFA replicated a two-factor structure for the RGPTS and the associated reference and persecution scales were reliable. CHR individuals scored significantly higher on both reference and persecution, relative to both healthy (ds = 1.03, 0.86) and clinical controls (ds = 0.64, 0.73). In CHR participants, correlations between reference and persecution and external measures were smaller than expected, though showed evidence of discriminant validity (e.g., interviewer-rated paranoia, r = 0.24). When examined in the full sample, correlation magnitude was larger and follow-up analyses indicated that reference related most specifically to paranoia (β = 0.32), whereas persecution uniquely related to poor social functioning (β = -0.29).

CONCLUSION

These results demonstrate the reliability and validity of the RGPTS, though its scales related more weakly to severity in CHR individuals. The RGPTS may be useful in future work aiming to develop symptom-specific models of emerging paranoia in CHR individuals.

Negative Symptom Inventory-Self-Report (NSI-SR): Initial development and validation

Negative symptoms (i.e., anhedonia, avolition, asociality, blunted affect, alogia) are frequently observed in the schizophrenia-spectrum (SZ) and associated with functional disability. While semi-structured interviews of negative symptoms represent a gold-standard approach, they require specialized training and may be vulnerable to rater biases. Thus, brief self-report questionnaires measuring negative symptoms may be useful. Existing negative symptom questionnaires demonstrate that this approach may be promising in schizophrenia, but no measure has been devised for use across stages of psychotic illness. The present study reports initial psychometric validation of the Negative Symptom Inventory-Self-Report (NSI-SR), the self-report counterpart of the Negative Symptom Inventory-Psychosis Risk clinical interview. The NSI-SR is a novel transphasic negative symptoms measure assessing the domains of anhedonia, avolition, and asociality. The NSI-SR and related measures were administered to two samples: 1) undergraduates (n = 335), 2) community participants, including: SZ (n = 32), clinical-high risk for psychosis (CHR, n = 25), and healthy controls matched to SZ (n = 31) and CHR (n = 30). The psychometrically trimmed 11-item NSI-SR showed good internal consistency and a three-factor solution reflecting avolition, asociality, and anhedonia. The NSI-SR demonstrated convergent validity via moderate to large correlations with clinician-rated negative symptoms and related constructs in both samples. Discriminant validity was supported by lower correlations with positive symptoms in both samples; however, correlations with positive symptoms were still significant. These initial psychometric findings suggest that the NSI-SR is a reliable and valid brief questionnaire capable of measuring negative symptoms across phases of psychotic illness.

"Goals in Focus"-a targeted CBT approach for motivational negative symptoms of psychosis: study protocol for a randomized-controlled feasibility trial

BACKGROUND

The reduction of goal-directed behavior is the main characteristic in motivational negative symptoms of psychosis as it accounts for the long-term decline in psychological well-being and psychosocial functioning. However, the available treatment options are largely unspecific and show only small effects on motivational negative symptoms. Interventions that directly target the relevant psychological mechanisms are likely to be more effective. For "Goals in Focus", we translated findings from basic clinical research on mechanisms underlying motivational negative symptoms into a tailored and comprehensive novel psychological outpatient treatment program. With this study, we will test the feasibility of the therapy manual and the trial procedures. We also aim to examine first estimates of the effect size that can be expected from "Goals in Focus" to inform the sample size calculation of a subsequent fully powered trial.

METHODS

Thirty participants diagnosed with a schizophrenia spectrum disorder and at least moderate motivational negative symptoms will be randomly assigned to either 24 sessions of "Goals in Focus" over the course of 6 months (n = 15) or to a 6-month wait-list control group (n = 15). Single-blind assessments will be conducted at baseline (t₀) and 6 months after baseline completion (t₁). Feasibility outcomes include patient recruitment, retention, and attendance rates. Acceptability will be rated by trial therapists and by participants at end of treatment. Primary outcome for effect size estimation is the motivational negative symptom subscale sum score of the Brief Negative Symptom Scale at t₁ corrected for baseline values. Secondary outcomes include psychosocial functioning, psychological well-being, depressive symptoms, expressive negative symptoms, negative symptom factor scores, and goal pursuit in everyday life.

DISCUSSION

The feasibility and acceptability data will be used to improve trial procedures and the "Goals in Focus" intervention where necessary. The treatment effect on the primary outcome will provide the basis for the sample size calculation for a fully powered RCT.

TRIAL REGISTRATION

1) ClinicalTrials.gov, NCT05252039 . Registered on 23 February 2022. 2) Deutsches Register Klinischer Studien, DRKS00018083 . Registered on 28 August 2019.

Maladaptive explore/exploit trade-offs in schizophrenia

Schizophrenia is a complex disorder that remains poorly understood, particularly at the systems level. In this opinion article we argue that the explore/exploit trade-off concept provides a holistic and ecologically valid framework to resolve some of the apparent paradoxes that have emerged within schizophrenia research. We review recent evidence suggesting that fundamental explore/exploit behaviors may be maladaptive in schizophrenia during physical, visual, and cognitive foraging. We also describe how theories from the broader optimal foraging literature, such as the marginal value theorem (MVT), could provide valuable insight into how aberrant processing of reward, context, and cost/effort evaluations interact to produce maladaptive responses.

Emotion-behaviour decoupling and experiential pleasure deficits predict negative symptoms and functional outcome in first-episode schizophrenia patients

BACKGROUND

Emotion-behaviour decoupling refers to the failure to translate emotion into motivated behaviour, and is a putative marker for schizophrenia. The heterogeneity of experiential pleasure and emotion expressivity deficits has been reported in schizophrenia patients. These three constructs are believed to contribute to negative symptoms, but very few studies have examined their predictive ability for clinical and functional outcome of schizophrenia. This study aimed to clarify whether these three constructs influence clinical and functional outcome of schizophrenia.

METHOD

At baseline, 127 first-episode schizophrenia patients completed a behavioural paradigm for emotion-behaviour decoupling, and self-report scales for experiential pleasure and emotion expressivity deficits. Cluster-analysis was applied to characterize schizophrenia subgroups based on these three constructs. At end-point (mean follow-up = 5.37 years, SD = 1.03 years), 85 schizophrenia patients were reassessed using the Clinical Assessment Interview for Negative Symptoms (CAINS) and a clinician-rated social functioning scale.

RESULTS

Cluster 1 (n = 74) did not show emotion-behaviour decoupling, and had intact experiential pleasure and emotion expressivity. Cluster 2 (n = 29) showed emotion-behaviour decoupling and experiential pleasure deficits. Cluster 3 (n = 24) showed emotion expressivity deficits. At endpoint, the three clusters differed significantly in CAINS MAP factor (p = 0.016) and social functioning (p = 0.019), but not CAINS EXP factor. Specifically, Cluster 2 (n = 18) showed more severe negative symptoms of CAINS MAP factor (p = 0.046) and poorer social functioning (p = 0.022) than Cluster 1 (n = 49). Cluster 3 (n = 18) did not differ from Cluster 1 and Cluster 2 in negative symptoms and social functioning.

DISCUSSION

Emotion-behaviour decoupling and experiential pleasure deficits predicted clinical and functional outcome of schizophrenia.

Negative symptoms, striatal dopamine and model-free reward decision-making in schizophrenia

Negative symptoms, such as lack of motivation or social withdrawal, are highly prevalent and debilitating in patients with schizophrenia. Underlying mechanisms of negative symptoms are incompletely understood, thereby preventing the development of targeted treatments. We hypothesized that in patients with schizophrenia during psychotic remission, impaired influences of both model-based and model-free reward predictions on decision-making ('reward prediction influence', RPI) underlie negative symptoms. We focused on psychotic remission, because psychotic symptoms might confound reward-based decision-making. Moreover, we hypothesized that impaired model-based/model-free RPIs depend on alterations of both associative striatum dopamine synthesis and storage (DSS) and executive functioning. Both factors influence RPI in healthy subjects and are typically impaired in schizophrenia. Twenty-five patients with schizophrenia with pronounced negative symptoms during psychotic remission and 24 healthy controls were included in the study. Negative symptom severity was measured by the Positive and Negative Syndrome Scale negative subscale, model-based/model-free RPI by the two-stage decision task, associative striatum DSS by 18F-DOPA positron emission tomography and executive functioning by the symbol coding task. Model-free RPI was selectively reduced in patients and associated with negative symptom severity as well as with reduced associative striatum DSS (in patients only) and executive functions (both in patients and controls). In contrast, model-based RPI was not altered in patients. Results provide evidence for impaired model-free reward prediction influence as a mechanism for negative symptoms in schizophrenia as well as for reduced associative striatum dopamine and executive dysfunction as relevant factors. Data suggest potential treatment targets for patients with schizophrenia and pronounced negative symptoms.

Aberrant reward processing to positive versus negative outcomes across psychotic disorders

Several studies of reward processing in schizophrenia have shown reduced sensitivity to positive, but not negative, outcomes although inconsistencies have been reported. In addition, few studies have investigated whether patients show a relative deficit to social versus nonsocial rewards, whether deficits occur across the spectrum of psychosis, or whether deficits relate to negative symptoms and functioning. This study examined probabilistic implicit learning via two visually distinctive slot machines for social and nonsocial rewards in 101 outpatients with diverse psychotic disorders and 48 community controls. The task consisted of two trial types: positive (optimal to choose a positive vs. neutral machine) and negative (optimal to choose a neutral vs. negative machine), with two reward conditions: social (faces) and nonsocial (money) reward conditions. A significant group X trial type interaction indicated that controls performed better on positive than negative trials, whereas patients showed the opposite pattern of better performance on negative than positive trials. In addition, both groups performed better for social than nonsocial stimuli, despite lower overall task performance in patients. Within patients, worse performance on negative trials showed significant, small-to-moderate correlations with motivation and pleasure-related negative symptoms and social functioning. The current findings suggest reward processing disturbances, particularly decreased sensitivity to positive outcomes, extend beyond schizophrenia to a broader spectrum of psychotic disorders and relate to important clinical outcomes.

Social affective forecasting and social anhedonia in schizophrenia-spectrum disorders: a daily diary study

Social anhedonia (SA) is a trait-like phenomenon observed across schizophrenia-spectrum disorders (SSDs). While in-the-moment social pleasure experiences are intact in SSDs, anticipatory pleasure experiences may be disrupted. Thus, the prediction of future emotions in social situations, or social affective forecasting (SAF), may play a role in SA. Therefore, we utilized daily diary methods to examine SAF in SSD and the association between SAF and SA in 34 SSD and 43 non-SSD individuals. SAF was calculated as the absolute difference between anticipatory and consummatory ratings of 13 positive and negative emotions for daily social interactions reported across eight days. Results suggest that individuals with SSDs are less accurate in forecasting negative, but not positive emotions, for future social interactions. Further, poorer forecasting accuracy of negative emotions were associated with elevated levels of SA and lower social pleasure. Together, these data suggest that inaccuracies in forecasting negative emotions may be a worthwhile intervention target for reducing SA in SSDs.

Decision value signals in the ventromedial prefrontal cortex and motivational and hedonic symptoms across mood and psychotic disorders

Deficits in motivation and pleasure are common across many psychiatric disorders, and manifest as symptoms of amotivation and anhedonia, which are prominent features of both mood and psychotic disorders. Here we provide evidence for an association between neural value signals and symptoms of amotivation and anhedonia across adults with major depression, bipolar disorder, schizophrenia, or no psychiatric diagnosis. We found that value signals in the ventromedial prefrontal cortex (vmPFC) during intertemporal decision-making were dampened in individuals with greater motivational and hedonic deficits, after accounting for primary diagnosis. This relationship remained significant while controlling for diagnosis-specific symptoms of mood and psychosis, such as depression as well as positive and negative symptoms. Our results demonstrate that dysfunction in the vmPFC during value-based decision-making is specifically linked to motivational and hedonic impairments. These findings provide a quantitative neural target for the potential development of novel treatments for amotivation and anhedonia.

Using Computational Modeling to Capture Schizophrenia-Specific Reinforcement Learning Differences and Their Implications on Patient Classification

BACKGROUND

Psychiatric diagnosis and treatment have historically taken a symptom-based approach, with less attention on identifying underlying symptom-producing mechanisms. Recent efforts have illuminated the extent to which different underlying circuitry can produce phenotypically similar symptomatology (e.g., psychosis in bipolar disorder vs. schizophrenia). Computational modeling makes it possible to identify and mathematically differentiate behaviorally unobservable, specific reinforcement learning differences in patients with schizophrenia versus other disorders, likely owing to a higher reliance on prediction error-driven learning associated with basal ganglia and underreliance on explicit value representations associated with orbitofrontal cortex.

METHODS

We used a well-established probabilistic reinforcement learning task to replicate those findings in individuals with schizophrenia both on (n = 120) and off (n = 44) antipsychotic medications and included a patient comparison group of bipolar patients with psychosis (n = 60) and healthy control subjects (n = 72).

RESULTS

Using accuracy, there was a main effect of group (F3,279 = 7.87, p < .001), such that all patient groups were less accurate than control subjects. Using computationally derived parameters, both medicated and unmediated individuals with schizophrenia, but not patients with bipolar disorder, demonstrated a reduced mixing parameter (F3,295 = 13.91, p < .001), indicating less dependence on learning explicit value representations as well as greater learning decay between training and test (F1,289 = 12.81, p < .001). Unmedicated patients with schizophrenia also showed greater decision noise (F3,295 = 2.67, p = .04).

CONCLUSIONS

Both medicated and unmedicated patients showed overreliance on prediction error-driven learning as well as significantly higher noise and value-related memory decay, compared with the healthy control subjects and the patients with bipolar disorder. Additionally, the computational model parameters capturing these processes can significantly improve patient/control classification, potentially providing useful diagnosis insight.

Cognitive [Computational] Neuroscience Test Reliability and Clinical Applications for Serious Mental Illness (CNTRaCS) Consortium: Progress and Future Directions

The development of treatments for impaired cognition in schizophrenia has been characterized as the most important challenge facing psychiatry at the beginning of the twenty-first century. The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) project was designed to build on the potential benefits of using tasks and tools from cognitive neuroscience to better understanding and treat cognitive impairments in psychosis. These benefits include: (1) the use of fine-grained tasks that measure discrete cognitive processes; (2) the ability to design tasks that distinguish between specific cognitive domain deficits and poor performance due to generalized deficits resulting from sedation, low motivation, poor test taking skills, etc.; and (3) the ability to link cognitive deficits to specific neural systems, using animal models, neuropsychology, and functional imaging. CNTRICS convened a series of meetings to identify paradigms from cognitive neuroscience that maximize these benefits and identified the steps need for translation into use in clinical populations. The Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRaCS) Consortium was developed to help carry out these steps. CNTRaCS consists of investigators at five different sites across the country with diverse expertise relevant to a wide range of the cognitive systems identified as critical as part of CNTRICs. This work reports on the progress and current directions in the evaluation and optimization carried out by CNTRaCS of the tasks identified as part of the original CNTRICs process, as well as subsequent extensions into the Positive Valence systems domain of Research Domain Criteria (RDoC). We also describe the current focus of CNTRaCS, which involves taking a computational psychiatry approach to measuring cognitive and motivational function across the spectrum of psychosis. Specifically, the current iteration of CNTRaCS is using computational modeling to isolate parameters reflecting potentially more specific cognitive and visual processes that may provide greater interpretability in understanding shared and distinct impairments across psychiatric disorders.

Both unmedicated and medicated individuals with schizophrenia show impairments across a wide array of cognitive and reinforcement learning tasks

BACKGROUND

Schizophrenia is a disorder characterized by pervasive deficits in cognitive functioning. However, few well-powered studies have examined the degree to which cognitive performance is impaired even among individuals with schizophrenia not currently on antipsychotic medications using a wide range of cognitive and reinforcement learning measures derived from cognitive neuroscience. Such research is particularly needed in the domain of reinforcement learning, given the central role of dopamine in reinforcement learning, and the potential impact of antipsychotic medications on dopamine function.

METHODS

The present study sought to fill this gap by examining healthy controls (N = 75), unmedicated (N = 48) and medicated (N = 148) individuals with schizophrenia. Participants were recruited across five sites as part of the CNTRaCS Consortium to complete tasks assessing processing speed, cognitive control, working memory, verbal learning, relational encoding and retrieval, visual integration and reinforcement learning.

RESULTS

Individuals with schizophrenia who were not taking antipsychotic medications, as well as those taking antipsychotic medications, showed pervasive deficits across cognitive domains including reinforcement learning, processing speed, cognitive control, working memory, verbal learning and relational encoding and retrieval. Further, we found that chlorpromazine equivalency rates were significantly related to processing speed and working memory, while there were no significant relationships between anticholinergic load and performance on other tasks.

CONCLUSIONS

These findings add to a body of literature suggesting that cognitive deficits are an enduring aspect of schizophrenia, present in those off antipsychotic medications as well as those taking antipsychotic medications.

Psychological Dimensions Relevant to Motivation and Pleasure in Schizophrenia

Motivation and pleasure deficits are common in schizophrenia, strongly linked with poorer functioning, and may reflect underlying alterations in brain functions governing reward processing and goal pursuit. While there is extensive research examining cognitive and reward mechanisms related to these deficits in schizophrenia, less attention has been paid to psychological characteristics that contribute to resilience against, or risk for, motivation and pleasure impairment. For example, psychological tendencies involving positive future expectancies (e.g., optimism) and effective affect management (e.g., reappraisal, mindfulness) are associated with aspects of reward anticipation and evaluation that optimally guide goal-directed behavior. Conversely, maladaptive thinking patterns (e.g., defeatist performance beliefs, asocial beliefs) and tendencies that amplify negative cognitions (e.g., rumination), may divert cognitive resources away from goal pursuit or reduce willingness to exert effort. Additionally, aspects of sociality, including the propensity to experience social connection as positive reinforcement may be particularly relevant for pursuing social goals. In the current review, we discuss the roles of several psychological characteristics with respect to motivation and pleasure in schizophrenia. We argue that individual variation in these psychological dimensions is relevant to the study of motivation and reward processing in schizophrenia, including interactions between these psychological dimensions and more well-characterized cognitive and reward processing contributors to motivation. We close by emphasizing the value of considering a broad set of modulating factors when studying motivation and pleasure functions in schizophrenia.

From Memories of Past Experiences to Present Motivation? A Meta-analysis on the Association Between Episodic Memory and Negative Symptoms in People With Psychosis

Based on findings from cognitive science, it has been theorized that the reductions in motivation and goal-directed behavior in people with psychosis could stem from impaired episodic memory. In the current meta-analysis, we investigated this putative functional link between episodic memory deficits and negative symptoms. We hypothesized that episodic memory deficits in psychosis would be related to negative symptoms in general but would be more strongly related to amotivation than to reduced expressivity. We included 103 eligible studies (13,622 participants) in the analyses. Results revealed significant, moderate negative associations of episodic memory with negative symptoms in general (k = 103; r = -.23; z = -13.40; P ≤ .001; 95% CI [-.26; -.20]), with amotivation (k = 16; r = -.18; z = -6.6; P ≤ .001; 95% CI [-.23; -.13]) and with reduced expressivity (k = 15; r = -.18; z = -3.30; P ≤.001; 95% CI[-.29; -.07]). These associations were not moderated by sociodemographic characteristics, positive symptoms, depression, antipsychotic medication or type of negative symptom scale. Although these findings provide sound evidence for the association between episodic memory deficits and amotivation, the rather small magnitude and the unspecific pattern of this relationship also indicate that episodic memory deficits are unlikely to be the only factor relevant to amotivation. This implicates that future research should investigate episodic memory in conjunction with other factors that could account for the association of episodic memory deficits and amotivation in psychosis.

Reduced learning bias towards the reward context in medication-naive first-episode schizophrenia patients

BACKGROUND

Reinforcement learning has been proposed to contribute to the development of amotivation in individuals with schizophrenia (SZ). Accumulating evidence suggests dysfunctional learning in individuals with SZ in Go/NoGo learning and expected value representation. However, previous findings might have been confounded by the effects of antipsychotic exposure. Moreover, reinforcement learning also rely on the learning context. Few studies have examined the learning performance in reward and loss-avoidance context separately in medication-naïve individuals with first-episode SZ. This study aimed to explore the behaviour profile of reinforcement learning performance in medication-naïve individuals with first-episode SZ, including the contextual performance, the Go/NoGo learning and the expected value representation performance.

METHODS

Twenty-nine medication-naïve individuals with first-episode SZ and 40 healthy controls (HCs) who have no significant difference in age and gender, completed the Gain and Loss Avoidance Task, a reinforcement learning task involving stimulus pairs presented in both the reward and loss-avoidance context. We assessed the group difference in accuracy in the reward and loss-avoidance context, the Go/NoGo learning and the expected value representation. The correlations between learning performance and the negative symptom severity were examined.

RESULTS

Individuals with SZ showed significantly lower accuracy when learning under the reward than the loss-avoidance context as compared to HCs. The accuracies under the reward context (90%win- 10%win) in the Acquisition phase was significantly and negatively correlated with the Scale for the Assessment of Negative Symptoms (SANS) avolition scores in individuals with SZ. On the other hand, individuals with SZ showed spared ability of Go/NoGo learning and expected value representation.

CONCLUSIONS

Despite our small sample size and relatively modest findings, our results suggest possible reduced learning bias towards reward context among medication-naïve individuals with first-episode SZ. The reward learning performance was correlated with amotivation symptoms. This finding may facilitate our understanding of the underlying mechanism of negative symptoms. Reinforcement learning performance under the reward context may be important to better predict and prevent the development of schizophrenia patients' negative symptom, especially amotivation.

The prediction-error hypothesis of schizophrenia: new data point to circuit-specific changes in dopamine activity

Schizophrenia is a severe psychiatric disorder affecting 21 million people worldwide. People with schizophrenia suffer from symptoms including psychosis and delusions, apathy, anhedonia, and cognitive deficits. Strikingly, schizophrenia is characterised by a learning paradox involving difficulties learning from rewarding events, whilst simultaneously 'overlearning' about irrelevant or neutral information. While dysfunction in dopaminergic signalling has long been linked to the pathophysiology of schizophrenia, a cohesive framework that accounts for this learning paradox remains elusive. Recently, there has been an explosion of new research investigating how dopamine contributes to reinforcement learning, which illustrates that midbrain dopamine contributes in complex ways to reinforcement learning, not previously envisioned. This new data brings new possibilities for how dopamine signalling contributes to the symptomatology of schizophrenia. Building on recent work, we present a new neural framework for how we might envision specific dopamine circuits contributing to this learning paradox in schizophrenia in the context of models of reinforcement learning. Further, we discuss avenues of preclinical research with the use of cutting-edge neuroscience techniques where aspects of this model may be tested. Ultimately, it is hoped that this review will spur to action more research utilising specific reinforcement learning paradigms in preclinical models of schizophrenia, to reconcile seemingly disparate symptomatology and develop more efficient therapeutics.

Confirmation Bias in the Course of Instructed Reinforcement Learning in Schizophrenia-Spectrum Disorders

A large body of research attributes learning deficits in schizophrenia (SZ) to the systems involved in value representation (prefrontal cortex, PFC) and reinforcement learning (basal ganglia, BG) as well as to the compromised connectivity of these regions. In this study, we employed learning tasks hypothesized to probe the function and interaction of the PFC and BG in patients with SZ-spectrum disorders in comparison to healthy control (HC) subjects. In the Instructed Probabilistic Selection task (IPST), participants received false instruction about one of the stimuli used in the course of probabilistic learning which creates confirmation bias, whereby the instructed stimulus is overvalued in comparison to its real experienced value. The IPST was administered to 102 patients with SZ and 120 HC subjects. We have shown that SZ patients and HC subjects were equally influenced by false instruction in reinforcement learning (RL) probabilistic task (IPST) (p-value = 0.441); however, HC subjects had significantly higher learning rates associated with the process of overcoming cognitive bias in comparison to SZ patients (p-value = 0.018). The behavioral results of our study could be hypothesized to provide further evidence for impairments in the SZ-BG circuitry; however, this should be verified by neurofunctional imaging studies.

Lower Levels of Directed Exploration and Reflective Thinking Are Associated With Greater Anxiety and Depression

Anxiety and depression are often associated with strong beliefs that entering specific situations will lead to aversive outcomes - even when these situations are objectively safe and avoiding them reduces well-being. A possible mechanism underlying this maladaptive avoidance behavior is a failure to reflect on: (1) appropriate levels of uncertainty about the situation, and (2) how this uncertainty could be reduced by seeking further information (i.e., exploration). To test this hypothesis, we asked a community sample of 416 individuals to complete measures of reflective cognition, exploration, and symptoms of anxiety and depression. Consistent with our hypotheses, we found significant associations between each of these measures in expected directions (i.e., positive relationships between reflective cognition and strategic information-seeking behavior or "directed exploration", and negative relationships between these measures and anxiety/depression symptoms). Further analyses suggested that the relationship between directed exploration and depression/anxiety was due in part to an ambiguity aversion promoting exploration in conditions where information-seeking was not beneficial (as opposed to only being due to under-exploration when more information would aid future choices). In contrast, reflectiveness was associated with greater exploration in appropriate settings and separately accounted for differences in reaction times, decision noise, and choice accuracy in expected directions. These results shed light on the mechanisms underlying information-seeking behavior and how they may contribute to symptoms of emotional disorders. They also highlight the potential clinical relevance of individual differences in reflectiveness and exploration and should motivate future research on their possible contributions to vulnerability and/or maintenance of affective disorders.

The Role of Dopaminergic Genes in Probabilistic Reinforcement Learning in Schizophrenia Spectrum Disorders

Schizophrenia spectrum disorders (SZ) are characterized by impairments in probabilistic reinforcement learning (RL), which is associated with dopaminergic circuitry encompassing the prefrontal cortex and basal ganglia. However, there are no studies examining dopaminergic genes with respect to probabilistic RL in SZ. Thus, the aim of our study was to examine the impact of dopaminergic genes on performance assessed by the Probabilistic Selection Task (PST) in patients with SZ in comparison to healthy control (HC) subjects. In our study, we included 138 SZ patients and 188 HC participants. Genetic analysis was performed with respect to the following genetic polymorphisms: rs4680 in COMT, rs907094 in DARP-32, rs2734839, rs936461, rs1800497, and rs6277 in DRD2, rs747302 and rs1800955 in DRD4 and rs28363170 and rs2975226 in DAT1 genes. The probabilistic RL task was completed by 59 SZ patients and 95 HC subjects. SZ patients performed significantly worse in acquiring reinforcement contingencies during the task in comparison to HCs. We found no significant association between genetic polymorphisms and RL among SZ patients; however, among HC participants with respect to the DAT1 rs28363170 polymorphism, individuals with 10-allele repeat genotypes performed better in comparison to 9-allele repeat carriers. The present study indicates the relevance of the DAT1 rs28363170 polymorphism in RL in HC participants.

To learn or to gain: neural signatures of exploration in human decision-making

Individuals not only take actions to obtain immediate rewards but also to gain more information to guide future choices. An ideal exploration-exploitation balance is crucial for maximizing reward over the long run. However, the neural signatures of exploration in humans remain unclear. Using quantitative meta-analyses of functional magnetic resonance imaging experiments on exploratory behaviors, we sought to identify the concordant activity pertaining to exploration over a range of experiments. The results revealed that exploration activates concordant brain activity associated with risk (e.g., dorsal medial prefrontal cortex and anterior insula), cognitive control (e.g., dorsolateral prefrontal cortex and inferior frontal gyrus), and motor processing (e.g., premotor cortex). These stereotaxic maps of exploration may indicate that exploration is highly linked to risk processing, but is also specifically associated with regions involved in executive control processes. Although this explanation should be treated as exploratory, these findings support theories positing an important role for the prefrontal-insular-motor cortical network in exploration.

Mechanisms Underlying Motivational Dysfunction in Schizophrenia

Negative symptoms are a debilitating feature of schizophrenia which are often resistant to pharmacological intervention. The mechanisms underlying them remain poorly understood, and diagnostic methods rely on phenotyping through validated questionnaires. Deeper endo-phenotyping is likely to be necessary in order to improve current understanding. In the last decade, valuable behavioural insights have been gained through the use of effort-based decision making (EBDM) tasks. These have highlighted impairments in reward-related processing in schizophrenia, particularly associated with negative symptom severity. Neuroimaging investigations have related these changes to dysfunction within specific brain networks including the ventral striatum (VS) and frontal brain regions. Here, we review the behavioural and neural evidence associated with negative symptoms, shedding light on potential underlying mechanisms and future therapeutic possibilities. Findings in the literature suggest that schizophrenia is characterised by impaired reward based learning and action selection, despite preserved hedonic responses. Associations between amotivation and reward-processing deficits have not always been clear, and may be mediated by factors including cognitive dysfunction or dysfunctional or self-defeatist beliefs. Successful endo-phenotyping of negative symptoms as a function of objective behavioural and neural measurements is crucial in advancing our understanding of this complex syndrome. Additionally, transdiagnostic research–leveraging findings from other brain disorders, including neurological ones–can shed valuable light on the possible common origins of motivation disorders across diseases and has important implications for future treatment development.

An informatics approach to examine decision-making impairments in the daily life of individuals with depression

Mental health informatics studies methods that collect, model, and interpret a wide variety of data to generate useful information with theoretical or clinical relevance to improve mental health and mental health care. This article presents a mental health informatics approach that is based on the decision-making theory of depression, whereby daily life data from a natural sequential decision-making task are collected and modeled using a reinforcement learning method. The model parameters are then estimated to uncover specific aspects of decision-making impairment in individuals with depression. Empirical results from a pilot study conducted to examine decision-making impairments in the daily lives of university students with depression are presented to illustrate this approach. Future research can apply and expand on this approach to investigate a variety of daily life situations and psychiatric conditions and to facilitate new informatics applications. Using this approach in mental health research may generate useful information with both theoretical and clinical relevance and high ecological validity.

Psychiatric symptoms influence reward-seeking and loss-avoidance decision-making through common and distinct computational processes

AIM

Psychiatric symptoms are often accompanied by impairments in decision-making to attain rewards and avoid losses. However, due to the complex nature of mental disorders (e.g., high comorbidity), symptoms that are specifically associated with deficits in decision-making remain unidentified. Furthermore, the influence of psychiatric symptoms on computations underpinning reward-seeking and loss-avoidance decision-making remains elusive. Here, we aim to address these issues by leveraging a large-scale online experiment and computational modeling.

METHODS

In the online experiment, we recruited 1900 non-diagnostic participants from the general population. They performed either a reward-seeking or loss-avoidance decision-making task, and subsequently completed questionnaires about psychiatric symptoms.

RESULTS

We found that one trans-diagnostic dimension of psychiatric symptoms related to compulsive behavior and intrusive thought (CIT) was negatively correlated with overall decision-making performance in both the reward-seeking and loss-avoidance tasks. A deeper analysis further revealed that, in both tasks, the CIT psychiatric dimension was associated with lower preference for the options that recently led to better outcomes (i.e. reward or no-loss). On the other hand, in the reward-seeking task only, the CIT dimension was associated with lower preference for recently unchosen options.

CONCLUSION

These findings suggest that psychiatric symptoms influence the two types of decision-making, reward-seeking and loss-avoidance, through both common and distinct computational processes.

Aberrant salience and reward processing: a comparison of measures in schizophrenia and anxiety

BACKGROUND

Aberrant salience may contribute to the development of schizophrenia symptoms via alterations in reward processing and motivation. However, tests of this hypothesis have yielded inconsistent results. These inconsistencies may reflect problems with the validity and specificity of measures of aberrant salience in schizophrenia. Therefore, we investigated relationships among measures of aberrant salience, reward, and motivation in schizophrenia and anxiety.

METHOD

Individuals with schizophrenia (n = 30), anxiety (n = 33) or unaffected by mental disorder (n = 30) completed measures of aberrant salience [Aberrant Salience Inventory (ASI), Salience Attribution Test (SAT)], motivation (Effort Expenditure for Reward Task), and reinforcer sensitivity (Stimulus Chase Task).

RESULTS

Schizophrenia participants scored higher than anxiety (d = 0.71) and unaffected (d = 1.54) groups on the ASI and exhibited greater aberrant salience (d = 0.60) and lower adaptive salience (d = 0.98) than anxious participants on the SAT. There was no evidence of a correlation between measures of aberrant salience. Schizophrenia was associated with related deficits in motivated behaviour and maladaptive reward processing. However, these differences in reward processing did not correlate with aberrant salience measures.

CONCLUSIONS

The results suggest that key measures of aberrant salience have limited specificity and validity. These problems may account for inconsistent findings reported in the literature.

Reinforcement learning abnormalities in the attenuated psychosis syndrome and first episode psychosis

Prior studies indicate that chronic schizophrenia (SZ) is associated with a specific profile of reinforcement learning abnormalities. These impairments are characterized by: 1) reductions in learning rate, and 2) impaired Go learning and intact NoGo learning. Furthermore, each of these deficits are associated with greater severity of negative symptoms, consistent with theoretical perspectives positing that avolition and anhedonia are associated with impaired value representation. However, it is unclear whether these deficits extend to earlier phases of psychotic illness and when individuals are unmedicated. Two studies were conducted to examine reinforcement learning deficits in earlier phases of psychosis and in high risk patients. In study 1, participants included 35 participants with first episode psychosis (FEP) with limited antipsychotic medication exposure and 25 healthy controls (HC). Study 2 included 17 antipsychotic naïve individuals who were at clinical high-risk for psychosis (CHR) (i.e., attenuated psychosis syndrome) and 18 matched healthy controls (HC). In both studies, participants completed the Temporal Utility Integration Task, a measure of probabilistic reinforcement learning that contained Go and NoGo learning blocks. FEP displayed impaired Go and NoGo learning. In contrast, CHR did not display impairments in Go or NoGo learning. Impaired Go learning was not significantly associated with clinical outcomes in the CHR or FEP samples. Findings provide new evidence for areas of spared and impaired reinforcement learning in early phases of psychosis.

Proof-of-Mechanism Study of the Phosphodiesterase 10 Inhibitor RG7203 in Patients With Schizophrenia and Negative Symptoms

Background

Reduced activation of dopamine D₁ receptor signaling may be implicated in reward functioning as a potential driver of negative symptoms in schizophrenia. Phosphodiesterase 10A (PDE10A), an enzyme that is highly expressed in the striatum, modulates both dopamine D₂- and D₁-dependent signaling.

Methods

We assessed whether augmentation of D₁ signaling by the PDE10 inhibitor RG7203 enhances imaging and behavioral markers of reward functions in patients with schizophrenia and negative symptoms. In a 3-period, double-blind, crossover study, we investigated the effects of RG7203 (5 mg and 15 mg doses) and placebo as adjunctive treatment to stable background antipsychotic treatment in patients with chronic schizophrenia with moderate levels of negative symptoms. Effects on reward functioning and reward-based effortful behavior were evaluated using the monetary incentive delay task during functional magnetic resonance imaging and the effort-cost-benefit and working memory reinforcement learning tasks.

Results

Patients (N = 33; 30 male, mean age ± SD 36.6 ± 7.0 years; Positive and Negative Syndrome Scale negative symptom factor score 23.0 ± 3.5 at screening) were assessed at three study centers in the United States; 24 patients completed the study. RG7203 at 5 mg significantly increased reward expectation–related activity in the monetary incentive delay task, but in the context of significantly decreased overall activity across all task conditions.

Conclusions

In contrast to our expectations, RG7203 significantly worsened reward-based effortful behavior and indices of reward learning. The results do not support the utility of RG7203 as adjunctive treatment for negative symptoms in patients with schizophrenia.

Reliability and Replicability of Implicit and Explicit Reinforcement Learning Paradigms in People With Psychotic Disorders

BACKGROUND

Motivational deficits in people with psychosis may be a result of impairments in reinforcement learning (RL). Therefore, behavioral paradigms that can accurately measure these impairments and their change over time are essential.

METHODS

We examined the reliability and replicability of 2 RL paradigms (1 implicit and 1 explicit, each with positive and negative reinforcement components) given at 2 time points to healthy controls (n = 75), and people with bipolar disorder (n = 62), schizoaffective disorder (n = 60), and schizophrenia (n = 68).

RESULTS

Internal consistency was acceptable (mean α = 0.78 ± 0.15), but test-retest reliability was fair to low (mean intraclass correlation = 0.33 ± 0.25) for both implicit and explicit RL. There were no clear effects of practice for these tasks. Largely, performance on these tasks shows intact implicit and impaired explicit RL in psychosis. Symptom presentation did not relate to performance in any robust way.

CONCLUSIONS

Our findings replicate previous literature showing spared implicit RL and impaired explicit reinforcement in psychosis. This suggests typical basal ganglia dopamine release, but atypical recruitment of the orbitofrontal and dorsolateral prefrontal cortices. However, we found that these tasks have only fair to low test-retest reliability and thus may not be useful for assessing change over time in clinical trials.

Psychiatric Neural Networks and Precision Therapeutics by Machine Learning

Learning and environmental adaptation increase the likelihood of survival and improve the quality of life. However, it is often difficult to judge optimal behaviors in real life due to highly complex social dynamics and environment. Consequentially, many different brain regions and neuronal circuits are involved in decision-making. Many neurobiological studies on decision-making show that behaviors are chosen through coordination among multiple neural network systems, each implementing a distinct set of computational algorithms. Although these processes are commonly abnormal in neurological and psychiatric disorders, the underlying causes remain incompletely elucidated. Machine learning approaches with multidimensional data sets have the potential to not only pathologically redefine mental illnesses but also better improve therapeutic outcomes than DSM/ICD diagnoses. Furthermore, measurable endophenotypes could allow for early disease detection, prognosis, and optimal treatment regime for individuals. In this review, decision-making in real life and psychiatric disorders and the applications of machine learning in brain imaging studies on psychiatric disorders are summarized, and considerations for the future clinical translation are outlined. This review also aims to introduce clinicians, scientists, and engineers to the opportunities and challenges in bringing artificial intelligence into psychiatric practice.