Probabilistic Reinforcement Learning in Patients With Schizophrenia: Relationships to Anhedonia and Avolition

Apathy and Motivation: Biological Basis and Drug Treatment

Apathy is a disabling syndrome associated with poor functional outcomes that is common across a broad range of neurological and psychiatric conditions. Currently, there are no established therapies specifically for the condition, and safe and effective treatments are urgently needed. Advances in the understanding of motivation and goal-directed behavior in humans and animals have shed light on the cognitive and neurobiological mechanisms contributing to apathy, providing an important foundation for the development of new treatments. Here, we review the cognitive components, neural circuitry, and pharmacology of apathy and motivation, highlighting converging evidence of shared transdiagnostic mechanisms. Though no pharmacological treatments have yet been licensed, we summarize trials of existing and novel compounds to date, identifying several promising candidates for clinical use and avenues of future drug development.

Global reward processing deficits predict negative symptoms transdiagnostically and transphasically in a severe mental illness-spectrum sample

Reward processing impairments are a key factor associated with negative symptoms in those with severe mental illnesses. However, past findings are inconsistent regarding which reward processing components are impaired and most strongly linked to negative symptoms. The current study examined the hypothesis that these mixed findings may be the result of multiple reward processing pathways (i.e., equifinality) to negative symptoms that cut across diagnostic boundaries and phases of illness. Participants included healthy controls (n = 100) who served as a reference sample and a severe mental illness-spectrum sample (n = 92) that included psychotic-like experiences, clinical high-risk for psychosis, bipolar disorder, and schizophrenia participants. All participants completed tasks measuring four RDoC Positive Valence System constructs: value representation, reinforcement learning, effort-cost computation, and hedonic reactivity. A k-means cluster analysis of the severe mental illness-spectrum samples identified three clusters with differential reward processing profiles that were characterized by: (1) global reward processing deficits (22.8%), (2) selective impairments in hedonic reactivity alone (40.2%), and (3) preserved reward processing (37%). Elevated negative symptoms were only observed in the global reward processing cluster. All clusters contained participants from each clinical group, and the distribution of these groups did not significantly differ among the clusters. Findings identified one pathway contributing to negative symptoms that was transdiagnostic and transphasic. Future work further characterizing divergent pathways to negative symptoms may help to improve symptom trajectories and personalized treatments.

Revisiting reward impairments in schizophrenia spectrum disorders: a systematic review and meta-analysis for neuroimaging findings

BACKGROUND

Abnormal reward functioning is central to anhedonia and amotivation symptoms of schizophrenia (SCZ). Reward processing encompasses a series of psychological components. This systematic review and meta-analysis examined the brain dysfunction related to reward processing of individuals with SCZ spectrum disorders and risks, covering multiple reward components.

METHODS

After a systematic literature search, 37 neuroimaging studies were identified and divided into four groups based on their target psychology components (i.e. reward anticipation, reward consumption, reward learning, effort computation). Whole-brain Seed-based d Mapping (SDM) meta-analyses were conducted for all included studies and each component.

RESULTS

The meta-analysis for all reward-related studies revealed reduced functional activation across the SCZ spectrum in the striatum, orbital frontal cortex, cingulate cortex, and cerebellar areas. Meanwhile, distinct abnormal patterns were found for reward anticipation (decreased activation of the cingulate cortex and striatum), reward consumption (decreased activation of cerebellum IV/V areas, insula and inferior frontal gyri), and reward learning processing (decreased activation of the striatum, thalamus, cerebellar Crus I, cingulate cortex, orbitofrontal cortex, and parietal and occipital areas). Lastly, our qualitative review suggested that decreased activation of the ventral striatum and anterior cingulate cortex was also involved in effort computation.

CONCLUSIONS

These results provide deep insights on the component-based neuro-psychopathological mechanisms for anhedonia and amotivation symptoms of the SCZ spectrum.

Belief updating in psychosis, depression and anxiety disorders: A systematic review across computational modelling approaches

Alterations in belief updating are proposed to underpin symptoms of psychiatric illness, including psychosis, depression, and anxiety. Key parameters underlying belief updating can be captured using computational modelling techniques, aiding the identification of unique and shared deficits, and improving diagnosis and treatment. We systematically reviewed research that applied computational modelling to probabilistic tasks measuring belief updating in stable and volatile (changing) environments, across clinical and subclinical psychosis (n = 17), anxiety (n = 9), depression (n = 9) and transdiagnostic samples (n = 9). Depression disorders related to abnormal belief updating in response to the valence of rewards, evidenced in both stable and volatile environments. Whereas psychosis and anxiety disorders were associated with difficulties adapting to changing contingencies specifically, indicating an inflexibility and/or insensitivity to environmental volatility. Higher-order learning models revealed additional difficulties in the estimation of overall environmental volatility across psychosis disorders, showing increased updating to irrelevant information. These findings stress the importance of investigating belief updating in transdiagnostic samples, using homogeneous experimental and computational modelling approaches.

Neural correlates of overvaluation and the effort to save possessions in a novel decision task: An exploratory fMRI study

Introduction

People exhibit a strong attachment to possessions, observed in behavioral economics through loss aversion using new items in the Endowment or IKEA effects and in clinical psychology through pathological trouble discarding domestic items in Hoarding Disorder. These fields rarely intersect, but both document a reticence to relinquish a possessed item, even at a cost, which is associated with feelings of loss but can include enhanced positive states as well.

Methods

To demonstrate the shared properties of these loss-related ownership effects, we developed the Pretzel Decorating Task (PDT), which concurrently measures overvaluation of one's own over others' items and feelings of loss associated with losing a possession, alongside enhanced positive appraisals of one's items and an effort to save them. The PDT was piloted with 31 participants who decorated pretzels and responded to their own or others' items during functional neuroimaging (fMRI). Participants observed one item per trial (self or other) and could work to save it (high or low probability loss) before learning the fate of the item (trashed or saved). Finally, participants rated items and completed hoarding tendency scales.

Results

The hypotheses were supported, as even non-clinical participants overvalued, viewed as nicer, feared losing, and worked harder to save their items over others'-a response that correlated with hoarding tendencies and motor-motivational brain activation. Our region of interest in the nucleus accumbens (NAcc) was engaged when viewing one's own items to the extent that people worked harder to save them and was more active when their items were saved when they felt emotionally attached to possessions in real life. When their items were trashed, NAcc activity negatively correlated with trouble discarding and emotional attachments to possessions. Right anterior insula was more active when working to save one's own over others' items. Extensive motor-motivational areas were engaged when working to save one's own over others' items, including cerebellum, primary motor and somatosensory regions, and retrosplenial/parahippocampal regions-even after controlling for tapping.

Discussion

Our attachments to items are emotional, continuous across typical and pathological populations, and drive us to save possessions that we value.

Beyond Drift Diffusion Models: Fitting a Broad Class of Decision and Reinforcement Learning Models with HDDM

Computational modeling has become a central aspect of research in the cognitive neurosciences. As the field matures, it is increasingly important to move beyond standard models to quantitatively assess models with richer dynamics that may better reflect underlying cognitive and neural processes. For example, sequential sampling models (SSMs) are a general class of models of decision-making intended to capture processes jointly giving rise to RT distributions and choice data in n-alternative choice paradigms. A number of model variations are of theoretical interest, but empirical data analysis has historically been tied to a small subset for which likelihood functions are analytically tractable. Advances in methods designed for likelihood-free inference have recently made it computationally feasible to consider a much larger spectrum of SSMs. In addition, recent work has motivated the combination of SSMs with reinforcement learning models, which had historically been considered in separate literatures. Here, we provide a significant addition to the widely used HDDM Python toolbox and include a tutorial for how users can easily fit and assess a (user-extensible) wide variety of SSMs and how they can be combined with reinforcement learning models. The extension comes batteries included, including model visualization tools, posterior predictive checks, and ability to link trial-wise neural signals with model parameters via hierarchical Bayesian regression.

Both unmedicated and medicated individuals with schizophrenia show impairments across a wide array of cognitive and reinforcement learning tasks

BACKGROUND

Schizophrenia is a disorder characterized by pervasive deficits in cognitive functioning. However, few well-powered studies have examined the degree to which cognitive performance is impaired even among individuals with schizophrenia not currently on antipsychotic medications using a wide range of cognitive and reinforcement learning measures derived from cognitive neuroscience. Such research is particularly needed in the domain of reinforcement learning, given the central role of dopamine in reinforcement learning, and the potential impact of antipsychotic medications on dopamine function.

METHODS

The present study sought to fill this gap by examining healthy controls (N = 75), unmedicated (N = 48) and medicated (N = 148) individuals with schizophrenia. Participants were recruited across five sites as part of the CNTRaCS Consortium to complete tasks assessing processing speed, cognitive control, working memory, verbal learning, relational encoding and retrieval, visual integration and reinforcement learning.

RESULTS

Individuals with schizophrenia who were not taking antipsychotic medications, as well as those taking antipsychotic medications, showed pervasive deficits across cognitive domains including reinforcement learning, processing speed, cognitive control, working memory, verbal learning and relational encoding and retrieval. Further, we found that chlorpromazine equivalency rates were significantly related to processing speed and working memory, while there were no significant relationships between anticholinergic load and performance on other tasks.

CONCLUSIONS

These findings add to a body of literature suggesting that cognitive deficits are an enduring aspect of schizophrenia, present in those off antipsychotic medications as well as those taking antipsychotic medications.

Psychological Dimensions Relevant to Motivation and Pleasure in Schizophrenia

Motivation and pleasure deficits are common in schizophrenia, strongly linked with poorer functioning, and may reflect underlying alterations in brain functions governing reward processing and goal pursuit. While there is extensive research examining cognitive and reward mechanisms related to these deficits in schizophrenia, less attention has been paid to psychological characteristics that contribute to resilience against, or risk for, motivation and pleasure impairment. For example, psychological tendencies involving positive future expectancies (e.g., optimism) and effective affect management (e.g., reappraisal, mindfulness) are associated with aspects of reward anticipation and evaluation that optimally guide goal-directed behavior. Conversely, maladaptive thinking patterns (e.g., defeatist performance beliefs, asocial beliefs) and tendencies that amplify negative cognitions (e.g., rumination), may divert cognitive resources away from goal pursuit or reduce willingness to exert effort. Additionally, aspects of sociality, including the propensity to experience social connection as positive reinforcement may be particularly relevant for pursuing social goals. In the current review, we discuss the roles of several psychological characteristics with respect to motivation and pleasure in schizophrenia. We argue that individual variation in these psychological dimensions is relevant to the study of motivation and reward processing in schizophrenia, including interactions between these psychological dimensions and more well-characterized cognitive and reward processing contributors to motivation. We close by emphasizing the value of considering a broad set of modulating factors when studying motivation and pleasure functions in schizophrenia.

The prediction-error hypothesis of schizophrenia: new data point to circuit-specific changes in dopamine activity

Schizophrenia is a severe psychiatric disorder affecting 21 million people worldwide. People with schizophrenia suffer from symptoms including psychosis and delusions, apathy, anhedonia, and cognitive deficits. Strikingly, schizophrenia is characterised by a learning paradox involving difficulties learning from rewarding events, whilst simultaneously 'overlearning' about irrelevant or neutral information. While dysfunction in dopaminergic signalling has long been linked to the pathophysiology of schizophrenia, a cohesive framework that accounts for this learning paradox remains elusive. Recently, there has been an explosion of new research investigating how dopamine contributes to reinforcement learning, which illustrates that midbrain dopamine contributes in complex ways to reinforcement learning, not previously envisioned. This new data brings new possibilities for how dopamine signalling contributes to the symptomatology of schizophrenia. Building on recent work, we present a new neural framework for how we might envision specific dopamine circuits contributing to this learning paradox in schizophrenia in the context of models of reinforcement learning. Further, we discuss avenues of preclinical research with the use of cutting-edge neuroscience techniques where aspects of this model may be tested. Ultimately, it is hoped that this review will spur to action more research utilising specific reinforcement learning paradigms in preclinical models of schizophrenia, to reconcile seemingly disparate symptomatology and develop more efficient therapeutics.

Probabilistic Reinforcement Learning and Anhedonia

Despite the prominence of anhedonic symptoms associated with diverse neuropsychiatric conditions, there are currently no approved therapeutics designed to attenuate the loss of responsivity to previously rewarding stimuli. However, the search for improved treatment options for anhedonia has been reinvigorated by a recent reconceptualization of the very construct of anhedonia, including within the Research Domain Criteria (RDoC) initiative. This chapter will focus on the RDoC Positive Valence Systems construct of reward learning generally and sub-construct of probabilistic reinforcement learning specifically. The general framework emphasizes objective measurement of a subject's responsivity to reward via reinforcement learning under asymmetrical probabilistic contingencies as a means to quantify reward learning. Indeed, blunted reward responsiveness and reward learning are central features of anhedonia and have been repeatedly described in major depression. Moreover, these probabilistic reinforcement techniques can also reveal neurobiological mechanisms to aid development of innovative treatment approaches. In this chapter, we describe how investigating reward learning can improve our understanding of anhedonia via the four RDoC-recommended tasks that have been used to probe sensitivity to probabilistic reinforcement contingencies and how such task performance is disrupted in various neuropsychiatric conditions. We also illustrate how reverse translational approaches of probabilistic reinforcement assays in laboratory animals can inform understanding of pharmacological and physiological mechanisms. Next, we briefly summarize the neurobiology of probabilistic reinforcement learning, with a focus on the prefrontal cortex, anterior cingulate cortex, striatum, and amygdala. Finally, we discuss treatment implications and future directions in this burgeoning area.

Emotion prediction errors guide socially adaptive behaviour

People make decisions based on deviations from expected outcomes, known as prediction errors. Past work has focused on reward prediction errors, largely ignoring violations of expected emotional experiences—emotion prediction errors. We leverage a method to measure real-time fluctuations in emotion as people decide to punish or forgive others. Across four studies (N=1,016), we reveal that emotion and reward prediction errors have distinguishable contributions to choice, such that emotion prediction errors exert the strongest impact during decision-making. We additionally find that a choice to punish or forgive can be decoded in less than a second from an evolving emotional response, suggesting emotions swiftly influence choice. Finally, individuals reporting significant levels of depression exhibit selective impairments in using emotion—but not reward—prediction errors. Evidence for emotion prediction errors potently guiding social behaviors challenge standard decision-making models that have focused solely on reward.

Computational Psychiatry Needs Time and Context

Why has computational psychiatry yet to influence routine clinical practice? One reason may be that it has neglected context and temporal dynamics in the models of certain mental health problems. We develop three heuristics for estimating whether time and context are important to a mental health problem: Is it characterized by a core neurobiological mechanism? Does it follow a straightforward natural trajectory? And is intentional mental content peripheral to the problem? For many problems the answers are no, suggesting that modeling time and context is critical. We review computational psychiatry advances toward this end, including modeling state variation, using domain-specific stimuli, and interpreting differences in context. We discuss complementary network and complex systems approaches. Novel methods and unification with adjacent fields may inspire a new generation of computational psychiatry. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Increased face detection responses on the mooney faces test in people at clinical high risk for psychosis

Identifying state-sensitive measures of perceptual and cognitive processes implicated in psychosis may allow for objective, earlier, and better monitoring of changes in mental status that are predictive of an impending psychotic episode, relative to traditional self-report-based clinical measures. To determine whether a measure of visual perception that has demonstrated sensitivity to the clinical state of schizophrenia in multiple prior studies is sensitive to features of the at-risk mental state, we examined differences between young people identified as being at clinical high risk for psychosis (CHR; n = 37) and non-psychiatric matched controls (n = 29) on the Mooney Faces Test (MFT). On each trial of the MFT, participants report whether they perceive a face in a degraded face image. The CHR group reported perceiving a greater number of faces in both upright and inverted MFT stimuli. Consistent with prior work, males reported more faces on the MFT than females in both conditions. However, the finding of greater reported face perception among CHR subjects was robustly observed in the female CHR group relative to the female control group. Among male CHR participants, greater reported face perception was related to increased perceptual abnormalities. These preliminary results are consistent with a small but growing literature suggesting that heightened perceptual sensitivity may characterize individuals at increased clinical risk for psychosis. Further studies are needed to determine the contributions of specific perceptual, cognitive, and motivational mechanisms to the findings.

Abnormal prediction error processing in schizophrenia and depression

To make adaptive decisions under uncertainty, individuals need to actively monitor the discrepancy between expected outcomes and actual outcomes, known as prediction errors. Reward‐based learning deficits have been shown in both depression and schizophrenia patients. For this study, we compiled studies that investigated prediction error processing in depression and schizophrenia patients and performed a series of meta‐analyses. In both groups, positive t‐maps of prediction error tend to yield striatum activity across studies. The analysis of negative t‐maps of prediction error revealed two large clusters within the right superior and inferior frontal lobes in schizophrenia and the medial prefrontal cortex and bilateral insula in depression. The concordant posterior cingulate activity was observed in both patient groups, more prominent in the depression group and absent in the healthy control group. These findings suggest a possible role in dopamine‐rich areas associated with the encoding of prediction errors in depression and schizophrenia.

Unlocking the reinforcement-learning circuits of the orbitofrontal cortex

Neuroimaging studies have consistently identified the orbitofrontal cortex (OFC) as being affected in individuals with neuropsychiatric disorders. OFC dysfunction has been proposed to be a key mechanism by which decision-making impairments emerge in diverse clinical populations, and recent studies employing computational approaches have revealed that distinct reinforcement-learning mechanisms of decision-making differ among diagnoses. In this perspective, we propose that these computational differences may be linked to select OFC circuits and present our recent work that has used a neurocomputational approach to understand the biobehavioral mechanisms of addiction pathology in rodent models. We describe how combining translationally analogous behavioral paradigms with reinforcement-learning algorithms and sophisticated neuroscience techniques in animals can provide critical insights into OFC pathology in biobehavioral disorders. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Retention of Value Representations Across Time in People With Schizophrenia and Healthy Control Subjects

BACKGROUND

The current study aimed to further etiological understanding of the psychological mechanisms underlying negative symptoms in people with schizophrenia. Specifically, we tested whether negative symptom severity is associated with reduced retention of reward-related information over time and thus a degraded ability to utilize such information to guide future action selection.

METHODS

Forty-four patients with a diagnosis of schizophrenia or schizoaffective disorder and 28 healthy control volunteers performed a probabilistic reinforcement-learning task involving stimulus pairs in which choices resulted in reward or in loss avoidance. Following training, participants indicated their valuation of learned stimuli in a test/transfer phase. The test/transfer phase was administered immediately following training and 1 week later. Percent retention was defined as accuracy at week-long delay divided by accuracy at immediate delay.

RESULTS

Healthy control subjects and people with schizophrenia showed similarly robust retention of reinforcement learning over a 1-week delay interval. However, in the schizophrenia group, negative symptom severity was associated with reduced retention of information regarding the value of actions across a week-long interval. This pattern was particularly notable for stimuli associated with reward compared with loss avoidance.

CONCLUSIONS

Our results show that although individuals with schizophrenia may initially learn about rewarding aspects of their environment, such learning decays at a more rapid rate in patients with severe negative symptoms. Thus, previously learned reward-related information may be more difficult to access to guide future decision making and to motivate action selection.

Disruption of Nrxn1α within excitatory forebrain circuits drives value-based dysfunction

Goal-directed behaviors are essential for normal function and significantly impaired in neuropsychiatric disorders. Despite extensive associations between genetic mutations and these disorders, the molecular contributions to goal-directed dysfunction remain unclear. We examined mice with constitutive and brain region-specific mutations in Neurexin1α, a neuropsychiatric disease-associated synaptic molecule, in value-based choice paradigms. We found Neurexin1α knockouts exhibited reduced selection of beneficial outcomes and impaired avoidance of costlier options. Reinforcement modeling suggested that this was driven by deficits in updating and representation of value. Disruption of Neurexin1α within telencephalic excitatory projection neurons, but not thalamic neurons, recapitulated choice abnormalities of global Neurexin1α knockouts. Furthermore, this selective forebrain excitatory knockout of Neurexin1α perturbed value-modulated neural signals within striatum, a central node in feedback-based reinforcement learning. By relating deficits in value-based decision-making to region-specific Nrxn1α disruption and changes in value-modulated neural activity, we reveal potential neural substrates for the pathophysiology of neuropsychiatric disease-associated cognitive dysfunction.

Orbitofrontal-Striatal Structural Alterations Linked to Negative Symptoms at Different Stages of the Schizophrenia Spectrum

Negative symptoms such as anhedonia and apathy are among the most debilitating manifestations of schizophrenia (SZ). Imaging studies have linked these symptoms to morphometric abnormalities in 2 brain regions implicated in reward and motivation: the orbitofrontal cortex (OFC) and striatum. Higher negative symptoms are generally associated with reduced OFC thickness, while higher apathy specifically maps to reduced striatal volume. However, it remains unclear whether these tissue losses are a consequence of chronic illness and its treatment or an underlying phenotypic trait. Here, we use multicentre magnetic resonance imaging data to investigate orbitofrontal-striatal abnormalities across the SZ spectrum from healthy populations with high schizotypy to unmedicated and medicated first-episode psychosis (FEP), and patients with chronic SZ. Putamen, caudate, accumbens volume, and OFC thickness were estimated from T1-weighted images acquired in all 3 diagnostic groups and controls from 4 sites (n = 337). Results were first established in 1 discovery dataset and replicated in 3 independent samples. There was a negative correlation between apathy and putamen/accumbens volume only in healthy individuals with schizotypy; however, medicated patients exhibited larger putamen volume, which appears to be a consequence of antipsychotic medications. The negative association between reduced OFC thickness and total negative symptoms also appeared to vary along the SZ spectrum, being significant only in FEP patients. In schizotypy, there was increased OFC thickness relative to controls. Our findings suggest that negative symptoms are associated with a temporal continuum of orbitofrontal-striatal abnormalities that may predate the occurrence of SZ. Thicker OFC in schizotypy may represent either compensatory or pathological mechanisms prior to the disease onset.

Reward processing electrophysiology in schizophrenia: Effects of age and illness phase

BACKGROUND

Reward processing abnormalities may underlie characteristic pleasure and motivational impairments in schizophrenia. Some neural measures of reward processing show age-related modulation, highlighting the importance of considering age effects on reward sensitivity. We compared event-related potentials (ERPs) reflecting reward anticipation (stimulus-preceding negativity, SPN) and evaluation (reward positivity, RewP; late positive potential, LPP) across individuals with schizophrenia (SZ) and healthy controls (HC), with an emphasis on examining the effects of chronological age, brain age (i.e., predicted age based on neurobiological measures), and illness phase.

METHODS

Subjects underwent EEG while completing a slot-machine task for which rewards were not dependent on performance accuracy, speed, or response preparation. Slot-machine task EEG responses were compared between 54 SZ and 54 HC individuals, ages 19 to 65. Reward-related ERPs were analyzed with respect to chronological age, categorically-defined illness phase (early; ESZ versus chronic schizophrenia; CSZ), and were used to model brain age relative to chronological age.

RESULTS

Illness phase-focused analyses indicated there were no group differences in average SPN or RewP amplitudes. However, a group × reward outcome interaction revealed that ESZ differed from HC in later outcome processing, reflected by greater LPP responses following loss versus reward (a reversal of the HC pattern). While brain age estimates did not differ among groups, depressive symptoms in SZ were associated with older brain age estimates while controlling for negative symptoms.

CONCLUSIONS

ESZ and CSZ did not differ from HC in reward anticipation or early outcome processing during a cognitively undemanding reward task, highlighting areas of preserved functioning. However, ESZ showed altered later reward outcome evaluation, pointing to selective reward deficits during the early illness phase of schizophrenia. Further, an association between ERP-derived brain age and depressive symptoms in SZ extends prior findings linking depression with reward-related ERP blunting. Taken together, both illness phase and age may impact reward processing among SZ, and brain aging may offer a promising, novel marker of reward dysfunction that warrants further study.

From apathy to addiction: Insights from neurology and psychiatry

The tendency to engage in addictive behaviors has long been tied to the actions of the dopamine system. Early theories were based on the fact that all addictive drugs and behaviors (such as gambling) increase dopamine levels in the striatum, and the evidence that dopamine signaled reward or reward prediction error. However, with a changing emphasis of addiction away from purely pharmacological models that emphasize tolerance and withdrawal, towards one of behavioral dyscontrol, is there still a place for abnormal dopamine signaling in addiction? Here we recast the dopamine theory of addiction based on the idea that tonic dopamine may index a continuous phenotype that goes from apathy to impulsivity and compulsivity. Higher tonic dopamine signaling would make individuals vulnerable to drug reinforcement and cue-induced craving. We relate this to computational models of dopamine signaling, and review clinical and neuroimaging evidence from Parkinson's Disease, schizophrenia and bipolar disorder in support of this model.

Striatal Dopamine and Reward Prediction Error Signaling in Unmedicated Schizophrenia Patients

Increased striatal dopamine synthesis capacity has consistently been reported in patients with schizophrenia. However, the mechanism translating this into behavior and symptoms remains unclear. It has been proposed that heightened striatal dopamine may blunt dopaminergic reward prediction error signaling during reinforcement learning. In this study, we investigated striatal dopamine synthesis capacity, reward prediction errors, and their association in unmedicated schizophrenia patients (n = 19) and healthy controls (n = 23). They took part in FDOPA-PET and underwent functional magnetic resonance imaging (fMRI) scanning, where they performed a reversal-learning paradigm. The groups were compared regarding dopamine synthesis capacity (Kicer), fMRI neural prediction error signals, and the correlation of both. Patients did not differ from controls with respect to striatal Kicer. Taking into account, comorbid alcohol abuse revealed that patients without such abuse showed elevated Kicer in the associative striatum, while those with abuse did not differ from controls. Comparing all patients to controls, patients performed worse during reversal learning and displayed reduced prediction error signaling in the ventral striatum. In controls, Kicer in the limbic striatum correlated with higher reward prediction error signaling, while there was no significant association in patients. Kicer in the associative striatum correlated with higher positive symptoms and blunted reward prediction error signaling was associated with negative symptoms. Our results suggest a dissociation between striatal subregions and symptom domains, with elevated dopamine synthesis capacity in the associative striatum contributing to positive symptoms while blunted prediction error signaling in the ventral striatum related to negative symptoms.

Amphetamine disrupts haemodynamic correlates of prediction errors in nucleus accumbens and orbitofrontal cortex

In an uncertain world, the ability to predict and update the relationships between environmental cues and outcomes is a fundamental element of adaptive behaviour. This type of learning is typically thought to depend on prediction error, the difference between expected and experienced events and in the reward domain that has been closely linked to mesolimbic dopamine. There is also increasing behavioural and neuroimaging evidence that disruption to this process may be a cross-diagnostic feature of several neuropsychiatric and neurological disorders in which dopamine is dysregulated. However, the precise relationship between haemodynamic measures, dopamine and reward-guided learning remains unclear. To help address this issue, we used a translational technique, oxygen amperometry, to record haemodynamic signals in the nucleus accumbens (NAc) and orbitofrontal cortex (OFC), while freely moving rats performed a probabilistic Pavlovian learning task. Using a model-based analysis approach to account for individual variations in learning, we found that the oxygen signal in the NAc correlated with a reward prediction error, whereas in the OFC it correlated with an unsigned prediction error or salience signal. Furthermore, an acute dose of amphetamine, creating a hyperdopaminergic state, disrupted rats' ability to discriminate between cues associated with either a high or a low probability of reward and concomitantly corrupted prediction error signalling. These results demonstrate parallel but distinct prediction error signals in NAc and OFC during learning, both of which are affected by psychostimulant administration. Furthermore, they establish the viability of tracking and manipulating haemodynamic signatures of reward-guided learning observed in human fMRI studies by using a proxy signal for BOLD in a freely behaving rodent.

Grant Report on Social Reward Learning in Schizophrenia †

We report on the ongoing R21 project “Social Reward Learning in Schizophrenia”. Impairments in social cognition are a hallmark of schizophrenia. However, little work has been done on social reward learning deficits in schizophrenia. The overall goal of the project is to assess social reward learning in schizophrenia. A probabilistic reward learning (PRL) task is being used in the MRI scanner to evaluate reward learning to negative and positive social feedback. Monetary reward learning is used as a comparison to assess specificity. Behavioral outcomes and brain areas, included those involved in reward, are assessed in patients with schizophrenia or schizoaffective disorder and controls. It is also critical to determine whether decreased expected value (EV) of social stimuli and/or reward prediction error (RPE) learning underlie social reward learning deficits to inform potential treatment pathways. Our central hypothesis is that the pattern of social learning deficits is an extension of a more general reward learning impairment in schizophrenia and that social reward learning deficits critically contribute to deficits in social motivation and pleasure. We hypothesize that people with schizophrenia will show impaired behavioral social reward learning compared to controls, as well as decreased ventromedial prefrontal cortex (vmPFC) EV signaling at time of choice and decreased striatal RPE signaling at time of outcome, with potentially greater impairment to positive than negative feedback. The grant is in its second year. It is hoped that this innovative approach may lead to novel and more targeted treatment approaches for social cognitive impairments, using cognitive remediation and/or brain stimulation.

Ten simple rules for the computational modeling of behavioral data

Computational modeling of behavior has revolutionized psychology and neuroscience. By fitting models to experimental data we can probe the algorithms underlying behavior, find neural correlates of computational variables and better understand the effects of drugs, illness and interventions. But with great power comes great responsibility. Here, we offer ten simple rules to ensure that computational modeling is used with care and yields meaningful insights. In particular, we present a beginner-friendly, pragmatic and details-oriented introduction on how to relate models to data. What, exactly, can a model tell us about the mind? To answer this, we apply our rules to the simplest modeling techniques most accessible to beginning modelers and illustrate them with examples and code available online. However, most rules apply to more advanced techniques. Our hope is that by following our guidelines, researchers will avoid many pitfalls and unleash the power of computational modeling on their own data.

Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals

Individuals from across the psychosis spectrum display impairments in reinforcement learning. In some individuals, these deficits may result from aberrations in reward prediction error (RPE) signaling, conveyed by dopaminergic projections to the ventral striatum (VS). However, there is mounting evidence that VS RPE signals are relatively intact in medicated people with schizophrenia (PSZ). We hypothesized that, in PSZ, reinforcement learning deficits often are not related to RPE signaling per se but rather their impact on learning and behavior (i.e., learning rate modulation), due to dysfunction in anterior cingulate and dorsomedial prefrontal cortex (dmPFC). Twenty-six PSZ and 23 healthy volunteers completed a probabilistic reinforcement learning paradigm with occasional, sudden, shifts in contingencies. Using computational modeling, we found evidence of an impairment in trial-wise learning rate modulation (α) in PSZ before and after a reinforcement contingency shift, expressed most in PSZ with more severe motivational deficits. In a subsample of 22 PSZ and 22 healthy volunteers, we found little evidence for between-group differences in VS RPE and dmPFC learning rate signals, as measured with fMRI. However, a follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. These findings point to an impairment in learning rate modulation in PSZ, leading to a reduced ability to adjust task behavior in response to unexpected outcomes. At the level of the brain, learning rate modulation deficits may be associated with decreased involvement of the dmPFC within a greater RL network.

Predictive Modulation of Corticospinal Excitability and Implicit Encoding of Movement Probability in Schizophrenia

The ability to infer from uncertain information is impaired in schizophrenia and is associated with hallucinations and false beliefs. The accumulation of information is a key process for generating a predictive internal model, which statistically estimates an outcome from a specific situation. This study examines if updating the predictive model by the accumulation of information in absence of feedback is impaired in schizophrenia. We explored the implicit adaptation to the probability of being instructed to perform a movement (33%-Go, 50%-Go, or 66%-Go) in a Go/NoGo task in terms of reaction times (RTs), electromyographic activity, and corticospinal excitability (CSE) of primary motor cortex (M1). CSE was assessed at two time points to evaluate prediction of the upcoming instruction based on previously accumulated information: at rest (preceding the warning signal) and at the Go/NoGo signal onset. Three groups were compared: patients with schizophrenia (n = 20), unaffected siblings (n = 16), and healthy controls (n = 20). Controls and siblings showed earlier movement onset and increased CSE with higher Go probability. CSE adaptation seemed long-lasting, because the two CSE measures, at least 1500 ms apart, strongly correlated. Patients with schizophrenia failed to show movement onset (RT) adaptation and modulation of CSE. In contrast, all groups decreased movement duration with increasing Go probability. Modulation of CSE in the anticipatory phase of the potential movement reflected the estimation of upcoming response probability in unaffected controls and siblings. Impaired modulation of CSE supports the hypothesis that implicit adaptation to probabilistic context is altered in schizophrenia.

The Hyperfocusing Hypothesis: A New Account of Cognitive Dysfunction in Schizophrenia

Impairments in basic cognitive processes such as attention and working memory are commonly observed in people with schizophrenia and are predictive of long-term outcome. In this review, we describe a new theory-the hyperfocusing hypothesis-which provides a unified account of many aspects of impaired cognition in schizophrenia. This hypothesis proposes that schizophrenia involves an abnormally narrow but intense focusing of processing resources. This hyperfocusing impairs the ability of people with schizophrenia to distribute attention among multiple locations, decreases the number of representations that can simultaneously be maintained in working memory, and causes attention to be abnormally captured by irrelevant inputs that share features with active representations. Evidence supporting the hyperfocusing hypothesis comes from a variety of laboratory tasks and from both behavioral and electrophysiological measures of processing. In many of these tasks, people with schizophrenia exhibit supranormal effects of task manipulations, which cannot be explained by a generalized cognitive deficit or by nonspecific factors such as reduced motivation or poor task comprehension. In addition, the degree of hyperfocusing in these tasks is often correlated with the degree of impairment in measures of broad cognitive function, which are known to be related to long-term outcome. Thus, the mechanisms underlying hyperfocusing may be a good target for new treatments targeting cognitive deficits in schizophrenia.

Towards a Unifying Cognitive, Neurophysiological, and Computational Neuroscience Account of Schizophrenia

Psychotic experiences may be understood as altered information processing due to aberrant neural computations. A prominent example of such neural computations is the computation of prediction errors (PEs), which signal the difference between expected and experienced events. Among other areas showing PE coding, hippocampal-prefrontal-striatal neurocircuits play a prominent role in information processing. Dysregulation of dopaminergic signaling, often secondary to psychosocial stress, is thought to interfere with the processing of biologically important events (such as reward prediction errors) and result in the aberrant attribution of salience to irrelevant sensory stimuli and internal representations. Bayesian hierarchical predictive coding offers a promising framework for the identification of dysfunctional neurocomputational processes and the development of a mechanistic understanding of psychotic experience. According to this framework, mismatches between prior beliefs encoded at higher levels of the cortical hierarchy and lower-level (sensory) information can also be thought of as PEs, with important consequences for belief updating. Low levels of precision in the representation of prior beliefs relative to sensory data, as well as dysfunctional interactions between prior beliefs and sensory data in an ever-changing environment, have been suggested as a general mechanism underlying psychotic experiences. Translating the promise of the Bayesian hierarchical predictive coding into patient benefit will come from integrating this framework with existing knowledge of the etiology and pathophysiology of psychosis, especially regarding hippocampal-prefrontal-striatal network function and neural mechanisms of information processing and belief updating.

From neuroimaging to daily functioning: A multimethod analysis of reward anticipation in people with schizophrenia

Negative symptoms are a core clinical feature of schizophrenia that are only marginally responsive to current treatments. Recent work suggests that deficits in reinforcement learning and anticipatory responses to reward may be two mechanisms that help explain impairments in motivation in those with schizophrenia. The present study utilized a reinforcement-learning paradigm, which allowed us to examine both reward anticipation and reinforcement learning. Twenty-eight people with schizophrenia and 30 healthy controls completed a reinforcement-learning task while undergoing functional MRI. Participants with schizophrenia also completed a weeklong ecological momentary assessment protocol reporting anticipated motivation and pleasure in their daily activities. Unexpectedly, we found no significant group differences in performance or neural response in reinforcement learning. However, we found that poorer reward learning was associated with greater clinician ratings of negative symptoms and daily reports of anticipatory motivation and pleasure negative symptoms. In regards to anticipatory responses, we found that people with schizophrenia showed blunted activation in the anterior cingulate, insula, caudate, and putamen while anticipating reward. Further, blood oxygen level-dependent (BOLD) response in reward related regions during anticipation of reward was significantly related to both clinician-rated motivation and pleasure deficits as well as daily reports of motivation and pleasure. Our results provide further evidence of deficits during reward anticipation in individuals with schizophrenia, particularly for those with severe negative symptoms, and some evidence for worse reward learning among those with greater negative symptoms. Moreover, our findings suggest that these deficits show important relationships with emotional and motivational functioning in everyday life. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Nonsocial and social cognition in schizophrenia: current evidence and future directions

Cognitive impairment in schizophrenia involves a broad array of nonsocial and social cognitive domains. It is a core feature of the illness, and one with substantial implications for treatment and prognosis. Our understanding of the causes, consequences and interventions for cognitive impairment in schizophrenia has grown substantially in recent years. Here we review a range of topics, including: a) the types of nonsocial cognitive, social cognitive, and perceptual deficits in schizophrenia; b) how deficits in schizophrenia are similar or different from those in other disorders; c) cognitive impairments in the prodromal period and over the lifespan in schizophrenia; d) neuroimaging of the neural substrates of nonsocial and social cognition, and e) relationships of nonsocial and social cognition to functional outcome. The paper also reviews the considerable efforts that have been directed to improve cognitive impairments in schizophrenia through novel psychopharmacology, cognitive remediation, social cognitive training, and alternative approaches. In the final section, we consider areas that are emerging and have the potential to provide future insights, including the interface of motivation and cognition, the influence of childhood adversity, metacognition, the role of neuroinflammation, computational modelling, the application of remote digital technology, and novel methods to evaluate brain network organization. The study of cognitive impairment has provided a way to approach, examine and comprehend a wide range of features of schizophrenia, and it may ultimately affect how we define and diagnose this complex disorder.

Clarifying the overlap between motivation and negative symptom measures in schizophrenia research: A meta-analysis

Motivation and negative symptom research has recently been hampered by a series of inconsistent findings, leading to calls for a greater consensus on the type of measures used across studies. To inform this issue, we conducted a meta-analysis that quantified the association between motivation measures (self-report, performance-based) and clinician-rated negative symptom measures as well as a series of moderator analyses to develop a greater understanding of the measurement factors impacting this relationship. Forty-seven eligible studies with people with schizophrenia-spectrum disorders were included. Using a random-effects meta-analytic model, a small but significant overall effect size emerged between motivation and clinician-rated negative symptoms (r = -0.18). Several significant moderators were identified, including the generation of negative symptom measures such that there was a significantly stronger relationship between motivation and second-generation (r = -0.38) than first-generation negative symptom measures (r = -0.17). Further, the type of performance-based measure used moderated the relationship, with effort discounting tasks most strongly related to negative symptoms (r = -0.44). The domain of motivation assessed (intrinsic, extrinsic, amotivation) also moderated the relationship. These findings help to identify sources of inconsistencies observed in prior studies and point to both second-generation and effort discounting tasks as the most promising types of measures, particularly for those interested in validating motivation measures or assessing the effectiveness of motivation treatments. Although additional research is needed, our results suggest that using these measures may help to reduce inconsistencies across studies and move the field forward.

Reduced Neural Sensitivity to Social vs Nonsocial Reward in Schizophrenia

BACKGROUND

Human beings find social stimuli rewarding, which is thought to facilitate efficient social functioning. Although reward processing has been extensively studied in schizophrenia, a few studies have examined neural processes specifically involved in social reward processing. This study examined neural sensitivity to social and nonsocial rewards in schizophrenia.

METHODS

Twenty-seven patients with schizophrenia and 25 community controls completed a One-Armed Bandit Task, an implicit reinforcement learning task, in the scanner. There were 2 conditions with an identical trial structure, one with social rewards and the other with nonsocial rewards. The data were analyzed using a region of interest (ROI) approach, focusing on the ventral striatum, ventromedial prefrontal cortex, and anterior cingulate cortex.

RESULTS

Across all 3 ROIs, patients showed reduced activation for social rewards compared to controls. However, the 2 groups showed comparable levels of activation for nonsocial rewards. Within the patient group, levels of neural activation in these ROIs during the social reward condition were associated with better performance.

CONCLUSIONS

This study found reduced neural sensitivity in patients with schizophrenia in key reward-processing regions for social but not for nonsocial rewards. These findings suggest a relatively specific social reward-processing deficit in schizophrenia during an implicit reinforcement learning task.

Socially Learned Attitude Change is not reduced in Medicated Patients with Schizophrenia

Schizophrenia is often associated with distinctive or odd social behaviours. Previous work suggests this could be due to a general reduction in conformity; however, this work only assessed the tendency to publicly agree with others, which may involve a number of different mechanisms. In this study, we specifically investigated whether patients display a reduced tendency to adopt other people’s opinions (socially learned attitude change). We administered a computerized conformity task, assumed to rely on reinforcement learning circuits, to 32 patients with schizophrenia or schizo-affective disorder and 39 matched controls. Each participant rated 153 faces for trustworthiness. After each rating, they were immediately shown the opinion of a group. After approximately 1 hour, participants were unexpectedly asked to rate all the faces again. We compared the degree of attitude change towards group opinion in patients and controls. Patients presented equal or more social influence on attitudes than controls. This effect may have been medication induced, as increased conformity was seen with higher antipsychotic dose. The results suggest that there is not a general decline in conformity in medicated patients with schizophrenia and that previous findings of reduced conformity are likely related to mechanisms other than reinforcement based social influence on attitudes.

Translational tests involving non-reward: methodological considerations

This review is concerned with methods for assessing the processing of unrewarded responses in experimental animals and the mechanisms underlying performance of these tasks. A number of clinical populations, including Parkinson's disease, depression, compulsive disorders, and schizophrenia demonstrate either abnormal processing or learning from non-rewarded responses in laboratory-based reinforcement learning tasks. These effects are hypothesized to result from disturbances in modulatory neurotransmitter systems, including dopamine and serotonin. Parallel work in experimental animals has revealed consistent behavioral patterns associated with non-reward and, consistent with the human literature, modulatory roles for specific neurotransmitters. Classical tests involving an important reward omission component include appetitive extinction, ratio schedules of responding, reversal learning, and delay and probability discounting procedures. In addition, innovative behavioral tests have recently been developed leverage probabilistic feedback to specifically assay accommodation of, and learning from, non-rewarded responses. These procedures will be described and reviewed with discussion of the behavioral and neural determinants of performance. A final section focusses specifically on the benefits of trial-by-trial analysis of responding during such tasks, and the implications of such analyses for the translation of findings to clinical studies.

Impact of Positive Emotion Regulation Training on Negative Symptoms and Social Functioning in Schizophrenia: A Field Test

Background: The poor efficacy of drug or psychological treatments on the primary negative symptoms of schizophrenia has led to the development of new interventions. The Positive Emotions Program for Schizophrenia (PEPS) is an emotion regulation strategy training that aims to intensify positive emotions and develop positive performance beliefs. A randomized controlled trial showed that PEPS is effective in reducing the composite score of the reduction of experience syndrome (anhedonia and apathy). The present study is designed to evaluate its feasibility in natural conditions to measure external validity of PEPS. Materials and Methods: Twenty-one participants recruited through the French national network of expert centers followed eight sessions of PEPS and were assessed pre- and posttest with the Scale for Assessment of Negative Symptoms (SANS) and the Personal and Social Performance (PSP). The scales of the SANS were divided into a composite score of the reduction of the ability to experience and a composite score of the reduction of expression. Results: All participants followed the 8 sessions of PEPS, and both composite scores were significantly and clinically improved at posttest. Social functioning assessed with the PSP was also improved. Conclusions: This field test shows that participation in PEPS is accompanied by a reduction of negative symptoms and an improvement of social functioning. Both negative syndromes, reduction of expression and reduction of experience, are improved. Participants are younger than those in previous studies, which may explain this unexpected result. However, this calls for a controlled study with younger participants.

Impaired Expected Value Computations in Schizophrenia Are Associated With a Reduced Ability to Integrate Reward Probability and Magnitude of Recent Outcomes

BACKGROUND

Motivational deficits in people with schizophrenia (PSZ) are associated with an inability to integrate the magnitude and probability of previous outcomes. The mechanisms that underlie probability-magnitude integration deficits, however, are poorly understood. We hypothesized that increased reliance on "valueless" stimulus-response associations, in lieu of expected value (EV)-based learning, could drive probability-magnitude integration deficits in PSZ with motivational deficits.

METHODS

Healthy volunteers (n = 38) and PSZ (n = 49) completed a learning paradigm consisting of four stimulus pairs. Reward magnitude (3, 2, 1, 0 points) and probability (90%, 80%, 20%, 10%) determined each stimulus's EV. Following a learning phase, new and familiar stimulus pairings were presented. Participants were asked to select stimuli with the highest reward value.

RESULTS

PSZ with high motivational deficits made increasingly less optimal choices as the difference in reward value (probability × magnitude) between two competing stimuli increased. Using a previously validated computational hybrid model, PSZ relied less on EV ("Q-learning") and more on stimulus-response learning ("actor-critic"), which correlated with Scale for the Assessment of Negative Symptoms motivational deficit severity. PSZ specifically failed to represent reward magnitude, consistent with model demonstrations showing that response tendencies in the actor-critic were preferentially driven by reward probability.

CONCLUSIONS

Probability-magnitude deficits in PSZ with motivational deficits arise from underutilization of EV in favor of reliance on valueless stimulus-response associations. Confirmed by our computational hybrid framework, probability-magnitude integration deficits were driven specifically by a failure to represent reward magnitude. This work provides a first mechanistic explanation of complex EV-based learning deficits in PSZ with motivational deficits that arise from an inability to combine information from different reward modalities.

Daily-life stress differentially impacts ventral striatal dopaminergic modulation of reward processing in first-degree relatives of individuals with psychosis

Emerging evidence shows that stress can impair the ability to learn from and pursue rewards, which in turn has been linked to motivational impairments characteristic of the psychotic disorder. Ventral striatal dopaminergic neurotransmission has been found to modulate reward processing, and appears to be disrupted by exposure to stress. We investigated the hypothesis that stress experienced in the everyday life has a blunting effect on reward-induced dopamine release in the ventral striatum of 16 individuals at a familial risk for psychosis compared to 16 matched control subjects. Six days of ecological momentary assessments quantified the amount of daily-life stress prior to [¹⁸F]fallypride PET imaging while performing a probabilistic reinforcement learning task. Relative to the controls, individuals at a familial risk for psychosis who encountered more daily-life stress showed significantly diminished extent of reward-induced dopamine release in the right ventral striatum, as well as poorer performance on the reward task. These findings provide the first neuromolecular evidence for stress-related deregulation of reward processing in familial predisposition to psychosis. The implication of daily-life stress in compromised modulation of reward function may facilitate the design of targeted neuropharmacological and ecological interventions.

How pupil responses track value-based decision-making during and after reinforcement learning

Cognition can reveal itself in the pupil, as latent cognitive processes map onto specific pupil responses. For instance, the pupil dilates when we make decisions and these pupil size fluctuations reflect decision-making computations during and after a choice. Surprisingly little is known, however, about how pupil responses relate to decisions driven by the learned value of stimuli. This understanding is important, as most real-life decisions are guided by the outcomes of earlier choices. The goal of this study was to investigate which cognitive processes the pupil reflects during value-based decision-making. We used a reinforcement learning task to study pupil responses during value-based decisions and subsequent decision evaluations, employing computational modeling to quantitatively describe the underlying cognitive processes. We found that the pupil closely tracks reinforcement learning processes independently across participants and across trials. Prior to choice, the pupil dilated as a function of trial-by-trial fluctuations in value beliefs about the to-be chosen option and predicted an individual's tendency to exploit high value options. After feedback a biphasic pupil response was observed, the amplitude of which correlated with participants' learning rates. Furthermore, across trials, early feedback-related dilation scaled with value uncertainty, whereas later constriction scaled with signed reward prediction errors. These findings show that pupil size fluctuations can provide detailed information about the computations underlying value-based decisions and the subsequent updating of value beliefs. As these processes are affected in a host of psychiatric disorders, our results indicate that pupillometry can be used as an accessible tool to non-invasively study the processes underlying ongoing reinforcement learning in the clinic.

Differential neural reward mechanisms in treatment-responsive and treatment-resistant schizophrenia

BACKGROUND

The significant proportion of schizophrenia patients refractory to treatment, primarily directed at the dopamine system, suggests that multiple mechanisms may underlie psychotic symptoms. Reinforcement learning tasks have been employed in schizophrenia to assess dopaminergic functioning and reward processing, but these have not directly compared groups of treatment-refractory and non-refractory patients.

METHODS

In the current functional magnetic resonance imaging study, 21 patients with treatment-resistant schizophrenia (TRS), 21 patients with non-treatment-resistant schizophrenia (NTR), and 24 healthy controls (HC) performed a probabilistic reinforcement learning task, utilizing emotionally valenced face stimuli which elicit a social bias toward happy faces. Behavior was characterized with a reinforcement learning model. Trial-wise reward prediction error (RPE)-related neural activation and the differential impact of emotional bias on these reward signals were compared between groups.

RESULTS

Patients showed impaired reinforcement learning relative to controls, while all groups demonstrated an emotional bias favoring happy faces. The pattern of RPE signaling was similar in the HC and TRS groups, whereas NTR patients showed significant attenuation of RPE-related activation in striatal, thalamic, precentral, parietal, and cerebellar regions. TRS patients, but not NTR patients, showed a positive relationship between emotional bias and RPE signal during negative feedback in bilateral thalamus and caudate.

CONCLUSION

TRS can be dissociated from NTR on the basis of a different neural mechanism underlying reinforcement learning. The data support the hypothesis that a favorable response to antipsychotic treatment is contingent on dopaminergic dysfunction, characterized by aberrant RPE signaling, whereas treatment resistance may be characterized by an abnormality of a non-dopaminergic mechanism - a glutamatergic mechanism would be a possible candidate.

Deficits in context-dependent adaptive coding in early psychosis and healthy individuals with schizotypal personality traits

Adaptive coding of information is a fundamental principle of brain functioning. It allows for efficient representation over a large range of inputs and thereby alleviates the limited coding range of neurons. In the present study, we investigated for the first time potential alterations in context-dependent reward adaptation and its association with symptom dimensions in the schizophrenia spectrum. We studied 27 patients with first-episode psychosis, 26 individuals with schizotypal personality traits and 25 healthy controls. We used functional MRI in combination with a variant of the monetary incentive delay task and assessed adaptive reward coding in two reward conditions with different reward ranges. Compared to healthy controls, patients with first-episode psychosis and healthy individuals with schizotypal personality traits showed a deficit in increasing the blood oxygen level-dependent response slope in the right caudate for the low reward range compared to the high reward range. In other words, the two groups showed inefficient neural adaptation to the current reward context. In addition, we found impaired adaptive coding of reward in the caudate nucleus and putamen to be associated with total symptom severity across the schizophrenia spectrum. Symptom severity was more strongly associated with neural deficits in adaptive coding than with the neural coding of absolute reward outcomes. Deficits in adaptive coding were prominent across the schizophrenia spectrum and even detectable in unmedicated (healthy) individuals with schizotypal personality traits. Furthermore, the association between total symptom severity and impaired adaptive coding in the right caudate and putamen suggests a dimensional mechanism underlying imprecise neural adaptation. Our findings support the idea that impaired adaptive coding may be a general information-processing deficit explaining disturbances within the schizophrenia spectrum over and above a simple model of blunted absolute reward signals.

Modeling subjective relevance in schizophrenia and its relation to aberrant salience

In schizophrenia, increased aberrant salience to irrelevant events and reduced learning of relevant information may relate to an underlying deficit in relevance detection. So far, subjective estimates of relevance have not been probed in schizophrenia patients. The mechanisms underlying belief formation about relevance and their translation into decisions are unclear. Using novel computational methods, we investigated relevance detection during implicit learning in 42 schizophrenia patients and 42 healthy individuals. Participants underwent functional magnetic resonance imaging while detecting the outcomes in a learning task. These were preceded by cues differing in color and shape, which were either relevant or irrelevant for outcome prediction. We provided a novel definition of relevance based on Bayesian precision and modeled reaction times as a function of relevance weighted unsigned prediction errors (UPE). For aberrant salience, we assessed responses to subjectively irrelevant cue manifestations. Participants learned the contingencies and slowed down their responses following unexpected events. Model selection revealed that individuals inferred the relevance of cue features and used it for behavioral adaption to the relevant cue feature. Relevance weighted UPEs correlated with dorsal anterior cingulate cortex activation and hippocampus deactivation. In patients, the aberrant salience bias to subjectively task-irrelevant information was increased and correlated with decreased striatal UPE activation and increased negative symptoms. This study shows that relevance estimates based on Bayesian precision can be inferred from observed behavior. This underscores the importance of relevance detection as an underlying mechanism for behavioral adaptation in complex environments and enhances the understanding of aberrant salience in schizophrenia.

Schizophrenia patients display deficits in the appropriate attribution of meaningfulness to stimuli; such as aberrantly increased processing of irrelevant and insufficient processing of relevant information. We aimed to investigate the subjective nature of relevance detection and its deficit in schizophrenia and developed an implicit learning paradigm that allowed for parallel learning from relevant and irrelevant information. Based on the idea that subjective relevance might be captured by Bayesian precision we set up different computational models of how subjective relevance guides learning and behavioral adaptation. We found that subjects use Bayesian precision to estimate stimulus relevance in order to integrate multidimensional information and adapt more to the subjectively relevant stimuli. This relevance weighted adaptation correlated with brain activation within the salience network. Further, schizophrenia patients displayed an increased aberrant tendency to irrelevant events which related to decreased striatal coding of the relevant learning signal. To conclude, our findings demonstrate how individual beliefs about relevance can be inferred from computational models. Furthermore, we suggest that aberrant salience observed in patients with schizophrenia reflects an idiosyncratic bias in states of high subjective uncertainty.

Abnormal reward prediction-error signalling in antipsychotic naive individuals with first-episode psychosis or clinical risk for psychosis

Ongoing research suggests preliminary, though not entirely consistent, evidence of neural abnormalities in signalling prediction errors in schizophrenia. Supporting theories suggest mechanistic links between the disruption of these processes and the generation of psychotic symptoms. However, it is unknown at what stage in the pathogenesis of psychosis these impairments in prediction-error signalling develop. One major confound in prior studies is the use of medicated patients with strongly varying disease durations. Our study aims to investigate the involvement of the meso-cortico-striatal circuitry during reward prediction-error signalling in earliest stages of psychosis. We studied patients with first-episode psychosis (FEP) and help-seeking individuals at-risk for psychosis due to sub-threshold prodromal psychotic symptoms. Patients with either FEP (n = 14), or at-risk for developing psychosis (n = 30), and healthy volunteers (n = 39) performed a reinforcement learning task during fMRI scanning. ANOVA revealed significant (p < 0.05 family-wise error corrected) prediction-error signalling differences between groups in the dopaminergic midbrain and right middle frontal gyrus (dorsolateral prefrontal cortex, DLPFC). FEP patients showed disrupted reward prediction-error signalling compared to controls in both regions. At-risk patients showed intermediate activation in the midbrain that significantly differed from controls and from FEP patients, but DLPFC activation that did not differ from controls. Our study confirms that FEP patients have abnormal meso-cortical signalling of reward-prediction errors, whereas reward-prediction-error dysfunction in the at-risk patients appears to show a more nuanced pattern of activation with a degree of midbrain impairment but preserved cortical function.

Investigating the association of ventral and dorsal striatal dysfunction during reward anticipation with negative symptoms in patients with schizophrenia and healthy individuals

Background

Negative symptoms are a core feature of schizophrenia and also found in healthy individuals in subclinical forms. According to the current literature the two negative symptom domains, apathy and diminished expression may have different underlying neural mechanisms. Previous observations suggest that striatal dysfunction is associated with apathy in schizophrenia. However, it is unclear whether apathy is specifically related to ventral or dorsal striatal alterations. Here, we investigated striatal dysfunction during reward anticipation in patients with schizophrenia and a non-clinical population, to determine whether it is associated with apathy.

Methods

Chronic schizophrenia patients (n = 16) and healthy controls (n = 23) underwent an event- related functional MRI, while performing a variant of the Monetary Incentive Delay Task. The two negative symptom domains were assessed in both groups using the Brief Negative Symptoms Scale.

Results

In schizophrenia patients, we saw a strong negative correlation between apathy and ventral and dorsal striatal activation during reward anticipation. In contrast, there was no correlation with diminished expression. In healthy controls, apathy was not correlated with ventral or dorsal striatal activation during reward anticipation.

Conclusion

This study replicates our previous findings of a correlation between ventral striatal activity and apathy but not diminished expression in chronic schizophrenia patients. The association between apathy and reduced dorsal striatal activity during reward anticipation suggests that impaired action-outcome selection is involved in the pathophysiology of motivational deficits in schizophrenia.

Impaired Expected Value Computations Coupled With Overreliance on Stimulus-Response Learning in Schizophrenia

BACKGROUND

While many have emphasized impaired reward prediction error signaling in schizophrenia, multiple studies suggest that some decision-making deficits may arise from overreliance on stimulus-response systems together with a compromised ability to represent expected value. Guided by computational frameworks, we formulated and tested two scenarios in which maladaptive representations of expected value should be most evident, thereby delineating conditions that may evoke decision-making impairments in schizophrenia.

METHODS

In a modified reinforcement learning paradigm, 42 medicated people with schizophrenia and 36 healthy volunteers learned to select the most frequently rewarded option in a 75-25 pair: once when presented with a more deterministic (90-10) pair and once when presented with a more probabilistic (60-40) pair. Novel and old combinations of choice options were presented in a subsequent transfer phase. Computational modeling was employed to elucidate contributions from stimulus-response systems (actor-critic) and expected value (Q-learning).

RESULTS

People with schizophrenia showed robust performance impairments with increasing value difference between two competing options, which strongly correlated with decreased contributions from expected value-based learning (Q-learning). Moreover, a subtle yet consistent contextual choice bias for the probabilistic 75 option was present in people with schizophrenia, which could be accounted for by a context-dependent reward prediction error in the actor-critic.

CONCLUSIONS

We provide evidence that decision-making impairments in schizophrenia increase monotonically with demands placed on expected value computations. A contextual choice bias is consistent with overreliance on stimulus-response learning, which may signify a deficit secondary to the maladaptive representation of expected value. These results shed new light on conditions under which decision-making impairments may arise.

Meta-analytic evidence for altered mesolimbic responses to reward in schizophrenia

Dysfunction of reward-related neural circuitry in schizophrenia (SCZ) has been widely reported, and may provide insight into the motivational and cognitive disturbances that characterize the disorder. Although previous meta-analyses of reward learning paradigms in SCZ have been performed, a meta-analysis of whole-brain coordinate maps in SCZ alone has not been conducted. In this study, we performed an activation likelihood estimate (ALE) meta-analysis, and performed a follow-up analysis of functional connectivity and functional decoding of identified regions. We report several salient findings that extend prior work in this area. First, an alteration in reward-related activation was observed in the right ventral striatum, but this was not solely driven by hypoactivation in the SCZ group compared to healthy controls. Second, the region was characterized by functional connectivity primarily with the lateral prefrontal cortex and pre-supplementary motor area (preSMA), as well as subcortical regions such as the thalamus which show structural deficits in SCZ. Finally, although the meta-analysis showed no regions outside the ventral striatum to be significantly altered, regions with higher functional connectivity with the ventral striatum showed a greater number of subthreshold foci. Together, these findings confirm the alteration of ventral striatal function in SCZ, but suggest that a network-based approach may assist future analysis of the functional underpinnings of the disorder.

Influence of facial feedback during a cooperative human-robot task in schizophrenia

Rapid progress in the area of humanoid robots offers tremendous possibilities for investigating and improving social competences in people with social deficits, but remains yet unexplored in schizophrenia. In this study, we examined the influence of social feedbacks elicited by a humanoid robot on motor coordination during a human-robot interaction. Twenty-two schizophrenia patients and twenty-two matched healthy controls underwent a collaborative motor synchrony task with the iCub humanoid robot. Results revealed that positive social feedback had a facilitatory effect on motor coordination in the control participants compared to non-social positive feedback. This facilitatory effect was not present in schizophrenia patients, whose social-motor coordination was similarly impaired in social and non-social feedback conditions. Furthermore, patients' cognitive flexibility impairment and antipsychotic dosing were negatively correlated with patients' ability to synchronize hand movements with iCub. Overall, our findings reveal that patients have marked difficulties to exploit facial social cues elicited by a humanoid robot to modulate their motor coordination during human-robot interaction, partly accounted for by cognitive deficits and medication. This study opens new perspectives for comprehension of social deficits in this mental disorder.

Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia

BACKGROUND

When studying learning, researchers directly observe only the participants' choices, which are often assumed to arise from a unitary learning process. However, a number of separable systems, such as working memory (WM) and reinforcement learning (RL), contribute simultaneously to human learning. Identifying each system's contributions is essential for mapping the neural substrates contributing in parallel to behavior; computational modeling can help to design tasks that allow such a separable identification of processes and infer their contributions in individuals.

METHODS

We present a new experimental protocol that separately identifies the contributions of RL and WM to learning, is sensitive to parametric variations in both, and allows us to investigate whether the processes interact. In experiments 1 and 2, we tested this protocol with healthy young adults (n = 29 and n = 52, respectively). In experiment 3, we used it to investigate learning deficits in medicated individuals with schizophrenia (n = 49 patients, n = 32 control subjects).

RESULTS

Experiments 1 and 2 established WM and RL contributions to learning, as evidenced by parametric modulations of choice by load and delay and reward history, respectively. They also showed interactions between WM and RL, where RL was enhanced under high WM load. Moreover, we observed a cost of mental effort when controlling for reinforcement history: participants preferred stimuli they encountered under low WM load. Experiment 3 revealed selective deficits in WM contributions and preserved RL value learning in individuals with schizophrenia compared with control subjects.

CONCLUSIONS

Computational approaches allow us to disentangle contributions of multiple systems to learning and, consequently, to further our understanding of psychiatric diseases.

Glutamatergic regulation of cognition and functional brain connectivity: insights from pharmacological, genetic and translational schizophrenia research

The pharmacological modulation of glutamatergic neurotransmission to improve cognitive function has been a focus of intensive research, particularly in relation to the cognitive deficits seen in schizophrenia. Despite this effort, there has been little success in the clinical use of glutamatergic compounds as procognitive drugs. Here, we review a selection of the drugs used to modulate glutamatergic signalling and how they impact on cognitive function in rodents and humans. We highlight how glutamatergic dysfunction, and NMDA receptor hypofunction in particular, is a key mechanism contributing to the cognitive deficits observed in schizophrenia and outline some of the glutamatergic targets that have been tested as putative procognitive targets for this disorder. Using translational research in this area as a leading exemplar, namely, models of NMDA receptor hypofunction, we discuss how the study of functional brain network connectivity can provide new insight into how the glutamatergic system impacts on cognitive function. Future studies characterizing functional brain network connectivity will increase our understanding of how glutamatergic compounds regulate cognition and could contribute to the future success of glutamatergic drug validation. Linked Articles This article is part of a themed section on Pharmacology of Cognition: a Panacea for Neuropsychiatric Disease? To view the other articles in this section visit http://onlinelibrary.wiley.com/doi/10.1111/bph.v174.19/issuetoc.