Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence

Pharmacological Enhancement of Dopamine Neurotransmission Does Not Affect Illusory Pattern Perception

Psychotic symptoms and delusional beliefs have been linked to dopamine transmission in both healthy and clinical samples and are assumed to result at least in part from perceiving illusory patterns in noise. However, the existing literature on the role of dopamine in detecting patterns in noise is inconclusive. To address this issue, we assessed the effect of manipulating dopaminergic neurotransmission on illusory pattern perception in healthy individuals (n = 48, n = 19 female) in a double-blind placebo-controlled within-subjects design (see preregistration at https://osf.io/a4k9j/). We predicted individuals on versus off ʟ-DOPA to be more likely to perceive illusory patterns, specifically objects in images containing only noise. Using a signal detection model, however, we found no credible evidence that ʟ-DOPA compared with placebo increased false alarm rates. Further, ʟ-DOPA did not reliably modulate measures of accuracy, discrimination sensitivity, and response bias. In all cases, Bayesian statistics revealed strong evidence in favor of the null hypothesis. The task design followed previous work on illusory pattern perception and comprised a limited number of items per condition. The results therefore need to be interpreted with caution, as power was limited. Future studies should address illusory pattern perception using more items and take into account potential dose-dependent effects and differential effects in healthy versus clinical samples.

Impaired flexible reward learning in ADHD patients is associated with blunted reinforcement sensitivity and neural signals in ventral striatum and parietal cortex

Reward-based learning and decision-making are prime candidates to understand symptoms of attention deficit hyperactivity disorder (ADHD). However, only limited evidence is available regarding the neurocomputational underpinnings of the alterations seen in ADHD. This concerns flexible behavioral adaption in dynamically changing environments, which is challenging for individuals with ADHD. One previous study points to elevated choice switching in adolescent ADHD, which was accompanied by disrupted learning signals in medial prefrontal cortex. Here, we investigated young adults with ADHD (n = 17) as compared to age- and sex-matched controls (n = 17) using a probabilistic reversal learning experiment during functional magnetic resonance imaging (fMRI). The task requires continuous learning to guide flexible behavioral adaptation to changing reward contingencies. To disentangle the neurocomputational underpinnings of the behavioral data, we used reinforcement learning (RL) models, which informed the analysis of fMRI data. ADHD patients performed worse than controls particularly in trials before reversals, i.e., when reward contingencies were stable. This pattern resulted from 'noisy' choice switching regardless of previous feedback. RL modelling showed decreased reinforcement sensitivity and enhanced learning rates for negative feedback in ADHD patients. At the neural level, this was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Moreover, modelling showed a marginal reduction of learning about the unchosen option, which was paralleled by a marginal reduction in learning signals incorporating the unchosen option in the left ventral striatum. Taken together, we show that impaired flexible behavior in ADHD is due to excessive choice switching ('hyper-flexibility'), which can be detrimental or beneficial depending on the learning environment. Computationally, this resulted from blunted sensitivity to reinforcement of which we detected neural correlates in the attention-control network, specifically in the parietal cortex. These neurocomputational findings remain preliminary due to the relatively small sample size.

The right superior temporal gyrus plays a role in semantic-rule learning: Evidence supporting a reinforcement learning model

In real-life communication, individuals use language that carries evident rewarding and punishing elements, such as praise and criticism. A common trend is to seek more praise while avoiding criticism. Furthermore, semantics is crucial for conveying information, but such semantic access to native and foreign languages is subtly distinct. To investigate how rule learning occurs in different languages and to highlight the importance of semantics in this process, we investigated both verbal and non-verbal rule learning in first (L1) and second (L2) languages using a reinforcement learning framework, including a semantic rule and a color rule. Our computational modeling on behavioral and brain imaging data revealed that individuals may be more motivated to learn and adhere to rules in an L1 compared to L2, with greater striatum activation during the outcome phase in the L1. Additionally, results on the learning rates and inverse temperature in the two rule learning tasks showed that individuals tend to be conservative and are reluctant to change their judgments regarding rule learning of semantic information. Moreover, the greater the prediction errors, the greater activation of the right superior temporal gyrus in the semantic-rule learning condition, demonstrating that such learning has differential neural correlates than symbolic rule learning. Overall, the findings provide insight into the neural mechanisms underlying rule learning in different languages, and indicate that rule learning involving verbal semantics is not a general symbolic learning that resembles a conditioned stimulus-response, but rather has its own specific characteristics.

Effects of Adult Age and Functioning of the Locus Coeruleus Norepinephrinergic System on Reward-Based Learning

Age-related impairments in value representations and updating during decision-making and reward-based learning are often related to age-related attenuation in the catecholamine system such as dopamine (DA) and norepinephrine (NE). However, it is unclear to what extent age-related declines in NE functioning in humans affect reward-based decision-making. We conducted a probabilistic decision-making task and applied a Q-learning model to investigate participants' anticipatory values and value sensitivities. Task-related pupil dilations and locus coeruleus (LC) magnetic resonance imaging (MRI) contrast, which served as a potential window of the LC-NE functions, were assessed in younger and older adults. Results showed that in both choice and feedback phases, younger adults' (N = 42, 22 males) pupil dilations negatively correlated with anticipatory values, indicating uncertainty about outcome probabilities. Uncertainty-evoked pupil dilations in older adults (N = 41, 27 males) were smaller, indicating age-related impairments in value estimation and updating. In both age groups, participants who showed a larger uncertainty-evoked pupil dilation exhibited a higher value sensitivity as reflected in the β parameter of the reinforcement Q-learning model. Furthermore, older adults (N = 34, 29 males) showed a lower LC-MRI contrast than younger adults (N = 25, 15 males). The LC-MRI contrast positively correlated with value sensitivity only in older but not in younger adults. These findings suggest that task-related pupillary responses can reflect age-related deficits in value estimation and updating during reward-based decision-making. Our evidence with the LC-MRI contrast further showed the age-related decline of the LC structure in modulating value representations during reward-based learning.SIGNIFICANCE STATEMENT Age-related impairments in value representation and updating during reward-based learning are associated with declines in the catecholamine modulation with age. However, it is unclear how age-related declines in the LC-NE system may affect reward-based learning. Here, we show that compared with younger adults, older adults exhibited reduced uncertainty-induced pupil dilations, suggesting age-related deficits in value estimation and updating. Older adults showed a lower structural MRI of the LC contrast than younger adults, indicating age-related degeneration of the LC structure. The association between the LC-MRI contrast and value sensitivity was only observed in older adults. Our findings may demonstrate a pioneering model to unravel the role of the LC-NE system in reward-based learning in aging.

Insular and Striatal Correlates of Uncertain Risky Reward Pursuit in Schizophrenia

BACKGROUND AND HYPOTHESIS

Risk-taking in specific contexts can be beneficial, leading to rewarding outcomes. Schizophrenia is associated with disadvantageous decision-making, as subjects pursue uncertain risky rewards less than controls. However, it is unclear whether this behavior is associated with more risk sensitivity or less reward incentivization. Matching on demographics and intelligence quotient (IQ), we determined whether risk-taking was more associated with brain activation in regions affiliated with risk evaluation or reward processing.

STUDY DESIGN

Subjects (30 schizophrenia/schizoaffective disorder, 30 controls) completed a modified, fMRI Balloon Analogue Risk Task. Brain activation was modeled during decisions to pursue risky rewards and parametrically modeled according to risk level.

STUDY RESULTS

The schizophrenia group exhibited less risky-reward pursuit despite previous adverse outcomes (Average Explosions; F(1,59) = 4.06, P = .048) but the comparable point at which risk-taking was volitionally discontinued (Adjusted Pumps; F(1,59) = 2.65, P = .11). Less activation was found in schizophrenia via whole brain and region of interest (ROI) analyses in the right (F(1,59) = 14.91, P < 0.001) and left (F(1,59) = 16.34, P < 0.001) nucleus accumbens (NAcc) during decisions to pursue rewards relative to riskiness. Risk-taking correlated with IQ in schizophrenia, but not controls. Path analyses of average ROI activation revealed less statistically determined influence of anterior insula upon dorsal anterior cingulate bilaterally (left: χ2 = 12.73, P < .001; right: χ2 = 9.54, P = .002) during risky reward pursuit in schizophrenia.

CONCLUSIONS

NAcc activation in schizophrenia varied less according to the relative riskiness of uncertain rewards compared to controls, suggesting aberrations in reward processing. The lack of activation differences in other regions suggests similar risk evaluation. Less insular influence on the anterior cingulate may relate to attenuated salience attribution or inability for risk-related brain region collaboration to sufficiently perceive situational risk.

The relationship between resting-state amplitude fluctuations and memory-related deactivations of the default mode network in young and older adults

The default mode network (DMN) typically exhibits deactivations during demanding tasks compared to periods of relative rest. In functional magnetic resonance imaging (fMRI) studies of episodic memory encoding, increased activity in DMN regions even predicts later forgetting in young healthy adults. This association is attenuated in older adults and, in some instances, increased DMN activity even predicts remembering rather than forgetting. It is yet unclear whether this phenomenon is due to a compensatory mechanism, such as self-referential or schema-dependent encoding, or whether it reflects overall reduced DMN activity modulation in older age. We approached this question by systematically comparing DMN activity during successful encoding and tonic, task-independent, DMN activity at rest in a sample of 106 young (18-35 years) and 111 older (60-80 years) healthy participants. Using voxel-wise multimodal analyses, we assessed the age-dependent relationship between DMN resting-state amplitude (mean percent amplitude of fluctuation, mPerAF) and DMN fMRI signals related to successful memory encoding, as well as their modulation by age-related hippocampal volume loss, while controlling for regional grey matter volume. Older adults showed lower resting-state DMN amplitudes and lower task-related deactivations. However, a negative relationship between resting-state mPerAF and subsequent memory effect within the precuneus was observed only in young, but not older adults. Hippocampal volumes showed no relationship with the DMN subsequent memory effect or mPerAF. Lastly, older adults with higher mPerAF in the DMN at rest tend to show higher memory performance, pointing towards the importance of a maintained ability to modulate DMN activity in old age.

Functional neuroimaging in psychiatry and the case for failing better

Psychiatric disorders encompass complex aberrations of cognition and affect and are among the most debilitating and poorly understood of any medical condition. Current treatments rely primarily on interventions that target brain function (drugs) or learning processes (psychotherapy). A mechanistic understanding of how these interventions mediate their therapeutic effects remains elusive. From the early 1990s, non-invasive functional neuroimaging, coupled with parallel developments in the cognitive neurosciences, seemed to signal a new era of neurobiologically grounded diagnosis and treatment in psychiatry. Yet, despite three decades of intense neuroimaging research, we still lack a neurobiological account for any psychiatric condition. Likewise, functional neuroimaging plays no role in clinical decision making. Here, we offer a critical commentary on this impasse and suggest how the field might fare better and deliver impactful neurobiological insights.

Brain network dysfunctions in addiction: a meta-analysis of resting-state functional connectivity

Resting-state functional connectivity (rsFC) provides novel insights into variabilities in neural networks associated with the use of addictive drugs or with addictive behavioral repertoire. However, given the broad mix of inconsistent findings across studies, identifying specific consistent patterns of network abnormalities is warranted. Here we aimed at integrating rsFC abnormalities and systematically searching for large-scale functional brain networks in substance use disorder (SUD) and behavioral addictions (BA), through a coordinate-based meta-analysis of seed-based rsFC studies. A total of fifty-two studies are eligible in the meta-analysis, including 1911 SUD and BA patients and 1580 healthy controls. In addition, we performed multilevel kernel density analysis (MKDA) for the brain regions reliably involved in hyperconnectivity and hypoconnectivity in SUD and BA. Data from fifty-two studies showed that SUD was associated with putamen, caudate and middle frontal gyrus hyperconnectivity relative to healthy controls. Eight BA studies showed hyperconnectivity clusters within the putamen and medio-temporal lobe relative to healthy controls. Altered connectivity in salience or emotion-processing areas may be related to dysregulated affective and cognitive control-related networks, such as deficits in regulating elevated sensitivity to drug-related stimuli. These findings confirm that SUD and BA might be characterized by dysfunctions in specific brain networks, particularly those implicated in the core cognitive and affective functions. These findings might provide insight into the development of neural mechanistic biomarkers for SUD and BA.

L-DOPA modulates activity in the vmPFC, nucleus accumbens, and VTA during threat extinction learning in humans

Learning to be safe is central for adaptive behaviour when threats are no longer present. Detecting the absence of an expected threat is key for threat extinction learning and an essential process for the behavioural treatment of anxiety-related disorders. One possible mechanism underlying extinction learning is a dopaminergic mismatch signal that encodes the absence of an expected threat. Here we show that such a dopamine-related pathway underlies extinction learning in humans. Dopaminergic enhancement via administration of L-DOPA (vs. Placebo) was associated with reduced retention of differential psychophysiological threat responses at later test, which was mediated by activity in the ventromedial prefrontal cortex that was specific to extinction learning. L-DOPA administration enhanced signals at the time-point of an expected, but omitted threat in extinction learning within the nucleus accumbens, which were functionally coupled with the ventral tegmental area and the amygdala. Computational modelling of threat expectancies further revealed prediction error encoding in nucleus accumbens that was reduced when L-DOPA was administered. Our results thereby provide evidence that extinction learning is influenced by L-DOPA and provide a mechanistic perspective to augment extinction learning by dopaminergic enhancement in humans.

Children's Greenness Exposure and IQ-Associated DNA Methylation: A Prospective Cohort Study

Epigenetics is known to be involved in regulatory pathways through which greenness exposure influences child development and health. We aimed to investigate the associations between residential surrounding greenness and DNA methylation changes in children, and further assessed the association between DNA methylation and children's intelligence quotient (IQ) in a prospective cohort study. We identified cytosine-guanine dinucleotide sites (CpGs) associated with cognitive abilities from epigenome- and genome-wide association studies through a systematic literature review for candidate gene analysis. We estimated the residential surrounding greenness at age 2 using a geographic information system. DNA methylation was analyzed from whole blood using the HumanMethylationEPIC array in 59 children at age 2. We analyzed the association between greenness exposure and DNA methylation at age 2 at the selected CpGs using multivariable linear regression. We further investigated the relationship between DNA methylation and children's IQ. We identified 8743 CpGs associated with cognitive ability based on the literature review. Among these CpGs, we found that 25 CpGs were significantly associated with greenness exposure at age 2, including cg26269038 (Bonferroni-corrected p ≤ 0.05) located in the body of SLC6A3, which encodes a dopamine transporter. DNA methylation at cg26269038 at age 2 was significantly associated with children's performance IQ at age 6. Exposure to surrounding greenness was associated with cognitive ability-related DNA methylation changes, which was also associated with children's IQ. Further studies are warranted to clarify the epigenetic pathways linking greenness exposure and neurocognitive function.

Transdiagnostic neuroimaging of reward system phenotypes in ADHD and comorbid disorders

ADHD is a disorder characterized by changes in the reward system and which is highly comorbid with other mental disorders, suggesting common neurobiological pathways. Transdiagnostic neuroimaging findings could help to understand whether a dysregulated reward pathway might be the actual link between ADHD and its comorbidities. We here synthesize ADHD neuroimaging findings on the reward system with findings in obesity, depression, and substance use disorder including their comorbid appearance regarding neuroanatomical features (structural MRI) and activation patterns (resting-state and functional MRI). We focus on findings from monetary-incentive-delay (MID) and delay-discounting (DD) tasks and then review data on striatal connectivity and volumetry. Next, for better understanding of comorbidity in adult ADHD, we discuss these neuroimaging features in ADHD, obesity, depression and substance use disorder and ask whether ADHD heterogeneity and comorbidity are reflected by a common dysregulation in the reward system. Finally, we highlight conceptual issues related to heterogeneous paradigms, different phenotyping, longitudinal prediction and highlight some promising future directions for using striatal reward functioning as a clinical biomarker.

Neural correlates of RDoC-specific cognitive processes in a high-functional autistic patient: a statistically validated case report

The level of functioning of individuals with autism spectrum disorder (ASD) varies widely. To better understand the neurobiological mechanism associated with high-functioning ASD, we studied the rare case of a female patient with an exceptional professional career in the highly competitive academic field of Mathematics. According to the Research Domain Criteria (RDoC) approach, which proposes to describe the basic dimensions of functioning by integrating different levels of information, we conducted four fMRI experiments targeting the (1) social processes domain (Theory of mind (ToM) and face matching), (2) positive valence domain (reward processing), and (3) cognitive domain (N-back). Patient’s data were compared to data of 14 healthy controls (HC). Additionally, we assessed the subjective experience of our case during the experiments. The patient showed increased response times during face matching and achieved a higher total gain in the Reward task, whereas her performance in N-back and ToM was similar to HC. Her brain function differed mainly in the positive valence and cognitive domains. During reward processing, she showed reduced activity in a left-hemispheric frontal network and cortical midline structures but increased connectivity within this network. During the working memory task patients’ brain activity and connectivity in left-hemispheric temporo-frontal regions were elevated. In the ToM task, activity in posterior cingulate cortex and temporo-parietal junction was reduced. We suggest that the high level of functioning in our patient is rather related to the effects in brain connectivity than to local cortical information processing and that subjective report provides a fruitful framework for interpretation.

Rational inattention and tonic dopamine

Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA-the average reward theory and the Bayesian theory in which DA controls precision-have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of 'rational inattention,' which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock-thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.

Individual Differences in Reward-Based Learning Predict Fluid Reasoning Abilities

The ability to reason and problem-solve in novel situations, as measured by the Raven's Advanced Progressive Matrices (RAPM), is highly predictive of both cognitive task performance and real-world outcomes. Here we provide evidence that RAPM performance depends on the ability to reallocate attention in response to self-generated feedback about progress. We propose that such an ability is underpinned by the basal ganglia nuclei, which are critically tied to both reward processing and cognitive control. This hypothesis was implemented in a neurocomputational model of the RAPM task, which was used to derive novel predictions at the behavioral and neural levels. These predictions were then verified in one neuroimaging and two behavioral experiments. Furthermore, an effective connectivity analysis of the neuroimaging data confirmed a role for the basal ganglia in modulating attention. Taken together, these results suggest that individual differences in a neural circuit related to reward processing underpin human fluid reasoning abilities.

Personalized Estimates of Brain Structural Variability in Individuals With Early Psychosis

BACKGROUND

Early psychosis in first-episode psychosis (FEP) and clinical high-risk (CHR) individuals has been associated with alterations in mean regional measures of brain morphology. Examination of variability in brain morphology could assist in quantifying the degree of brain structural heterogeneity in clinical relative to healthy control (HC) samples.

METHODS

Structural magnetic resonance imaging data were obtained from CHR (n = 71), FEP (n = 72), and HC individuals (n = 55). Regional brain variability in cortical thickness (CT), surface area (SA), and subcortical volume (SV) was assessed with the coefficient of variation (CV). Furthermore, the person-based similarity index (PBSI) was employed to quantify the similarity of CT, SA, and SV profile of each individual to others within the same diagnostic group. Normative modeling of the PBSI-CT, PBSI-SA, and PBSI-SV was used to identify CHR and FEP individuals whose scores deviated markedly from those of the healthy individuals.

RESULTS

There was no effect of diagnosis on the CV for any regional measure (P > .38). CHR and FEP individuals differed significantly from the HC group in terms of PBSI-CT (P < .0001), PBSI-SA (P < .0001), and PBSI-SV (P = .01). In the clinical groups, normative modeling identified 32 (22%) individuals with deviant PBSI-CT, 12 (8.4%) with deviant PBSI-SA, and 21 (15%) with deviant PBSI-SV; differences of small effect size indicated that individuals with deviant PBSI scores had lower IQ and higher psychopathology.

CONCLUSIONS

Examination of brain structural variability in early psychosis indicated heterogeneity at the level of individual profiles and encourages further large-scale examination to identify individuals that deviate markedly from normative reference data.

[The IMAGEN cohort: perspectives and problems of longitudinal research]

BACKGROUND

Genetic risk factors for major mental disorders identified in psychiatric research show a substantial overlap. Therefore, it has been suggested that neurobiological research should focus on intermediate phenotypes that reflect shared aspects of different mental disorders due to overlapping genetic effects and environmental factors. Longitudinal studies are required to assess the interaction between genetic variability and modifying environmental factors and to investigate the effects on intermediate phenotypes and (mediated by them) on the expression of individual mental disorders.

OBJECTIVE

Discussion of the possibilities and limitations of longitudinal cohort studies using the IMAGEN study as an example.

MATERIAL AND METHODS

The results of the European IMAGEN study are presented with a focus on addiction.

RESULTS

The longitudinal assessments of the IMAGEN cohort revealed that neuroimaging data indicating a low activation of the dopaminergic reinforcement system detected at the age of 14 years are predictive for increased drug use. In addition to genetic factors, environmental influences such as maternal smoking during pregnancy were correlated with this low activation.

CONCLUSION

Longitudinal neurobiological basic research can validate the effects of candidate genes and reveal relevant environmental factors. Relevant modifiable factors indicated by the IMAGEN study and related datasets include drug use during pregnancy, trauma and other experiences of violence, social disadvantage and exclusion.

The IMAGEN study: a decade of imaging genetics in adolescents

Imaging genetics offers the possibility of detecting associations between genotype and brain structure as well as function, with effect sizes potentially exceeding correlations between genotype and behavior. However, study results are often limited due to small sample sizes and methodological differences, thus reducing the reliability of findings. The IMAGEN cohort with 2000 young adolescents assessed from the age of 14 onwards tries to eliminate some of these limitations by offering a longitudinal approach and sufficient sample size for analyzing gene-environment interactions on brain structure and function. Here, we give a systematic review of IMAGEN publications since the start of the consortium. We then focus on the specific phenotype 'drug use' to illustrate the potential of the IMAGEN approach. We describe findings with respect to frontocortical, limbic and striatal brain volume, functional activation elicited by reward anticipation, behavioral inhibition, and affective faces, and their respective associations with drug intake. In addition to describing its strengths, we also discuss limitations of the IMAGEN study. Because of the longitudinal design and related attrition, analyses are underpowered for (epi-) genome-wide approaches due to the limited sample size. Estimating the generalizability of results requires replications in independent samples. However, such densely phenotyped longitudinal studies are still rare and alternative internal cross-validation methods (e.g., leave-one out, split-half) are also warranted. In conclusion, the IMAGEN cohort is a unique, very well characterized longitudinal sample, which helped to elucidate neurobiological mechanisms involved in complex behavior and offers the possibility to further disentangle genotype × phenotype interactions.

Is There a Correlation Between the Number of Brain Cells and IQ?

Our access to a unique material of postmortem brains obtained from decades of data collection enabled a stereological analysis of the neuron numbers and correlation of results with individual premorbid intelligence quotient (IQ) data. In our sample of 50 brains from men, we find that IQ does not correlate with the number of brain cells in the human neocortex and was only weakly correlated to brain weight. Our stereological examination extended to measures of several other parameters that might be of relevance to intelligence, including numbers of cerebral glial cells (astrocytes, oligodendrocytes, and microglia) and the volume of key areas in the gray and white matter and of the cerebral ventricles, also showing near-zero nonsignificant correlations to IQ.

Neurocomputational mechanisms of adaptive learning in social exchanges

Prior work on prosocial and self-serving behavior in human economic exchanges has shown that counterparts' high social reputations bias striatal reward signals and elicit cooperation, even when such cooperation is disadvantageous. This phenomenon suggests that the human striatum is modulated by the other's social value, which is insensitive to the individual's own choices to cooperate or defect. We tested an alternative hypothesis that, when people learn from their interactions with others, they encode prediction error updates with respect to their own policy. Under this policy update account striatal signals would reflect positive prediction errors when the individual's choices correctly anticipated not only the counterpart's cooperation but also defection. We examined behavior in three samples using reinforcement learning and model-free analyses and performed an fMRI study of striatal learning signals. In order to uncover the dynamics of goal-directed learning, we introduced reversals in the counterpart's behavior and provided counterfactual (would-be) feedback when the individual chose not to engage with the counterpart. Behavioral data and model-derived prediction error maps (in both whole-brain and a priori striatal region of interest analyses) supported the policy update model. Thus, as people continually adjust their rate of cooperation based on experience, their behavior and striatal learning signals reveal a self-centered instrumental process corresponding to reciprocal altruism.

Striatal Dopamine and Reward Prediction Error Signaling in Unmedicated Schizophrenia Patients

Increased striatal dopamine synthesis capacity has consistently been reported in patients with schizophrenia. However, the mechanism translating this into behavior and symptoms remains unclear. It has been proposed that heightened striatal dopamine may blunt dopaminergic reward prediction error signaling during reinforcement learning. In this study, we investigated striatal dopamine synthesis capacity, reward prediction errors, and their association in unmedicated schizophrenia patients (n = 19) and healthy controls (n = 23). They took part in FDOPA-PET and underwent functional magnetic resonance imaging (fMRI) scanning, where they performed a reversal-learning paradigm. The groups were compared regarding dopamine synthesis capacity (Kicer), fMRI neural prediction error signals, and the correlation of both. Patients did not differ from controls with respect to striatal Kicer. Taking into account, comorbid alcohol abuse revealed that patients without such abuse showed elevated Kicer in the associative striatum, while those with abuse did not differ from controls. Comparing all patients to controls, patients performed worse during reversal learning and displayed reduced prediction error signaling in the ventral striatum. In controls, Kicer in the limbic striatum correlated with higher reward prediction error signaling, while there was no significant association in patients. Kicer in the associative striatum correlated with higher positive symptoms and blunted reward prediction error signaling was associated with negative symptoms. Our results suggest a dissociation between striatal subregions and symptom domains, with elevated dopamine synthesis capacity in the associative striatum contributing to positive symptoms while blunted prediction error signaling in the ventral striatum related to negative symptoms.

Dose-dependent effects of estrogen on prediction error related neural activity in the nucleus accumbens of healthy young women

RATIONALE

Whereas the effect of the sex steroid 17-beta-estradiol (E2) on dopaminergic (DA) transmission in the nucleus accumbens (NAc) is well evidenced in female rats, studies in humans are inconsistent. Moreover, linear and inverted u-shaped dose response curves have been observed for E2's effects on hippocampal plasticity, but the shape of dose response curves for E2's effects on the NAc is much less characterized.

OBJECTIVES

Investigation of dose response curves for E2's effects on DA-related neural activity in the human NAc.

METHODS

Placebo or E2 valerate in doses of 2, 4, 6 or 12 mg was orally administered to 125 naturally cycling young women during the low-hormone menstruation phase on two consecutive days using a randomized, double-blinded design. The E2 treatment regimen induced a wide range of E2 levels, from physiological (2- and 4-mg groups; equivalent to cycle peak) to supraphysiological levels (6- and 12-mg groups; equivalent to early pregnancy). This made it possible to study different dose response functions for E2's effects on NAc activity. During E2 peak, participants performed a well-established reversal learning paradigm. We used trial-wise prediction errors (PE) estimated via a computational reinforcement learning model as a proxy for dopaminergic activity. Linear and quadratic regression analyses predicting PE-related NAc activity from salivary E2 levels were calculated.

RESULTS

There was a positive linear relationship between PE-associated NAc activity and salivary E2 increases.

CONCLUSIONS

The randomized, placebo-controlled elevation of E2 levels stimulates NAc activity in the human brain, likely mediated by dopaminergic processes.

Chronic psychosocial stressors are associated with alterations in salience processing and corticostriatal connectivity

Psychosocial stressors including childhood adversity, migration, and living in an urban environment, have been associated with several psychiatric disorders, including psychotic disorders. The neural and psychological mechanisms mediating this relationship remain unclear. In parallel, alterations in corticostriatal connectivity and abnormalities in the processing of salience, are seen in psychotic disorders. Aberrant functioning of these mechanisms secondary to chronic stress exposure, could help explain how common environmental exposures are associated with a diverse range of symptoms. In the current study, we recruited two groups of adults, one with a high degree of exposure to chronic psychosocial stressors (the exposed group, n = 20), and one with minimal exposure (the unexposed group, n = 22). All participants underwent a resting state MRI scan, completed the Aberrant Salience Inventory, and performed a behavioural task - the Salience Attribution Test (SAT). The exposed group showed reduced explicit adaptive salience scores (cohen's d = 0.69, p = 0.03) and increased aberrant salience inventory scores (d = 0.65, p = 0.04). The exposed group also showed increased corticostriatal connectivity between the ventral striatum and brain regions previously implicated in salience processing. Corticostriatal connectivity in these regions negatively correlated with SAT explicit adaptive salience (r = -0.48, p = 0.001), and positively correlated with aberrant salience inventory scores (r = 0.42, p = 0.006). Furthermore, in a mediation analysis there was tentative evidence that differences in striato-cortical connectivity mediated the group differences in salience scores.

Towards a Unifying Cognitive, Neurophysiological, and Computational Neuroscience Account of Schizophrenia

Psychotic experiences may be understood as altered information processing due to aberrant neural computations. A prominent example of such neural computations is the computation of prediction errors (PEs), which signal the difference between expected and experienced events. Among other areas showing PE coding, hippocampal-prefrontal-striatal neurocircuits play a prominent role in information processing. Dysregulation of dopaminergic signaling, often secondary to psychosocial stress, is thought to interfere with the processing of biologically important events (such as reward prediction errors) and result in the aberrant attribution of salience to irrelevant sensory stimuli and internal representations. Bayesian hierarchical predictive coding offers a promising framework for the identification of dysfunctional neurocomputational processes and the development of a mechanistic understanding of psychotic experience. According to this framework, mismatches between prior beliefs encoded at higher levels of the cortical hierarchy and lower-level (sensory) information can also be thought of as PEs, with important consequences for belief updating. Low levels of precision in the representation of prior beliefs relative to sensory data, as well as dysfunctional interactions between prior beliefs and sensory data in an ever-changing environment, have been suggested as a general mechanism underlying psychotic experiences. Translating the promise of the Bayesian hierarchical predictive coding into patient benefit will come from integrating this framework with existing knowledge of the etiology and pathophysiology of psychosis, especially regarding hippocampal-prefrontal-striatal network function and neural mechanisms of information processing and belief updating.

Dopamine dysregulation hypothesis: the common basis for motivational anhedonia in major depressive disorder and schizophrenia?

Abnormalities in reward processing are crucial symptoms of major depressive disorder (MDD) and schizophrenia (SCH). Recent neuroscientific findings regarding MDD have led to conclusions about two different symptoms related to reward processing: motivational and consummatory anhedonia, corresponding, respectively, to impaired motivation to obtain rewards ('wanting'), and diminished satisfaction from consuming them ('liking'). One can ask: which of these is common for MDD and SCH. In our review of the latest neuroscientific studies, we show that MDD and SCH do not share consummatory anhedonia, as SCH patients usually have unaltered liking. Therefore, we investigated whether motivational anhedonia is the common symptom across MDD and SCH. With regard to the similarities and differences between the neural mechanisms of MDD and SCH, here we expand the current knowledge of motivation deficits and present the common underlying mechanism of motivational anhedonia - the dopamine dysregulation hypothesis - stating that any prolonged dysregulation in tonic dopamine signaling that exceeds the given equilibrium can lead to striatal dysfunction and motivational anhedonia. The implications for further research and treatment of MDD and SCH are also discussed.

Computational Phenotyping: Using Models to Understand Individual Differences in Personality, Development, and Mental Illness

This paper reviews progress in the application of computational models to personality, developmental, and clinical neuroscience. We first describe the concept of a computational phenotype, a collection of parameters derived from computational models fit to behavioral and neural data. This approach represents individuals as points in a continuous parameter space, complementing traditional trait and symptom measures. One key advantage of this representation is that it is mechanistic: The parameters have interpretations in terms of cognitive processes, which can be translated into quantitative predictions about future behavior and brain activity. We illustrate with several examples how this approach has led to new scientific insights into individual differences, developmental trajectories, and psychopathology. We then survey some of the challenges that lay ahead.

Dopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia

Distinguishing between meaningful and meaningless sensory information is fundamental to forming accurate representations of the world. Dopamine is thought to play a central role in processing the meaningful information content of observations, which motivates an agent to update their beliefs about the environment. However, direct evidence for dopamine's role in human belief updating is lacking. We addressed this question in healthy volunteers who performed a model-based fMRI task designed to separate the neural processing of meaningful and meaningless sensory information. We modeled participant behavior using a normative Bayesian observer model and used the magnitude of the model-derived belief update following an observation to quantify its meaningful information content. We also acquired PET imaging measures of dopamine function in the same subjects. We show that the magnitude of belief updates about task structure (meaningful information), but not pure sensory surprise (meaningless information), are encoded in midbrain and ventral striatum activity. Using PET we show that the neural encoding of meaningful information is negatively related to dopamine-2/3 receptor availability in the midbrain and dexamphetamine-induced dopamine release capacity in the striatum. Trial-by-trial analysis of task performance indicated that subclinical paranoid ideation is negatively related to behavioral sensitivity to observations carrying meaningful information about the task structure. The findings provide direct evidence implicating dopamine in model-based belief updating in humans and have implications for understating the pathophysiology of psychotic disorders where dopamine function is disrupted.

Epigenetic variance in dopamine D2 receptor: a marker of IQ malleability?

Genetic and environmental factors both contribute to cognitive test performance. A substantial increase in average intelligence test results in the second half of the previous century within one generation is unlikely to be explained by genetic changes. One possible explanation for the strong malleability of cognitive performance measure is that environmental factors modify gene expression via epigenetic mechanisms. Epigenetic factors may help to understand the recent observations of an association between dopamine-dependent encoding of reward prediction errors and cognitive capacity, which was modulated by adverse life events. The possible manifestation of malleable biomarkers contributing to variance in cognitive test performance, and thus possibly contributing to the "missing heritability" between estimates from twin studies and variance explained by genetic markers, is still unclear. Here we show in 1475 healthy adolescents from the IMaging and GENetics (IMAGEN) sample that general IQ (gIQ) is associated with (1) polygenic scores for intelligence, (2) epigenetic modification of DRD2 gene, (3) gray matter density in striatum, and (4) functional striatal activation elicited by temporarily surprising reward-predicting cues. Comparing the relative importance for the prediction of gIQ in an overlapping subsample, our results demonstrate neurobiological correlates of the malleability of gIQ and point to equal importance of genetic variance, epigenetic modification of DRD2 receptor gene, as well as functional striatal activation, known to influence dopamine neurotransmission. Peripheral epigenetic markers are in need of confirmation in the central nervous system and should be tested in longitudinal settings specifically assessing individual and environmental factors that modify epigenetic structure.

Lower dopamine tone in the striatum is associated with higher body mass index

Existing literature suggests that striatal dopamine (DA) tone may be altered in individuals with higher body mass index (BMI), but evidence accrued so far only offers an incomplete view of their relationship. Here, we characterized striatal DA tone using more comprehensive measures within a larger sample than previously reported. In addition, we explored if there was a relationship between striatal DA tone and disinhibited eating. 60 healthy participants underwent a 6-[¹⁸F]fluoro-L-3,4-dihydroxyphenylalanine (¹⁸F-DOPA) positron emission tomography (PET) scan. Disinhibited eating was measured with the Three-Factor Eating Questionnaire on a baseline visit. Individual whole-brain PET parameter estimates, namely ¹⁸F-DOPA influx rate constant (k_occ i.e. DA synthesis capacity), ¹⁸F-DA washout rate (k_loss) and effective distribution volume ratio (EDVR= k_occ/ k_loss), were derived with a reversible-tracer graphical analysis approach. We then computed parameter estimates for three regions-of-interests (ROIs), namely the ventral striatum, putamen and caudate. Overweight/mildly obese individuals had lowered EDVR than normal weight individuals in all three ROIs. The most prominent of these associations, driven by lowered k_occ (r = -.28, p = .035) and heightened k_loss (r = .48, p < .001), was found in the ventral striatum (r = -.46, p < .001). Disinhibition was greater in higher-BMI individuals (r = .31, p = .015), but was unrelated to PET measures and did not explain the relationship between PET measures and BMI. In sum, our findings resonate with the notion that overweight/mildly obese individuals have lower striatal DA tone and suggest new avenues for investigating their underlying mechanisms.

Brain neuroreceptor density and personality traits: towards dimensional biomarkers for psychiatric disorders

Positron emission tomography has, for 30 years, been used in numerous case-control studies searching for hypothesized differences in the density of neuroreceptor or transporter proteins in psychiatric disorders such as schizophrenia and depression. In most cases, the results have not been conclusive. One reason could be the sizeable interindividual variability in biochemical markers, which in twin studies have shown to emanate from both environmental and genetic factors, leading to low statistical power for the detection of group effects. On the other hand, the same interindividual variability has served as an opportunity for correlative studies on the biological underpinning of behaviour. Using this approach, a series of studies has linked markers for the dopamine and serotonin system to personality traits associated with psychiatric conditions. Based on increasing evidence for the view that many psychopathological states represent extremes of a continuum rather than distinct categories, this research strategy may lead to new biological insights about the vulnerability to and pathophysiology of major psychiatric disorders.This article is part of the theme issue 'Diverse perspectives on diversity: multi-disciplinary approaches to taxonomies of individual differences'.

Eye Movements and Cognitive Strategy in a Fluid Intelligence Test: Item Type Analysis

Eye movements help to infer the cognitive strategy that a person uses in fluid intelligence tests. However, intelligence tests demand different relations/rules tokens to be solved, such as rule direction, which is the continuation, variation or overlay of geometric figures in the matrix of the intelligence test. The aim of this study was to understand whether eye movements could predict the outcome of an intelligence test and in the rule item groups. Furthermore, we sought to identify which measure is best for predicting intelligence test scores and to understand if the rule item groups use the same strategy. Accordingly, 34 adults completed a computerized intelligence test with an eye-tracking device. The toggling rate, that is, the number of toggles on each test item equalized by the item latency explained 45% of the variance of the test scores and a significant amount of the rule tokens item groups. The regression analyses also indicated toggling rate as the best measure for predicting the score and that all the rule tokens seem to respect the same strategy. No correlation or difference were found between baseline pupil size and fluid intelligence. Wiener Matrizen-Test 2 was demonstrated to be a good instrument for the purpose of this study. Finally, the implications of these findings for an understanding of cognition are discussed.

The link between schizophrenia and substance use disorder: A unifying hypothesis

Substance use disorders occur commonly in patients with schizophrenia and dramatically worsen their overall clinical course. While the exact mechanisms contributing to substance use in schizophrenia are not known, a number of theories have been put forward to explain the basis of the co-occurrence of these disorders. We propose here a unifying hypothesis that combines recent evidence from epidemiological and genetic association studies with brain imaging and pre-clinical studies to provide an updated formulation regarding the basis of substance use in patients with schizophrenia. We suggest that the genetic determinants of risk for schizophrenia (especially within neural systems that contribute to the risk for both psychosis and addiction) make patients vulnerable to substance use. Since this vulnerability may arise prior to the appearance of psychotic symptoms, an increased use of substances in adolescence may both enhance the risk for developing a later substance use disorder, and also serve as an additional risk factor for the appearance of psychotic symptoms. Future studies that assess brain circuitry in a prospective longitudinal manner during adolescence prior to the appearance of psychotic symptoms could shed further light on the mechanistic underpinnings of these co-occurring disorders while identifying potential points of intervention for these difficult-to-treat co-occurring disorders.

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.

Learning and Choice in Mood Disorders: Searching for the Computational Parameters of Anhedonia

Computational approaches are increasingly being used to model behavioral and neural processes in mood and anxiety disorders. Here we explore the extent to which the parameters of popular learning and decision-making models are implicated in anhedonic symptoms of major depression. We first highlight the parameters of reinforcement learning that have been implicated in anhedonia, focusing, in particular, on the role that choice variability (i.e., "temperature") may play in explaining heterogeneity across previous findings. We then turn to neuroimaging findings implicating attenuated ventral striatum response in anhedonic responses and discuss possible causes of the heterogeneity in the literature. Taken together, the reviewed findings highlight the potential of the computational approach in teasing apart the observed heterogeneity in both behavioral and functional imaging results. Nevertheless, considerable challenges remain, and we conclude with five unresolved questions that seek to address issues highlighted by the reviewed data.

SLC6A3 Is Associated With Relational Aggression in Children

Abstract. Understanding the genetic influence on aggressive behavior in children is one way to understand pathways to the development of aggression in adults. While aggression is likely under some environmental influence, it is also likely under some genetic influence. Overt aggression associates with a variety of genes including dopaminergic and serotonergic genes. Dopaminergic and serotonergic genes are known to be associated with overt aggression. However, little is known regarding the genetic pathways associated with relational aggression. Detecting genetic associates of relational aggression is important to eventually understand pathways to socially aggressive behaviors in children. Therefore, we attempted to determine if relational aggression was also associated with dopaminergic and serotonergic genes. We invited the parents of 327 children to complete a modified version of the MacArthur Health and Behavior Questionnaire (HBQ-P), which has a subscale for relational aggression. We used logistic regression models that predicted relational aggression after controlling for covariates. One genetic predictor was added at a time until there was no model improvement. The covariates were overt aggression scores obtained from the HBQ-P and age. The final (best) model included as a significant predictor of relational aggression one single nucleotide polymorphisms (SNP) on SLC6A3 (rs2617605) and the covariates.

Altered activation of the ventral striatum under performance-related observation in social anxiety disorder

BACKGROUND

Social anxiety disorder (SAD) is characterized by fear of social and performance situations. The consequence of scrutiny by others for the neural processing of performance feedback in SAD is unknown.

METHODS

We used event-related functional magnetic resonance imaging to investigate brain activation to positive, negative, and uninformative performance feedback in patients diagnosed with SAD and age-, gender-, and education-matched healthy control subjects who performed a time estimation task during a social observation condition and a non-social control condition: while either being monitored or unmonitored by a body camera, subjects received performance feedback after performing a time estimation that they could not fully evaluate without external feedback.

RESULTS

We found that brain activation in ventral striatum (VS) and midcingulate cortex was modulated by an interaction of social context and feedback type. SAD patients showed a lack of social-context-dependent variation of feedback processing, while control participants showed an enhancement of brain responses specifically to positive feedback in VS during observation.

CONCLUSIONS

The present findings emphasize the importance of social-context processing in SAD by showing that scrutiny prevents appropriate reward-processing-related signatures in response to positive performances in SAD.

Attenuation of dopamine-modulated prefrontal value signals underlies probabilistic reward learning deficits in old age

Probabilistic reward learning is characterised by individual differences that become acute in aging. This may be due to age-related dopamine (DA) decline affecting neural processing in striatum, prefrontal cortex, or both. We examined this by administering a probabilistic reward learning task to younger and older adults, and combining computational modelling of behaviour, fMRI and PET measurements of DA D1 availability. We found that anticipatory value signals in ventromedial prefrontal cortex (vmPFC) were attenuated in older adults. The strength of this signal predicted performance beyond age and was modulated by D1 availability in nucleus accumbens. These results uncover that a value-anticipation mechanism in vmPFC declines in aging, and that this mechanism is associated with DA D1 receptor availability.

Computational approaches to schizophrenia: A perspective on negative symptoms

Schizophrenia is a heterogeneous spectrum disorder often associated with detrimental negative symptoms. In recent years, computational approaches to psychiatry have attracted growing attention. Negative symptoms have shown some overlap with general cognitive impairments and were also linked to impaired motivational processing in brain circuits implementing reward prediction. In this review, we outline how computational approaches may help to provide a better understanding of negative symptoms in terms of the potentially underlying behavioural and biological mechanisms. First, we describe the idea that negative symptoms could arise from a failure to represent reward expectations to enable flexible behavioural adaptation. It has been proposed that these impairments arise from a failure to use prediction errors to update expectations. Important previous studies focused on processing of so-called model-free prediction errors where learning is determined by past rewards only. However, learning and decision-making arise from multiple cognitive mechanisms functioning simultaneously, and dissecting them via well-designed tasks in conjunction with computational modelling is a promising avenue. Second, we move on to a proof-of-concept example on how generative models of functional imaging data from a cognitive task enable the identification of subgroups of patients mapping on different levels of negative symptoms. Combining the latter approach with behavioural studies regarding learning and decision-making may allow the identification of key behavioural and biological parameters distinctive for different dimensions of negative symptoms versus a general cognitive impairment. We conclude with an outlook on how this computational framework could, at some point, enrich future clinical studies.

Dorsolateral prefrontal cortex contributes to the impaired behavioral adaptation in alcohol dependence

Substance-dependent individuals often lack the ability to adjust decisions flexibly in response to the changes in reward contingencies. Prediction errors (PEs) are thought to mediate flexible decision-making by updating the reward values associated with available actions. In this study, we explored whether the neurobiological correlates of PEs are altered in alcohol dependence. Behavioral, and functional magnetic resonance imaging (fMRI) data were simultaneously acquired from 34 abstinent alcohol-dependent patients (ADP) and 26 healthy controls (HC) during a probabilistic reward-guided decision-making task with dynamically changing reinforcement contingencies. A hierarchical Bayesian inference method was used to fit and compare learning models with different assumptions about the amount of task-related information subjects may have inferred during the experiment. Here, we observed that the best-fitting model was a modified Rescorla-Wagner type model, the “double-update” model, which assumes that subjects infer the knowledge that reward contingencies are anti-correlated, and integrate both actual and hypothetical outcomes into their decisions. Moreover, comparison of the best-fitting model's parameters showed that ADP were less sensitive to punishments compared to HC. Hence, decisions of ADP after punishments were loosely coupled with the expected reward values assigned to them. A correlation analysis between the model-generated PEs and the fMRI data revealed a reduced association between these PEs and the BOLD activity in the dorsolateral prefrontal cortex (DLPFC) of ADP. A hemispheric asymmetry was observed in the DLPFC when positive and negative PE signals were analyzed separately. The right DLPFC activity in ADP showed a reduced correlation with positive PEs. On the other hand, ADP, particularly the patients with high dependence severity, recruited the left DLPFC to a lesser extent than HC for processing negative PE signals. These results suggest that the DLPFC, which has been linked to adaptive control of action selection, may play an important role in cognitive inflexibility observed in alcohol dependence when reinforcement contingencies change. Particularly, the left DLPFC may contribute to this impaired behavioral adaptation, possibly by impeding the extinction of the actions that no longer lead to a reward.

•

Alcohol-dependent patients (ADP) had difficulty adapting to the reversals.

•

The impaired adaptation was associated with a decrease in punishment sensitivity.

•

The dorsolateral prefrontal cortex (DLPFC) of ADP failed to track prediction errors.

•

A reduced tracking of the negative prediction error was present in the left DLPFC.

•

The clinical severity of dependence was correlated with abnormal DLPFC activity.

Regulation of dopaminergic function: an [18F]-DOPA PET apomorphine challenge study in humans

Dopaminergic function has a key role in normal brain function, dopaminergic dysfunction being implicated in numerous neuropsychiatric disorders. Animal studies show that dopaminergic stimulation regulates dopaminergic function, but it is not known whether this exists in humans. In the first study (study 1), we measured dopamine synthesis capacity (indexed as Kicer) to identify the relationship between baseline and change in Kicer under resting conditions for comparison with effects of dopaminergic stimulation. In the second study (study 2), we used a within-subjects design to test effects of dopaminergic stimulation on dopamine synthesis capacity. In study 1, eight volunteers received two 18F-DOPA scans on separate days, both at rest. In study 2, 12 healthy male volunteers received two 18F-DOPA positron emission tomographic (PET) scans after treatment with either the dopamine partial agonist apomorphine (0.03 or 0.005 mg kg-1) or placebo. In study 1, no significant correlation was found between baseline and change in dopamine synthesis capacity between scans (r=-0.57, n=8, P=0.17, two-tailed). In study 2, a significant negative correlation was found between baseline dopamine synthesis capacity and percentage change in dopamine synthesis capacity after apomorphine challenge (r=-0.71, n=12, P=0.01, two-tailed). This correlation was significantly different (P<0.01) from the correlation between baseline and change in dopamine synthesis capacity under unstimulated conditions. One-way repeated-measures analysis of variance showed a significant group (study 1/study 2) × time interaction (F(1,18)=11.5, P=0.003). Our findings suggest that regulation of dopamine synthesis capacity by apomorphine depends on baseline dopamine function, consistent with dopamine stimulation stabilizing dopaminergic function. Loss of this autoregulation may contribute to dopaminergic dysfunction in brain disorders such as schizophrenia, substance dependence, and Parkinson's disease.

Striatal dopamine, reward, and decision making in schizophrenia

Elevated striatal dopamine function is one of the best-established findings in schizophrenia. In this review, we discuss causes and consequences of this striata! dopamine alteration. We first summarize earlier findings regarding striatal reward processing and anticipation using functional neuroimaging. Secondly, we present a series of recent studies that are exemplary for a particular research approach: a combination of theory-driven reinforcement learning and decision-making tasks in combination with computational modeling and functional neuroimaging. We discuss why this approach represents a promising tool to understand underlying mechanisms of symptom dimensions by dissecting the contribution of multiple behavioral control systems working in parallel. We also discuss how it can advance our understanding of the neurobiological implementation of such functions. Thirdly, we review evidence regarding the topography of dopamine dysfunction within the striatum. Finally, we present conclusions and outline important aspects to be considered in future studies.

An Integrative Perspective on the Role of Dopamine in Schizophrenia

We propose that schizophrenia involves a combination of decreased phasic dopamine responses for relevant stimuli and increased spontaneous phasic dopamine release. Using insights from computational reinforcement-learning models and basic-science studies of the dopamine system, we show that each of these two disturbances contributes to a specific symptom domain and explains a large set of experimental findings associated with that domain. Reduced phasic responses for relevant stimuli help to explain negative symptoms and provide a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with negative symptoms: reduced learning from rewards; blunted activation of the ventral striatum, midbrain, and other limbic regions for rewards and positive prediction errors; blunted activation of the ventral striatum during reward anticipation; blunted autonomic responding for relevant stimuli; blunted neural activation for aversive outcomes and aversive prediction errors; reduced willingness to expend effort for rewards; and psychomotor slowing. Increased spontaneous phasic dopamine release helps to explain positive symptoms and provides a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with positive symptoms: aberrant learning for neutral cues (assessed with behavioral and autonomic responses), and aberrant, increased activation of the ventral striatum, midbrain, and other limbic regions for neutral cues, neutral outcomes, and neutral prediction errors. Taken together, then, these two disturbances explain many findings in schizophrenia. We review evidence supporting their co-occurrence and consider their differential implications for the treatment of positive and negative symptoms.

Reversal learning strategy in adolescence is associated with prefrontal cortex activation

Adolescence is a critical maturation period for human cognitive control and executive function. In this study, a large sample of adolescents (n = 85) performed a reversal learning task during functional magnetic resonance imaging. We analyzed behavioral data using a reinforcement learning model to provide individually fitted parameters and imaging data with regard to reward prediction errors (PE). Following a model-based approach, we formed two groups depending on whether individuals tended to update expectations predominantly for the chosen stimulus or also for the unchosen one. These groups significantly differed in their problem behavior score obtained using the child behavior checklist (CBCL) and in a measure of their developmental stage. Imaging results showed that dorsolateral striatal areas covaried with PE. Participants who relied less on learning based on task structure showed less prefrontal activation compared with participants who relied more on task structure. An exploratory analysis revealed that PE-related activity was associated with pubertal development in prefrontal areas, insula and anterior cingulate. These findings support the hypothesis that the prefrontal cortex is implicated in mediating flexible goal-directed behavioral control.

Models of neuromodulation for computational psychiatry

Psychiatry faces fundamental challenges: based on a syndrome-based nosology, it presently lacks clinical tests to infer on disease processes that cause symptoms of individual patients and must resort to trial-and-error treatment strategies. These challenges have fueled the recent emergence of a novel field-computational psychiatry-that strives for mathematical models of disease processes at physiological and computational (information processing) levels. This review is motivated by one particular goal of computational psychiatry: the development of 'computational assays' that can be applied to behavioral or neuroimaging data from individual patients and support differential diagnosis and guiding patient-specific treatment. Because the majority of available pharmacotherapeutic approaches in psychiatry target neuromodulatory transmitters, models that infer (patho)physiological and (patho)computational actions of different neuromodulatory transmitters are of central interest for computational psychiatry. This article reviews the (many) outstanding questions on the computational roles of neuromodulators (dopamine, acetylcholine, serotonin, and noradrenaline), outlines available evidence, and discusses promises and pitfalls in translating these findings to clinical applications. WIREs Cogn Sci 2017, 8:e1420. doi: 10.1002/wcs.1420 For further resources related to this article, please visit the WIREs website.

The Feedback-related Negativity Codes Components of Abstract Inference during Reward-based Decision-making

Behavioral control is influenced not only by learning from the choices made and the rewards obtained but also by “what might have happened,” that is, inference about unchosen options and their fictive outcomes. Substantial progress has been made in understanding the neural signatures of direct learning from choices that are actually made and their associated rewards via reward prediction errors (RPEs). However, electrophysiological correlates of abstract inference in decision-making are less clear. One seminal theory suggests that the so-called feedback-related negativity (FRN), an ERP peaking 200–300 msec after a feedback stimulus at frontocentral sites of the scalp, codes RPEs. Hitherto, the FRN has been predominantly related to a so-called “model-free” RPE: The difference between the observed outcome and what had been expected. Here, by means of computational modeling of choice behavior, we show that individuals employ abstract, “double-update” inference on the task structure by concurrently tracking values of chosen stimuli (associated with observed outcomes) and unchosen stimuli (linked to fictive outcomes). In a parametric analysis, model-free RPEs as well as their modification because of abstract inference were regressed against single-trial FRN amplitudes. We demonstrate that components related to abstract inference uniquely explain variance in the FRN beyond model-free RPEs. These findings advance our understanding of the FRN and its role in behavioral adaptation. This might further the investigation of disturbed abstract inference, as proposed, for example, for psychiatric disorders, and its underlying neural correlates.

Neurobiological Models of Self-Disorders in Early Schizophrenia

Self-disorders (SDs) (from the German Ichstörungen) are alterations of the first-person perspective, long associated with schizophrenia, particularly in early phases. Although psychopathological features of SDs continue to be studied, their neurobiological underpinnings are unknown. This makes it difficult to integrate SDs into contemporary models of psychosis. The present review aims to address this issue, starting from an historical excursus revealing an interconnection between neuroscientific models and the origin of the psychopathological concept of SDs. Subsequently, the more recent neurobiological models related to SDs are discussed, particularly with respect to the onset of schizophrenia.

Does Feedback-Related Brain Response during Reinforcement Learning Predict Socio-motivational (In-)dependence in Adolescence?

This multi-methodological study applied functional magnetic resonance imaging to investigate neural activation in a group of adolescent students (N = 88) during a probabilistic reinforcement learning task. We related patterns of emerging brain activity and individual learning rates to socio-motivational (in-)dependence manifested in four different motivation types (MTs): (1) peer-dependent MT, (2) teacher-dependent MT, (3) peer-and-teacher-dependent MT, (4) peer-and-teacher-independent MT. A multinomial regression analysis revealed that the individual learning rate predicts students’ membership to the independent MT, or the peer-and-teacher-dependent MT. Additionally, the striatum, a brain region associated with behavioral adaptation and flexibility, showed increased learning-related activation in students with motivational independence. Moreover, the prefrontal cortex, which is involved in behavioral control, was more active in students of the peer-and-teacher-dependent MT. Overall, this study offers new insights into the interplay of motivation and learning with (1) a focus on inter-individual differences in the role of peers and teachers as source of students’ individual motivation and (2) its potential neurobiological basis.