The medial orbitofrontal cortex encodes a general unsigned value signal during anticipation of both appetitive and aversive events

Neural Representation of Valenced and Generic Probability and Uncertainty

Representing the probability and uncertainty of outcomes facilitates adaptive behavior by allowing organisms to prepare in advance and devote attention to relevant events. Probability and uncertainty are often studied only for valenced (appetitive or aversive) outcomes, raising the question of whether the identified neural machinery also processes the probability and uncertainty of motivationally neutral outcomes. Here, we aimed to dissociate valenced from valence-independent (i.e., generic) probability (p; maximum at p = 1) and uncertainty (maximum at p = 0.5) signals using human neuroimaging. In a Pavlovian task (n = 41; 19 females), different cues predicted appetitive, aversive, or neutral liquids with different probabilities (p = 0, p = 0.5, p = 1). Cue-elicited motor responses accelerated, and pupil sizes increased primarily for cues that predicted valenced liquids with higher probability. For neutral liquids, uncertainty rather than probability tended to accelerate cue-induced responding and decrease pupil size. At the neural level, generic uncertainty signals were limited to the occipital cortex, while generic probability also activated the anterior ventromedial prefrontal cortex. These generic probability and uncertainty signals contrasted with cue-induced responses that only encoded the probability and uncertainty of valenced liquids in medial prefrontal, insular, and occipital cortices. Our findings show a behavioral and neural dissociation of generic and valenced signals. Thus, some parts of the brain keep track of motivational charge while others do not, highlighting the need and usefulness of characterizing the exact nature of learned representations.

How adverse childhood experiences get under the skin: A systematic review, integration and methodological discussion on threat and reward learning mechanisms

Adverse childhood experiences (ACEs) are a major risk factor for the development of multiple psychopathological conditions, but the mechanisms underlying this link are poorly understood. Associative learning encompasses key mechanisms through which individuals learn to link important environmental inputs to emotional and behavioral responses. ACEs may impact the normative maturation of associative learning processes, resulting in their enduring maladaptive expression manifesting in psychopathology. In this review, we lay out a systematic and methodological overview and integration of the available evidence of the proposed association between ACEs and threat and reward learning processes. We summarize results from a systematic literature search (following PRISMA guidelines) which yielded a total of 81 articles (threat: n=38, reward: n=43). Across the threat and reward learning fields, behaviorally, we observed a converging pattern of aberrant learning in individuals with a history of ACEs, independent of other sample characteristics, specific ACE types, and outcome measures. Specifically, blunted threat learning was reflected in reduced discrimination between threat and safety cues, primarily driven by diminished responding to conditioned threat cues. Furthermore, attenuated reward learning manifested in reduced accuracy and learning rate in tasks involving acquisition of reward contingencies. Importantly, this pattern emerged despite substantial heterogeneity in ACE assessment and operationalization across both fields. We conclude that blunted threat and reward learning may represent a mechanistic route by which ACEs may become physiologically and neurobiologically embedded and ultimately confer greater risk for psychopathology. In closing, we discuss potentially fruitful future directions for the research field, including methodological and ACE assessment considerations.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

Theta-Burst Stimulation Combined With Virtual-Reality Reconsolidation Intervention for Methamphetamine Use Disorder: Study Protocol for a Randomized-Controlled Trial

Background

Craving associated with drug-related memory is one of the key factors that induce the relapse of methamphetamine (MA). Disruption or modulation of the reconsolidation of drug-related memory may serve as an option for clinical treatment of MA addiction. This protocol proposes to use virtual reality (VR) to retrieve drug-associated memory and then use transcranial magnetic stimulation (TMS) at the neural circuit that encodes the reward value of drug cues to provide a non-invasive intervention during reconsolidation. We aim to evaluate the effectiveness of TMS treatment after VR retrieval on the reduction of cue reactivity and craving of MA.

Methods

This is a randomized, double-blind, sham-controlled, parallel group trial, targeting participants with MA use disorder aged from 18 to 45 years old. Forty-five eligible volunteers in Shanxi Drug Rehabilitation Center will be recruited and be randomly allocated into three parallel groups, receiving either 1) MA-related cues retrieval in VR combined with active TMS (MA VR scene + TBS) or 2) sham TMS (MA VR scene + sham TBS), or 3) neutral cues retrieval in VR combined with active TMS (neutral VR scene + TBS). Two sessions of post-VR-retrieval TBS will be scheduled on two separate days within 1 week. The primary outcome will detect the memory-related activity by the electroencephalography (EEG) reactivity to drug cues in VR scenes. Secondary outcomes are the self-reported MA craving in VR scene, the physiological parameter (cue-induced heart rate) and the scores of psychological questionnaires including anxiety, depression, and mood. All primary and secondary outcomes will be assessed at baseline, 1-week, and 1-month post-intervention. Assessments will be compared between the groups of 1) MA VR scene + TBS, 2) MA VR scene + sham TBS and 3) neutral VR scene + TBS.

Discussion

This will be the first study to examine whether the TMS modulation after VR retrieval can reduce self-reported craving and drug-related cue reactivity. It will promote the understanding of the neural circuit mechanism of the reconsolidation-based intervention and provide an effective treatment for MA use disorder patients.

Clinical Trial Registration

[Chinese Clinical Trial Registry], identifier [ChiCTR1900026902]. Registered on 26 October 2019.

Children's value-based decision making

To effectively navigate their environments, infants and children learn how to recognize events predict salient outcomes, such as rewards or punishments. Relatively little is known about how children acquire this ability to attach value to the stimuli they encounter. Studies often examine children’s ability to learn about rewards and threats using either classical conditioning or behavioral choice paradigms. Here, we assess both approaches and find that they yield different outcomes in terms of which individuals had efficiently learned the value of information presented to them. The findings offer new insights into understanding how to assess different facets of value learning in children.

Approach motivation and loneliness: Individual differences and parasympathetic activity

Loneliness, or perceived social isolation, is linked to a number of negative long‐term effects on both mental and physical health. However, how an individual responds to feeling lonely may influence their risk for later negative health outcomes. Here, we sought to clarify what influences variability in individuals' motivated responses to loneliness. Specifically, we assessed whether resting parasympathetic activity, a physiological marker linked to flexible adaptation, facilitates increased approach‐oriented behaviors. Seventy‐four adult participants underwent a conditioning paradigm assessing how they approach and avoid rewards and threats. Individuals with higher levels of loneliness and high resting parasympathetic activity were more likely to demonstrate approach behaviors. We discuss these findings in terms of the role resting parasympathetic activity may play in facilitating adaptive responses to feeling socially isolated.

How an individual responds to feeling lonely may influence their risk for later negative health outcomes. The current study provides new insight into what may influence variability in responses to feeling lonely. Individuals with higher resting parasympathetic activity and high levels of loneliness demonstrate increased approach‐oriented behaviors. This suggests loneliness increases approach motivations but only in the presence of other markers of adaptive responding, like high resting parasympathetic activity.

Early life stress and perceived social isolation influence how children use value information to guide behavior

Learning the value of environmental signals and using that information to guide behavior is critical for survival. Stress in childhood may influence these processes, but how it does so is still unclear. This study examined how stressful event exposures and perceived social isolation affect the ability to learn value signals and use that information in 72 children (8–9 years; 29 girls; 65.3% White). Stressful event exposures and perceived social isolation did not influence how children learned value information. But, children with high stressful event exposures and perceived social isolation were worse at using that information. These data suggest alterations in how value information is used, rather than learned, may be one mechanism linking early experiences to later behaviors.

Cognitive and hormonal regulation of appetite for food presented in the olfactory and visual modalities

The ability to regulate appetite is essential to avoid food over-consumption. The desire for a particular food can be triggered by its odor before it is even seen. Using fMRI, we identify the neural systems modulated by cognitive regulation when experiencing appetizing food stimuli presented in both olfactory and visual modalities, while being hungry. Regulatory instruction modulated bids for food items and inhalation patterns. Distinct brain regions were observed for up and down appetite-regulation, respectively the dorsomedial prefrontal cortex (dmPFC) and dorsolateral PFC. Food valuation engaged the ventromedial PFC and bilateral striatum. Furthermore, we identified a neurobiological marker for successful appetite upregulation. Individuals with higher blood levels of ghrelin were better at exercising up-regulation, and engaged the dmPFC more. These findings characterize the neural circuitry regulating food consumption within the healthy population and highlight how cognitive regulation modulates olfactomotor measures of olfaction.

Task- and Rest-based Functional Brain Connectivity in Food-related Reward Processes among Healthy Adolescents

It is known that the nucleus accumbens, orbitofrontal cortex and insula play a role in food-related reward processes. Although their interconnectedness would be an ideal topic for understanding food intake mechanisms, it nevertheless remains unclear especially in adolescent. Therefore, this study aims to investigate the effect of hunger on functional connectivity in healthy adolescents using task- and rest-based imaging. Fifteen participants underwent two MRI sessions, pre-lunch (hunger) and post-lunch (satiety), including food cue task and resting-state. During task- and rest-based imaging, functional connectivity was greater when hungry as opposed to satiated between the right posterior insula/nucleus accumbens, suggesting involvement of salient interoceptive stimuli signals. During task-based imaging, an increase was observed in functional connectivity when hungry as opposed to satiated between the medial and lateral orbitofrontal cortex which contributes to the perception of food deprivation as a frustration. A decrease was identified when hungry as opposed to satiated in functional connectivity in the right anterior orbitofrontal/accumbens and posterior insula/medial orbitofrontal cortices reflecting suppression of the affective and sensorial information. Conversely, functional connectivity was increased during aversive stimuli between the right medial orbitofrontal cortex and right posterior insula when hungry as opposed to satiated. This suggests that the value of valence could occur in the shift in connectivity between these two regions. In addition, during rest-based imaging, a left-sided lateralization was reported (accumbens/lateral orbitofrontal and accumbens/posterior insula) when hungry as opposed to satiated which may represent changes in internal state due to focus on the benefit of an upcoming meal.

Cortical plasticity elicited by acoustically cued monetary losses: an ERP study

Both human and animal studies have demonstrated remarkable findings of experience-induced plasticity in the cortex. Here, we investigated whether the widely used monetary incentive delay (MID) task changes the neural processing of incentive cues that code expected monetary outcomes. We used a novel auditory version of the MID task, where participants responded to acoustic cues that coded expected monetary losses. To investigate task-induced brain plasticity, we presented incentive cues as deviants during passive oddball tasks before and after two sessions of the MID task. During the oddball task, we recorded the mismatch-related negativity (MMN) as an index of cortical plasticity. We found that two sessions of the MID task evoked a significant enhancement of MMN for incentive cues that predicted large monetary losses, specifically when monetary cue discrimination was essential for maximising monetary outcomes. The task-induced plasticity correlated with the learning-related neural activity recorded during the MID task. Thus, our results confirm that the processing of (loss)incentive auditory cues is dynamically modulated by previously learned monetary outcomes.

Decomposing the neural pathways in a simple, value-based choice

Understanding the neural implementation of value-based choice has been an important focus of neuroscience for several decades. Although a consensus has emerged regarding the brain regions involved, including ventromedial prefrontal cortex (vmPFC), posterior parietal cortex (PPC), and the ventral striatum (vSTR), the multifaceted nature of decision processes is one cause of persistent debate regarding organization of the value-based choice network. In the current study, we isolate neural activity related to valuation and choice selection using a gambling task where expected gains and losses are dissociated from choice outcomes. We apply multilevel mediation analysis to formally test whether brain regions identified as part of the value-based choice network mediate between perceptions of expected value and choice to accept or decline a gamble. Our approach additionally makes predictions regarding interregional relationships to elucidate the chain of processing events within the value-based decision network. Finally, we use dynamic causal modelling (DCM) to compare plausible models of interregional relationships in value-based choice. We observe that activity in vmPFC does not predict take/pass choices, but rather is highly associated with outcome evaluation. By contrast, both PPC and bilateral vSTR (bilaterally) mediate the relationship between expected value and choice. Interregional mediation analyses reveal that vSTR fully mediates between PPC and choice, and this is supported by DCM. Together these results suggest that vSTR, and not vmPFC nor PPC, functions as an important driver of choice.

Neurocognitive Underpinnings of Aggressive Predation in Economic Contests

Competitions are part and parcel of daily life and require people to invest time and energy to gain advantage over others and to avoid (the risk of) falling behind. Whereas the behavioral mechanisms underlying competition are well documented, its neurocognitive underpinnings remain poorly understood. We addressed this using neuroimaging and computational modeling of individual investment decisions aimed at exploiting one's counterpart ("attack") or at protecting against exploitation by one's counterpart ("defense"). Analyses revealed that during attack relative to defense (i) individuals invest less and are less successful; (ii) computations of expected reward are strategically more sophisticated (reasoning level k = 4 vs. k = 3 during defense); (iii) ventral striatum activity tracks reward prediction errors; (iv) risk prediction errors were not correlated with neural activity in either ROI or whole-brain analyses; and (v) successful exploitation correlated with neural activity in the bilateral ventral striatum, left OFC, left anterior insula, left TPJ, and lateral occipital cortex. We conclude that, in economic contests, coming out ahead (vs. not falling behind) involves sophisticated strategic reasoning that engages both reward and value computation areas and areas associated with theory of mind.

Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals

Individuals from across the psychosis spectrum display impairments in reinforcement learning. In some individuals, these deficits may result from aberrations in reward prediction error (RPE) signaling, conveyed by dopaminergic projections to the ventral striatum (VS). However, there is mounting evidence that VS RPE signals are relatively intact in medicated people with schizophrenia (PSZ). We hypothesized that, in PSZ, reinforcement learning deficits often are not related to RPE signaling per se but rather their impact on learning and behavior (i.e., learning rate modulation), due to dysfunction in anterior cingulate and dorsomedial prefrontal cortex (dmPFC). Twenty-six PSZ and 23 healthy volunteers completed a probabilistic reinforcement learning paradigm with occasional, sudden, shifts in contingencies. Using computational modeling, we found evidence of an impairment in trial-wise learning rate modulation (α) in PSZ before and after a reinforcement contingency shift, expressed most in PSZ with more severe motivational deficits. In a subsample of 22 PSZ and 22 healthy volunteers, we found little evidence for between-group differences in VS RPE and dmPFC learning rate signals, as measured with fMRI. However, a follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. These findings point to an impairment in learning rate modulation in PSZ, leading to a reduced ability to adjust task behavior in response to unexpected outcomes. At the level of the brain, learning rate modulation deficits may be associated with decreased involvement of the dmPFC within a greater RL network.

Medial Orbitofrontal Cortex, Dorsolateral Prefrontal Cortex, and Hippocampus Differentially Represent the Event Saliency

Two primary functions attributed to the hippocampus and prefrontal cortex (PFC) network are retaining the temporal and spatial associations of events and detecting deviant events. It is unclear, however, how these two functions converge into one mechanism. Here, we tested whether increased activity with perceiving salient events is a deviant detection signal or contains information about the event associations by reflecting the magnitude of deviance (i.e., event saliency). We also tested how the deviant detection signal is affected by the degree of anticipation. We studied regional neural activity when people watched a movie that had varying saliency of a novel or an anticipated flow of salient events. Using intracranial electroencephalography from 10 patients, we observed that high-frequency activity (50-150 Hz) in the hippocampus, dorsolateral PFC, and medial OFC tracked event saliency. We also observed that medial OFC activity was stronger when the salient events were anticipated than when they were novel. These results suggest that dorsolateral PFC and medial OFC, as well as the hippocampus, signify the saliency magnitude of events, reflecting the hierarchical structure of event associations.

An fNIRS-based investigation of visual merchandising displays for fashion stores

This paper investigates a brain-based approach for visual merchandising display (VMD) in fashion stores. In marketing, VMD has become a research topic of interest. However, VMD research using brain activation information is rare. We examine the hemodynamic responses (HRs) in the prefrontal cortex (PFC) using functional near-infrared spectroscopy (fNIRS) while positive/negative displays of four stores (menswear, womenswear, underwear, and sportswear) are shown to 20 subjects. As features for classifying the HRs, the mean, variance, peak, skewness, kurtosis, t-value, and slope of the signals for a 20-sec time window for the activated channels are analyzed. Linear discriminant analysis is used for classifying two-class (positive and negative displays) and four-class (four fashion stores) models. PFC brain activation maps based on t-values depicting the data from the 16 channels are provided. In the two-class classification, the underwear store had the highest average classification result of 67.04%, whereas the menswear store had the lowest value of 64.15%. Men's classification accuracy for the underwear stores with positive and negative displays was 71.38%, whereas the highest classification accuracy obtained by women for womenswear stores was 73%. The average accuracy over the 20 subjects for positive displays was 50.68%, while that of negative displays was 51.07%. Therefore, these findings suggest that human brain activation is involved in the evaluation of the fashion store displays. It is concluded that fNIRS can be used as a brain-based tool in the evaluation of fashion stores in a daily life environment.

Spatiotemporal dynamics of reward and punishment effects induced by associative learning

While reward associative learning has been studied extensively across different species, punishment avoidance learning has received far less attention. Of particular interest is how the two types of learning change perceptual processing of the learned stimuli. We designed a task that required participants to learn the association of emotionally neutral images with reward, punishment, and no incentive value outcomes through trial-and-error. During learning, participants received monetary reward, neutral outcomes or avoided punishment by correctly identifying corresponding images. Results showed an early bias in favor of learning reward associations, in the form of higher accuracy and fewer trials needed to reach learning criterion. We subsequently assessed electrophysiological learning effects with a task in which participants viewed the stimuli with no feedback or reinforcement. Critically, we found modulation of two early event-related potential components for reward images: the frontocentral P2 (170-230 ms) and the anterior N2/Early Anterior Positivity (N2/EAP; 210-310 ms). We suggest that reward associations may change stimuli detection and incentive salience as indexed by P2 and N2/EAP. We also reported, on an exploratory basis, a late negativity with frontopolar distribution enhanced by punishment images.

Differences in BOLD responses in brain reward network reflect the tendency to assimilate a surprising flavor stimulus to an expected stimulus

External information can modify the subjective value of a tasted stimulus, but little is known about neural mechanisms underlying these behavioral modifications. This study used flavored drinks to produce variable degrees of discrepancy between expected and received flavor. During a learning session, 43 healthy young men learned 4 symbol-flavor associations. In a separate session, associations were presented again during an fMRI scan, but half of the trials introduced discrepancy with previously learned associations. Liking ratings of drinks were collected and were analyzed using a linear model to define the degree to which discrepant symbols affected liking ratings of the subjects during the fMRI session. Based on these results, a GLM analysis of fMRI data was conducted to determine neural correlates of observed behavior. Groups of subjects were composed based on their behavior in response to discrepant symbols, and comparison of brain activity between groups showed that activation in the PCC and the caudate nucleus was more potent in those subjects in which liking was not affected by discrepant symbols. These activations were not found in subjects who assimilated unexpected flavors to flavors preceeded by discrepant symbols. Instead, these subjects showed differences in the activity in the parietal operculum. The activity of reward network appears to be related to assimilation of received flavor to expected flavor in response to symbol-flavor discrepancy.

Impaired Expected Value Computations Coupled With Overreliance on Stimulus-Response Learning in Schizophrenia

BACKGROUND

While many have emphasized impaired reward prediction error signaling in schizophrenia, multiple studies suggest that some decision-making deficits may arise from overreliance on stimulus-response systems together with a compromised ability to represent expected value. Guided by computational frameworks, we formulated and tested two scenarios in which maladaptive representations of expected value should be most evident, thereby delineating conditions that may evoke decision-making impairments in schizophrenia.

METHODS

In a modified reinforcement learning paradigm, 42 medicated people with schizophrenia and 36 healthy volunteers learned to select the most frequently rewarded option in a 75-25 pair: once when presented with a more deterministic (90-10) pair and once when presented with a more probabilistic (60-40) pair. Novel and old combinations of choice options were presented in a subsequent transfer phase. Computational modeling was employed to elucidate contributions from stimulus-response systems (actor-critic) and expected value (Q-learning).

RESULTS

People with schizophrenia showed robust performance impairments with increasing value difference between two competing options, which strongly correlated with decreased contributions from expected value-based learning (Q-learning). Moreover, a subtle yet consistent contextual choice bias for the probabilistic 75 option was present in people with schizophrenia, which could be accounted for by a context-dependent reward prediction error in the actor-critic.

CONCLUSIONS

We provide evidence that decision-making impairments in schizophrenia increase monotonically with demands placed on expected value computations. A contextual choice bias is consistent with overreliance on stimulus-response learning, which may signify a deficit secondary to the maladaptive representation of expected value. These results shed new light on conditions under which decision-making impairments may arise.

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.

Learned Value Shapes Responses to Objects in Frontal and Ventral Stream Networks in Macaque Monkeys

We have an incomplete picture of how the brain links object representations to reward value, and how this information is stored and later retrieved. The orbitofrontal cortex (OFC), medial frontal cortex (MFC), and ventrolateral prefrontal cortex (VLPFC), together with the amygdala, are thought to play key roles in these processes. There is an apparent discrepancy, however, regarding frontal areas thought to encode value in macaque monkeys versus humans. To address this issue, we used fMRI in macaque monkeys to localize brain areas encoding recently learned image values. Each week, monkeys learned to associate images of novel objects with a high or low probability of water reward. Areas responding to the value of recently learned reward-predictive images included MFC area 10 m/32, VLPFC area 12, and inferior temporal visual cortex (IT). The amygdala and OFC, each thought to be involved in value encoding, showed little such effect. Instead, these 2 areas primarily responded to visual stimulation and reward receipt, respectively. Strong image value encoding in monkey MFC compared with OFC is surprising, but agrees with results from human imaging studies. Our findings demonstrate the importance of VLPFC, MFC, and IT in representing the values of recently learned visual images.

Neural correlates of subjective CS/UCS association in appetitive conditioning

Explicit knowledge of conditioned stimulus (CS)/unconditioned stimulus (UCS) associations is proposed as important factor in classical conditioning. However, while previous studies have focused on its roles in fear conditioning, it has been neglected in the context of appetitive conditioning. The present functional magnetic resonance study aimed to investigate neural activation and functional connectivity linked to subjective CS/UCS association in appetitive conditioning. In total, 85 subjects participated in an appetitive acquisition procedure in which a neutral stimulus (CS+) was paired with a monetary reward, while another neutral stimulus (CS-) was never paired with the reward. Directly afterwards, subjective CS/UCS association was assessed by measuring the extent to which the CS+ was thought to be associated with the UCS compared to the CS-. Close relationships were established between subjective CS/UCS association and activations in the primary visual cortex (V1) during the early phase of conditioning and in the striatum during the late conditioning phase. In addition, we observed inverse relationships between subjective CS/UCS association and both V1/ventromedial prefrontal cortex (vmPFC) and striatal/vmPFC connectivity. The results suggest the involvement of decoupling vmPFC connectivity in reward learning in general and the roles of attentional processes in the formation of the subjective CS/UCS association during the early phase and reward prediction during the late phase of appetitive conditioning.

Development of White Matter Microstructure and Intrinsic Functional Connectivity Between the Amygdala and Ventromedial Prefrontal Cortex: Associations With Anxiety and Depression

BACKGROUND

Connectivity between the amygdala and ventromedial prefrontal cortex (vmPFC) is compromised in multiple psychiatric disorders, many of which emerge during adolescence. To identify to what extent the deviations in amygdala-vmPFC maturation contribute to the onset of psychiatric disorders, it is essential to characterize amygdala-vmPFC connectivity changes during typical development.

METHODS

Using an accelerated cohort longitudinal design (1-3 time points, 10-25 years old, n = 246), we characterized developmental changes of the amygdala-vmPFC subregion functional and structural connectivity using resting-state functional magnetic resonance imaging and diffusion-weighted imaging.

RESULTS

Functional connectivity between the centromedial amygdala and rostral anterior cingulate cortex (rACC), anterior vmPFC, and subgenual cingulate significantly decreased from late childhood to early adulthood in male and female subjects. Age-associated decreases were also observed between the basolateral amygdala and the rACC. Importantly, these findings were replicated in a separate cohort (10-22 years old, n = 327). Similarly, structural connectivity, as measured by quantitative anisotropy, significantly decreased with age in the same regions. Functional connectivity between the centromedial amygdala and the rACC was associated with structural connectivity in these same regions during early adulthood (22-25 years old). Finally, a novel time-varying coefficient analysis showed that increased centromedial amygdala-rACC functional connectivity was associated with greater anxiety and depression symptoms during early adulthood, while increased structural connectivity in centromedial amygdala-anterior vmPFC white matter was associated with greater anxiety/depression during late childhood.

CONCLUSIONS

Specific developmental periods of functional and structural connectivity between the amygdala and the prefrontal systems may contribute to the emergence of anxiety and depressive symptoms and may play a critical role in the emergence of psychiatric disorders in adolescence.

Motivational Deficits in Schizophrenia Are Associated With Reduced Differentiation Between Gain and Loss-Avoidance Feedback in the Striatum

BACKGROUND

The current study was designed to test the hypothesis that motivational deficits in schizophrenia (SZ) are tied to a reduced ability to differentially signal gains and instances of loss-avoidance in the brain, leading to reduced ability to form adaptive representations of expected value.

METHODS

We administered a reinforcement learning paradigm to 27 medicated SZ patients and 27 control subjects in which participants learned three probabilistic discriminations. In regions of interest in reward networks identified a priori, we examined contrasts between trial types with different expected values (e.g., expected gain-nonmonetary) and between outcomes with the same prediction error valence but different experienced values (e.g., gain-loss-avoidance outcome, miss-loss outcome).

RESULTS

Both whole-brain and region of interest analyses revealed that SZ patients showed reduced differentiation between gain and loss-avoidance outcomes in the dorsal anterior cingulate cortex and bilateral anterior insula. That is, SZ patients showed reduced contrasts between positive prediction errors of different objective values in these areas. In addition, we observed significant correlations between gain-loss-avoidance outcome contrasts in the ventral striatum and ratings for avolition/anhedonia and between expected gain-nonmonetary contrasts in the ventral striatum and ventromedial prefrontal cortex.

CONCLUSIONS

These results provide further evidence for intact prediction error signaling in medicated SZ patients, especially with regard to loss-avoidance. By contrast, components of frontostriatal circuits appear to show reduced sensitivity to the absolute valence of expected and experienced outcomes, suggesting a mechanism by which motivational deficits may emerge.

Multivariate Neural Representations of Value during Reward Anticipation and Consummation in the Human Orbitofrontal Cortex

The role of the orbitofrontal cortex (OFC) in value processing is a focus of research. Conventional imaging analysis, where smoothing and averaging are employed, may not be sufficiently sensitive in studying the OFC, which has heterogeneous anatomical structures and functions. In this study, we employed representational similarity analysis (RSA) to reveal the multi-voxel fMRI patterns in the OFC associated with value processing during the anticipatory and the consummatory phases. We found that multi-voxel activation patterns in the OFC encoded magnitude and partial valence information (win vs. loss) but not outcome (favourable vs. unfavourable) during reward consummation. Furthermore, the lateral OFC rather than the medial OFC encoded loss information. Also, we found that OFC encoded values in a similar way to the ventral striatum (VS) or the anterior insula (AI) during reward anticipation regardless of motivated response and to the medial prefrontal cortex (MPFC) and the VS in reward consummation. In contrast, univariate analysis did not show changes of activation in the OFC. These findings suggest an important role of the OFC in value processing during reward anticipation and consummation.

Neural and neurochemical basis of reinforcement-guided decision making

Decision making is an adaptive behavior that takes into account several internal and external input variables and leads to the choice of a course of action over other available and often competing alternatives. While it has been studied in diverse fields ranging from mathematics, economics, ecology, and ethology to psychology and neuroscience, recent cross talk among perspectives from different fields has yielded novel descriptions of decision processes. Reinforcement-guided decision making models are based on economic and reinforcement learning theories, and their focus is on the maximization of acquired benefit over a defined period of time. Studies based on reinforcement-guided decision making have implicated a large network of neural circuits across the brain. This network includes a wide range of cortical (e.g., orbitofrontal cortex and anterior cingulate cortex) and subcortical (e.g., nucleus accumbens and subthalamic nucleus) brain areas and uses several neurotransmitter systems (e.g., dopaminergic and serotonergic systems) to communicate and process decision-related information. This review discusses distinct as well as overlapping contributions of these networks and neurotransmitter systems to the processing of decision making. We end the review by touching on neural circuitry and neuromodulatory regulation of exploratory decision making.

The neural dynamics of reward value and risk coding in the human orbitofrontal cortex

The orbitofrontal cortex is known to carry information regarding expected reward, risk and experienced outcome. Yet, due to inherent limitations in lesion and neuroimaging methods, the neural dynamics of these computations has remained elusive in humans. Here, taking advantage of the high temporal definition of intracranial recordings, we characterize the neurophysiological signatures of the intact orbitofrontal cortex in processing information relevant for risky decisions. Local field potentials were recorded from the intact orbitofrontal cortex of patients suffering from drug-refractory partial epilepsy with implanted depth electrodes as they performed a probabilistic reward learning task that required them to associate visual cues with distinct reward probabilities. We observed three successive signals: (i) around 400 ms after cue presentation, the amplitudes of the local field potentials increased with reward probability; (ii) a risk signal emerged during the late phase of reward anticipation and during the outcome phase; and (iii) an experienced value signal appeared at the time of reward delivery. Both the medial and lateral orbitofrontal cortex encoded risk and reward probability while the lateral orbitofrontal cortex played a dominant role in coding experienced value. The present study provides the first evidence from intracranial recordings that the human orbitofrontal cortex codes reward risk both during late reward anticipation and during the outcome phase at a time scale of milliseconds. Our findings offer insights into the rapid mechanisms underlying the ability to learn structural relationships from the environment.

Human ventromedial prefrontal lesions alter incentivisation by reward

Although medial frontal brain regions are implicated in valuation of rewards, evidence from focal lesions to these areas is scant, with many conflicting results regarding motivation and affect, and no human studies specifically examining incentivisation by reward. Here, 19 patients with isolated, focal damage in ventral and medial prefrontal cortex were selected from a database of 453 individuals with subarachnoid haemorrhage. Using a speeded saccadic task based on the oculomotor capture paradigm, we manipulated the maximum reward available on each trial using an auditory incentive cue. Modulation of behaviour by motivation permitted quantification of reward sensitivity. At the group level, medial frontal damage was overall associated with significantly reduced effects of reward on invigorating saccadic velocity and autonomic (pupil) responses compared to age-matched, healthy controls. Crucially, however, some individuals instead showed abnormally strong incentivisation effects for vigour. Increased sensitivity to rewards within the lesion group correlated with damage in subgenual ventromedial prefrontal cortex (vmPFC) areas, which have recently become the target for deep brain stimulation (DBS) in depression. Lesion correlations with clinical apathy suggested that the apathy associated with prefrontal damage is in fact reduced by damage at those coordinates. Reduced reward sensitivity showed a trend to correlate with damage near nucleus accumbens. Lesions did not, on the other hand, influence reward sensitivity of cognitive control, as measured by distractibility. Thus, although medial frontal lesions may generally reduce reward sensitivity, damage to key subregions paradoxically protect from this effect.