Neuronal Reward and Decision Signals: From Theories to Data

Neuroimaging glutamatergic mechanisms differentiating antipsychotic treatment-response

Glutamatergic dysfunction is associated with failure to respond to antipsychotic medication in individuals with schizophrenia. Our objective was to combine neurochemical and functional brain imaging methods to investigate glutamatergic dysfunction and reward processing in such individuals compared with those with treatment responsive schizophrenia, and healthy controls. 60 participants played a trust task, while undergoing functional magnetic resonance imaging: 21 classified as having treatment-resistant schizophrenia, 21 patients with treatment-responsive schizophrenia, and 18 healthy controls. Proton magnetic resonance spectroscopy was also acquired to measure glutamate in the anterior cingulate cortex. Compared to controls, treatment responsive and treatment-resistant participants showed reduced investments during the trust task. For treatment-resistant individuals, glutamate levels in the anterior cingulate cortex were associated with signal decreases in the right dorsolateral prefrontal cortex when compared to those treatment-responsive, and with bilateral dorsolateral prefrontal cortex and left parietal association cortex when compared to controls. Treatment-responsive participants showed significant signal decreases in the anterior caudate compared to the other two groups. Our results provide evidence that glutamatergic differences differentiate treatment resistant and responsive schizophrenia. The differentiation of cortical and sub-cortical reward learning substrates has potential diagnostic value. Future novel interventions might therapeutically target neurotransmitters affecting the cortical substrates of the reward network.

Aging, uncertainty, and decision making-A review

There is a great deal of uncertainty in the world. One common source of uncertainty results from incomplete or missing information about probabilistic outcomes (i.e., outcomes that may occur), which influences how people make decisions. The impact of this type of uncertainty may particularly pronounced for older adults, who, as the primary leaders around the world, make highly impactful decisions with lasting outcomes. This review examines the ways in which uncertainty about probabilistic outcomes is perceived, handled, and represented in the aging brain, with an emphasis on how uncertainty may specifically affect decision making in later life. We describe the role of uncertainty in decision making and aging from four perspectives, including 1) theoretical, 2) self-report, 3) behavioral, and 4) neuroscientific. We report evidence of any age-related differences in uncertainty among these contexts and describe how these changes may affect decision making. We then integrate the findings across the distinct perspectives, followed by a discussion of important future directions for research on aging and uncertainty, including prospection, domain-specificity in risk-taking behaviors, and choice overload.

Reward network dysfunction is associated with cognitive impairment after stroke

Stroke survivors not only suffer from severe motor, speech and neurocognitive deficits, but in many cases also from a "lack of pleasure" and a reduced motivational level. Especially apathy and anhedonic symptoms can be linked to a dysfunction of the reward system. Rewards are considered as important co-factor for learning, so the question arises as to why and how this affects the rehabilitation of stroke patients. We investigated reward behaviour, learning ability and brain network connectivity in acute (3-7d) mild to moderate stroke patients (n = 28) and age-matched healthy controls (n = 26). Reward system activity was assessed using the Monetary Incentive Delay task (MID) during magnetoencephalography (MEG). Coherence analyses were used to demonstrate reward effects on brain functional network connectivity. The MID-task showed that stroke survivors had lower reward sensitivity and required greater monetary incentives to improve performance and showed deficits in learning improvement. MEG-analyses showed a reduced network connectivity in frontal and temporoparietal regions. All three effects (reduced reward sensitivity, reduced learning ability and altered cerebral connectivity) were found to be closely related and differed strongly from the healthy group. Our results reinforce the notion that acute stroke induces reward network dysfunction, leading to functional impairment of behavioural systems. These findings are representative of a general pattern in mild strokes and are independent of the specific lesion localisation. For stroke rehabilitation, these results represent an important point to identify the reduced learning capacity after stroke and to implement individualised recovery exercises accordingly.

Anger, agency, risk and action: a neurobehavioral model with proof-of-concept in healthy young adults

Introduction

Anger can engender action by individuals and groups. It is thus important to understand anger's behavioral phenotypes and their underlying neural substrates. Here, we introduce a construct we term agentic anger, a negatively valenced internal state that motivates action to achieve risky goals. We evaluate our neurobehavioral model via testable hypotheses in two proof-of-concept studies.

Study 1 Methods

Study 1 used the Incentive Balloon Analogue Risk Task in a within-subjects, repeated measures design in 39 healthy volunteers to evaluate: (a) impact of blockade of reward on agentic anger, assessed by self-reports of negative activation (NA), (b) impact of achievement of reward on exuberance, assessed by self-reports of positive activation (PA), (c) the interrelationship of these valenced states, and (d) their relationship with personality.

Study 1 Results

Task-induced NA was positively correlated with task-induced PA, risk-taking on the task and trait Social Potency (SP), a measure of trait agency and reward sensitivity on the Multidimensional Personality Questionnaire Brief-Form.

Study 2 Methods

Study 2 assessed functional MRI response to stakes for risk-taking in healthy volunteers receiving 20 mg d-amphetamine in a double-blinded, placebo-controlled crossover design (N = 10 males), providing preliminary information on ventral striatal response to risky rewards during catecholamine activation.

Study 2 Results

Trait SP and task-induced PA were strongly positively related to catecholamine-facilitated BOLD response in the right nucleus accumbens, a brain region where DA prediction error signal shapes action value and selection. Participants' task-induced NA was strongly positively related with trait SP and task-induced PA, replicating the findings of Study 1.

Discussion

Together these results inform the phenomenology and neurobiology of agentic anger, which recruits incentive motivational circuitry and motivates personal action in response to goals that entail risk (defined as exposure to uncertainty, obstacles, potential harm, loss and/or financial, emotional, bodily, or moral peril). Neural mechanisms of agency, anger, exuberance, and risk-taking are discussed, with implications for personal and group action, decision-making, social justice, and behavior change.

The effect of prediction error on episodic memory encoding is modulated by the outcome of the predictions

Expectations can lead to prediction errors of varying degrees depending on the extent to which the information encountered in the environment conforms with prior knowledge. While there is strong evidence on the computationally specific effects of such prediction errors on learning, relatively less evidence is available regarding their effects on episodic memory. Here, we had participants work on a task in which they learned context/object-category associations of different strengths based on the outcomes of their predictions. We then used a reinforcement learning model to derive subject-specific trial-to-trial estimates of prediction error at encoding and link it to subsequent recognition memory. Results showed that model-derived prediction errors at encoding influenced subsequent memory as a function of the outcome of participants' predictions (correct vs. incorrect). When participants correctly predicted the object category, stronger prediction errors (as a consequence of weak expectations) led to enhanced memory. In contrast, when participants incorrectly predicted the object category, stronger prediction errors (as a consequence of strong expectations) led to impaired memory. These results highlight the important moderating role of choice outcome that may be related to interactions between the hippocampal and striatal dopaminergic systems.

Roles of octopamine neurons in the vertical lobe of the mushroom body for the execution of a conditioned response in cockroaches

Aminergic neurons mediate reward signals in mammals and insects. In crickets, we showed that blockade of synaptic transmission from octopamine neurons (OANs) impairs conditioning of an odor (conditioned stimulus, CS) with water or sucrose (unconditioned stimulus, US) and execution of a conditioned response (CR) to the CS. It has not yet been established, however, whether findings in crickets can be applied to other species of insects. In this study, we investigated the roles of OANs in conditioning of salivation, monitored by activities of salivary neurons, and in execution of the CR in cockroaches (Periplaneta americana). We showed that injection of epinastine (an OA receptor antagonist) into the head hemolymph impaired both conditioning and execution of the CR, in accordance with findings in crickets. Moreover, local injection of epinastine into the vertical lobes of the mushroom body (MB), the center for associative learning and control of the CR, impaired execution of the CR, whereas injection of epinastine into the calyces of the MB or the antennal lobes (primary olfactory centers) did not. We propose that OANs in the MB vertical lobes play critical roles in the execution of the CR in cockroaches. This is analogous to the fact that midbrain dopamine neurons govern execution of learned actions in mammals.

Expecting the unexpected: a review of learning under uncertainty across development

Many of our decisions take place under uncertainty. To successfully navigate the environment, individuals need to estimate the degree of uncertainty and adapt their behaviors accordingly by learning from experiences. However, uncertainty is a broad construct and distinct types of uncertainty may differentially influence our learning. We provide a semi-systematic review to illustrate cognitive and neurobiological processes involved in learning under two types of uncertainty: learning in environments with stochastic outcomes, and with volatile outcomes. We specifically reviewed studies (N = 26 studies) that included an adolescent population, because adolescence is a period in life characterized by heightened exploration and learning, as well as heightened uncertainty due to experiencing many new, often social, environments. Until now, reviews have not comprehensively compared learning under distinct types of uncertainties in this age range. Our main findings show that although the overall developmental patterns were mixed, most studies indicate that learning from stochastic outcomes, as indicated by increased accuracy in performance, improved with age. We also found that adolescents tended to have an advantage compared with adults and children when learning from volatile outcomes. We discuss potential mechanisms explaining these age-related differences and conclude by outlining future research directions.

Dopamine in the dorsal bed nucleus of stria terminalis signals Pavlovian sign-tracking and reward violations

Midbrain and striatal dopamine signals have been extremely well characterized over the past several decades, yet novel dopamine signals and functions in reward learning and motivation continue to emerge. A similar characterization of real-time sub-second dopamine signals in areas outside of the striatum has been limited. Recent advances in fluorescent sensor technology and fiber photometry permit the measurement of dopamine binding correlates, which can divulge basic functions of dopamine signaling in non-striatal dopamine terminal regions, like the dorsal bed nucleus of the stria terminalis (dBNST). Here, we record GRABDA signals in the dBNST during a Pavlovian lever autoshaping task. We observe greater Pavlovian cue-evoked dBNST GRABDA signals in sign-tracking (ST) compared to goal-tracking/intermediate (GT/INT) rats and the magnitude of cue-evoked dBNST GRABDA signals decreases immediately following reinforcer-specific satiety. When we deliver unexpected rewards or omit expected rewards, we find that dBNST dopamine signals encode bidirectional reward prediction errors in GT/INT rats, but only positive prediction errors in ST rats. Since sign- and goal-tracking approach strategies are associated with distinct drug relapse vulnerabilities, we examined the effects of experimenter-administered fentanyl on dBNST dopamine associative encoding. Systemic fentanyl injections do not disrupt cue discrimination but generally potentiate dBNST dopamine signals. These results reveal multiple dBNST dopamine correlates of learning and motivation that depend on the Pavlovian approach strategy employed.

Reward responsiveness in autism and autistic traits - Evidence from neuronal, autonomic, and behavioural levels

Autism has been linked to atypicalities in reward processing, especially in the social domain. However, results are heterogeneous, and their interpretation is hindered by the use of personally non-relevant social rewards. In this study we investigated behavioural (reaction times), neuronal (event-related potentials), and autonomic (pupil sizes) responses to personally relevant social rewards, money, and neutral outcomes in 26 autistic and 53 non-autistic subjects varying in levels of autistic traits. As hypothesised and preregistered, autism and autistic traits did not differently influence responses to social, monetary, or neutral outcomes on either response level. While groups did not differ in behaviour (reaction times), autism was linked to generally enhanced brain responses in early anticipation and larger pupil constrictions in reward reception. Together, these results suggest that when using personally relevant stimuli, autism is linked to generally preserved, although less neuronally efficient processing of rewards. Considering the role of social relevance in reward processing, we propose an interpretation of contradictory evidence from clinical practice and empirical research.

Association Between the Fronto-Limbic Network and Cognitive and Emotional Functioning in Individuals With Bipolar Disorder: A Systematic Review and Meta-analysis

Importance

Individuals with bipolar disorder (BD) experience cognitive and emotional dysfunctions. Various brain circuits are implicated in BD but have not been investigated in a meta-analysis of functional magnetic resonance imaging (fMRI) studies.

Objective

To investigate the brain functioning of individuals with BD compared with healthy control individuals in the domains of emotion processing, reward processing, and working memory.

Data Sources

All fMRI experiments on BD published before March 2020, as identified in a literature search of PubMed, Embase, Web of Science, Cochrane Library, PsycInfo, Emcare, Academic Search Premier, and ScienceDirect. The literature search was conducted on February 21, 2017, and March 2, 2020, and data were analyzed from January 2021 to January 2022.

Study Selection

fMRI experiments comparing adult individuals with BD and healthy control individuals were selected if they reported whole-brain results, including a task assessing at least 1 of the domains. In total, 2320 studies were screened, and 253 full-text articles were evaluated.

Data Extraction and Synthesis

A total of 49 studies were included after selection procedure. Coordinates reporting significant activation differences between individuals with BD and healthy control individuals were extracted. Differences in brain region activity were tested using the activation likelihood estimation method.

Main Outcomes and Measures

A whole-brain meta-analysis evaluated whether reported differences in brain activation in response to stimuli in 3 cognitive domains between individuals with BD and healthy control individuals were different.

Results

The study population included 999 individuals with BD (551 [55.2%] female) and 1027 healthy control individuals (532 [51.8%] female). Compared with healthy control individuals, individuals with BD showed amygdala and hippocampal hyperactivity and hypoactivation in the inferior frontal gyrus during emotion processing (20 studies; 324 individuals with BD and 369 healthy control individuals), hyperactivation in the orbitofrontal cortex during reward processing (9 studies; 195 individuals with BD and 213 healthy control individuals), and hyperactivation in the ventromedial prefrontal cortex and subgenual anterior cingulate cortex during working memory (20 studies; 530 individuals with BD and 417 healthy control individuals). Limbic hyperactivation was only found during euthymia in the emotion and reward processing domains; abnormalities in frontal cortex activity were also found in individuals with BD with mania and depression.

Conclusions and Relevance

This systematic review and meta-analysis revealed evidence for activity disturbances in key brain areas involved in cognitive and emotion processing in individuals with BD. Most of the regions are part of the fronto-limbic network. The results suggest that aberrations in the fronto-limbic network, present in both euthymic and symptomatic individuals, may be underlying cognitive and emotional dysfunctions in BD.

A meta-analysis of the neural substrates of monetary reward anticipation and outcome in alcohol use disorder

The capacity to anticipate and detect rewarding outcomes is fundamental for the development of adaptive decision-making and goal-oriented behavior. Delineating the neural correlates of different stages of reward processing is imperative for understanding the neurobiological mechanism underlying alcohol use disorder (AUD). To examine the neural correlates of monetary anticipation and outcome in AUD patients, we performed two separate voxel-wise meta-analyses of functional neuroimaging studies, including 12 studies investigating reward anticipation and 7 studies investigating reward outcome using the monetary incentive delay task. During the anticipation stage, AUD patients displayed decreased activation in response to monetary cues in mesocortical-limbic circuits and sensory areas, including the ventral striatum (VS), insula, hippocampus, inferior occipital gyrus, supramarginal gyrus, lingual gyrus and fusiform gyrus. During the outcome stage, AUD patients exhibited reduced activation in the dorsal striatum, VS and insula, and increased activation in the orbital frontal cortex and medial temporal area. Our findings suggest that different activation patterns are associated with nondrug rewards during different reward processing stages, potentially reflecting a changed sensitivity to monetary reward in AUD.

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.

A process model account of the role of dopamine in intertemporal choice

Theoretical accounts disagree on the role of dopamine in intertemporal choice and assume that dopamine either promotes delay of gratification by increasing the preference for larger rewards or that dopamine reduces patience by enhancing the sensitivity to waiting costs. Here, we reconcile these conflicting accounts by providing empirical support for a novel process model according to which dopamine contributes to two dissociable components of the decision process, evidence accumulation and starting bias. We re-analyzed a previously published data set where intertemporal decisions were made either under the D2 antagonist amisulpride or under placebo by fitting a hierarchical drift diffusion model that distinguishes between dopaminergic effects on the speed of evidence accumulation and the starting point of the accumulation process. Blocking dopaminergic neurotransmission not only strengthened the sensitivity to whether a reward is perceived as worth the delay costs during evidence accumulation (drift rate) but also attenuated the impact of waiting costs on the starting point of the evidence accumulation process (bias). In contrast, re-analyzing data from a D1 agonist study provided no evidence for a causal involvement of D1R activation in intertemporal choices. Taken together, our findings support a novel, process-based account of the role of dopamine for cost-benefit decision making, highlight the potential benefits of process-informed analyses, and advance our understanding of dopaminergic contributions to decision making.

Evidence for entropy maximisation in human free choice behaviour

The freedom to choose between options is strongly linked to notions of free will. Accordingly, several studies have shown that individuals demonstrate a preference for choice, or the availability of multiple options, over and above utilitarian value. Yet we lack a decision-making framework that integrates preference for choice with traditional utility maximisation in free choice behaviour. Here we test the predictions of an inference-based model of decision-making in which an agent actively seeks states yielding entropy (availability of options) in addition to utility (economic reward). We designed a study in which participants freely navigated a virtual environment consisting of two consecutive choices leading to reward locations in separate rooms. Critically, the choice of one room always led to two final doors while, in the second room, only one door was permissible to choose. This design allowed us to separately determine the influence of utility and entropy on participants' choice behaviour and their self-evaluation of free will. We found that choice behaviour was better predicted by an inference-based model than by expected utility alone, and that both the availability of options and the value of the context positively influenced participants' perceived freedom of choice. Moreover, this consideration of options was apparent in the ongoing motion dynamics as individuals navigated the environment. In a second study, in which participants selected between rooms that gave access to three or four doors, we observed a similar pattern of results, with participants preferring the room that gave access to more options and feeling freer in it. These results suggest that free choice behaviour is well explained by an inference-based framework in which both utility and entropy are optimised and supports the idea that the feeling of having free will is tightly related to options availability.

Age-related differences in ERP correlates of value-based decision making

This study evaluates age-related differences in the temporal dynamics underlying neural processing of value for decision-making in younger and older adults. We applied a lottery-choice task with event-related potentials to determine how and when brain activity during choice and outcome processing diverge between younger and older adults. Behaviorally, older adults accepted more losing stakes than younger adults. During choice, younger adults evinced higher P2 ERP-response positivity with a later P3 positivity that monotonically increased with low to middle to high win probability. Older adults evinced lower P2 responses and P3 amplitudes with more positivity for high and low relative to middle win probability. Both age groups showed similar feedback-related negativity and late parietal positivity, indicating intact reward prediction error representations and salience integration. Feedback-P3 showed more complex sensitivity to expectancy violations in older than younger adults, suggesting subjective uncertainty about reward expectations. Reduced early general neural processing of objective stimulus value with greater contribution of downstream subjective processes might underlie older adult risk-taking behaviors.

Designing an App for Parents and Caregivers to Promote Cognitive and Socioemotional Development and Well-being Among Children Aged 0 to 5 Years in Diverse Cultural Settings: Scientific Framework

Recent years have seen remarkable progress in our scientific understanding of early childhood social, emotional, and cognitive development, as well as our capacity to widely disseminate health information by using digital technologies. Together, these scientific and technological advances offer exciting opportunities to deliver high-quality information about early childhood development (ECD) to parents and families globally, which may ultimately lead to greater knowledge and confidence among parents and better outcomes among children (particularly in lower- and middle-income countries). With these potential benefits in mind, we set out to design, develop, implement, and evaluate a new parenting app-Thrive by Five-that will be available in 30 countries. The app will provide caregivers and families with evidence-based and culturally appropriate information about ECD, accompanied by sets of collective actions that go beyond mere tips for parenting practices. Herein, we describe this ongoing global project and discuss the components of our scientific framework for developing and prototyping the app's content. Specifically, we describe (1) 5 domains that are used to organize the content and goals of the app's information and associated practices; (2) 5 neurobiological systems that are relevant to ECD and can be behaviorally targeted to potentially influence social, emotional, and cognitive development; (3) our anthropological and cultural framework for learning about local contexts and appreciating decolonization perspectives; and (4) our approach to tailoring the app's content to local contexts, which involves collaboration with in-country partner organizations and local and international subject matter experts in ECD, education, medicine, psychology, and anthropology, among others. Finally, we provide examples of the content that was incorporated in Thrive by Five when it launched globally.

The architecture of abnormal reward behaviour in dementia: multimodal hedonic phenotypes and brain substrate

Abnormal reward processing is a hallmark of neurodegenerative diseases, most strikingly in frontotemporal dementia. However, the phenotypic repertoire and neuroanatomical substrates of abnormal reward behaviour in these diseases remain incompletely characterized and poorly understood. Here we addressed these issues in a large, intensively phenotyped patient cohort representing all major syndromes of sporadic frontotemporal dementia and Alzheimer's disease. We studied 27 patients with behavioural variant frontotemporal dementia, 58 with primary progressive aphasia (22 semantic variant, 24 non-fluent/agrammatic variant and 12 logopenic) and 34 with typical amnestic Alzheimer's disease, in relation to 42 healthy older individuals. Changes in behavioural responsiveness were assessed for canonical primary rewards (appetite, sweet tooth, sexual activity) and non-primary rewards (music, religion, art, colours), using a semi-structured survey completed by patients' primary caregivers. Changes in more general socio-emotional behaviours were also recorded. We applied multiple correspondence analysis and k-means clustering to map relationships between hedonic domains and extract core factors defining aberrant hedonic phenotypes. Neuroanatomical associations were assessed using voxel-based morphometry of brain MRI images across the combined patient cohort. Altered (increased and/or decreased) reward responsiveness was exhibited by most patients in the behavioural and semantic variants of frontotemporal dementia and around two-thirds of patients in other dementia groups, significantly (P < 0.05) more frequently than in healthy controls. While food-directed changes were most prevalent across the patient cohort, behavioural changes directed toward non-primary rewards occurred significantly more frequently (P < 0.05) in the behavioural and semantic variants of frontotemporal dementia than in other patient groups. Hedonic behavioural changes across the patient cohort were underpinned by two principal factors: a 'gating' factor determining the emergence of altered reward behaviour and a 'modulatory' factor determining how that behaviour is directed. These factors were expressed jointly in a set of four core, trans-diagnostic and multimodal hedonic phenotypes: 'reward-seeking', 'reward-restricted', 'eating-predominant' and 'control-like'-variably represented across the cohort and associated with more pervasive socio-emotional behavioural abnormalities. The principal gating factor was associated (P < 0.05 after correction for multiple voxel-wise comparisons over the whole brain) with a common profile of grey matter atrophy in anterior cingulate, bilateral temporal poles, right middle frontal and fusiform gyri: the cortical circuitry that mediates behavioural salience and semantic and affective appraisal of sensory stimuli. Our findings define a multi-domain phenotypic architecture for aberrant reward behaviours in major dementias, with novel implications for the neurobiological understanding and clinical management of these diseases.

Brain Reward Circuits Promote Stress Resilience and Health: Implications for Reward-Based Interventions

From the COVID-19 global pandemic to racial injustice and the continued impact of climate change on communities across the globe, the last couple of years have demonstrated the need for a greater understanding of how to protect people from the negative consequences of stress. Here, I outline a perspective on how the brain's reward system might be an important, but often understudied, protective mechanism for stress resilience and stress-related health outcomes. I describe work suggesting that reward system engagement inhibits the stress response and is associated with improved health outcomes including reduced depressive symptomatology and slowed cancer progression. I then highlight important future directions for translational research and illustrate the value of this perspective for improving behavioral interventions in clinical psychology and beyond.

Differential neural reward reactivity in response to food advertising medium in children

Introduction

Food cues including food advertisements (ads) activate brain regions related to motivation and reward. These responses are known to correlate with eating behaviors and future weight gain. The objective of this study was to compare brain responses to food ads by different types of ad mediums, dynamic (video) and static (images), to better understand how medium type impacts food cue response.

Methods

Children aged 9-12 years old were recruited to complete a functional magnetic resonance imaging (fMRI) paradigm that included both food and non-food dynamic and static ads. Anatomical and functional images were preprocessed using the fMRIPrep pipeline. A whole-brain analysis and a targeted region-of-interest (ROI) analysis for reward regions (nucleus accumbens, orbitofrontal cortex, amygdala, insula, hypothalamus, ventral tegmental area, substantia nigra) were conducted. Individual neural responses to dynamic and static conditions were compared using a paired t-test. Linear mixed-effects models were then constructed to test the differential response by ad condition after controlling for age, sex, BMI-z, physical activity, and % of kcal consumed of a participant's estimated energy expenditure in the pre-load prior to the MRI scan.

Results

A total of 115 children (mean=10.9 years) completed the fMRI paradigm. From the ROI analyses, the right and left hemispheres of the amygdala and insula, and the right hemisphere of the ventral tegmental area and substantia nigra showed significantly higher responses for the dynamic food ad medium after controlling for covariates and a false discovery rate correction. From the whole-brain analysis, 21 clusters showed significant differential responses between food ad medium including the precuneus, middle temporal gyrus, superior temporal gyrus, and inferior frontal gyrus, and all regions remained significant after controlling for covariates.

Discussion

Advertising medium has unique effects on neural response to food cues. Further research is needed to understand how this differential activation by ad medium ultimately affects eating behaviors and weight outcomes.

Human and macaque pairs employ different coordination strategies in a transparent decision game

Many real-world decisions in social contexts are made while observing a partner's actions. To study dynamic interactions during such decisions, we developed a setup where two agents seated face-to-face to engage in game-theoretical tasks on a shared transparent touchscreen display ('transparent games'). We compared human and macaque pairs in a transparent version of the coordination game 'Bach-or-Stravinsky', which entails a conflict about which of two individually-preferred opposing options to choose to achieve coordination. Most human pairs developed coordinated behavior and adopted dynamic turn-taking to equalize the payoffs. All macaque pairs converged on simpler, static coordination. Remarkably, two animals learned to coordinate dynamically after training with a human confederate. This pair selected the faster agent's preferred option, exhibiting turn-taking behavior that was captured by modeling the visibility of the partner's action before one's own movement. Such competitive turn-taking was unlike the prosocial turn-taking in humans, who equally often initiated switches to and from their preferred option. Thus, the dynamic coordination is not restricted to humans but can occur on the background of different social attitudes and cognitive capacities in rhesus monkeys. Overall, our results illustrate how action visibility promotes the emergence and maintenance of coordination when agents can observe and time their mutual actions.

Temporal prediction error triggers amygdala-dependent memory updating in appetitive operant conditioning in rats

Reinforcement learning theories postulate that prediction error, i.e., a discrepancy between the actual and expected outcomes, drives reconsolidation and new learning, inducing an updating of the initial memory. Pavlovian studies have shown that prediction error detection is a fundamental mechanism in triggering amygdala-dependent memory updating, where the temporal relationship between stimuli plays a critical role. However, in contrast to the well-established findings in aversive situations (e.g., fear conditioning), only few studies exist on prediction error in appetitive operant conditioning, and even less with regard to the role of temporal parameters. To explore if temporal prediction error in an appetitive operant paradigm could generate an updating and consequent reconsolidation and/or new learning of temporal association, we ran four experiments in adult male rats. Experiment 1 verified whether an unexpected delay in the time of reward's availability (i.e., a negative temporal prediction error) in a single session produces an updating in long-term memory of temporal expectancy in an appetitive operant conditioning. Experiment 2 showed that negative prediction errors, either due to the temporal change or through reward omission, increased in the basolateral amygdala nucleus (BLA) the activation of a protein that is critical for memory formation. Experiment 3 revealed that the presence of a protein synthesis inhibitor (anisomycin) in the BLA during the session when the reward was delayed (Error session) affected the temporal updating. Finally, Experiment 4 showed that anisomycin, when infused immediately after the Error session, interfered with the long-term memory of the temporal updating. Together, our study demonstrated an involvement of BLA after a change in temporal and reward contingencies, and in the resulting updating in long-term memory in appetitive operant conditioning.

Transforming experiences: Neurobiology of memory updating/editing

Long-term memory is achieved through a consolidation process where structural and molecular changes integrate information into a stable memory. However, environmental conditions constantly change, and organisms must adapt their behavior by updating their memories, providing dynamic flexibility for adaptive responses. Consequently, novel stimulation/experiences can be integrated during memory retrieval; where consolidated memories are updated by a dynamic process after the appearance of a prediction error or by the exposure to new information, generating edited memories. This review will discuss the neurobiological systems involved in memory updating including recognition memory and emotional memories. In this regard, we will review the salient and emotional experiences that promote the gradual shifting from displeasure to pleasure (or vice versa), leading to hedonic or aversive responses, throughout memory updating. Finally, we will discuss evidence regarding memory updating and its potential clinical implication in drug addiction, phobias, and post-traumatic stress disorder.

Reward enhances resilience to chronic social defeat stress in mice: Neural ECs and mGluR5 mechanism via neuroprotection in VTA and DRN

INTRODUCTION

Stress often leads to emotional disorders such as depression. The reward might render this effect through the enhancement of stress resilience. However, the effect of reward on stress resilience under different intensities of stress needs more evidence, and its potential neural mechanism has been poorly revealed. It has been reported that the endogenous cannabinoid system (ECs) and downstream metabolic glutamate receptor 5 (mGluR5) are closely related to stress and reward, which might be the potential cerebral mechanism between reward and stress resilience, but there is a lack of direct evidence. This study aims to observe the effect of reward on stress resilience under different intensities of stress and further explore potential cerebral mechanisms underlying this effect.

METHODS

Using the chronic social defeat stress model, we applied reward (accompanied by a female mouse) under different intensities of stress in mice during the modeling process. The impact of reward on stress resilience and the potential cerebral mechanism were observed after modeling through behavioral tests and biomolecules.

RESULTS

The results showed that stronger stress led to higher degrees of depression-like behavior. Reward reduced depression-like behavior and enhanced stress resilience (all p-value <0.05) (more social interaction in the social test, less immobility time in the forced swimming test, etc.), with a stronger effect under the large stress. Furthermore, the mRNA expression levels of CB1 and mGluR5, the protein expression level of mGluR5, and the expression level of 2-AG (2-arachidonoylglycerol) in both ventral tegmental area (VTA) and dorsal raphe nucleus (DRN) were significantly upregulated by reward after modeling (all p-value <0.05). However, the protein expression of CB1 in VTA and DRN and the expression of AEA (anandamide) in VTA did not differ significantly between groups. Intraperitoneal injection of a CB1 agonist (URB-597) during social defeat stress significantly reduced depression-like behavior compared with a CB1 inhibitor (AM251) (all p-value <0.05). Interestingly, in DRN, the expression of AEA in the stress group was lower than that of the control group, with or without reward (all p-value <0.05).

DISCUSSION

These findings demonstrate that combined social and sexual reward has a positive effect on stress resilience during chronic social defeat stress, potentially by influencing the ECs and mGluR5 in VTA and DRN.

Plasticity of synapses and reward circuit function in the genesis and treatment of depression

What changes in brain function cause the debilitating symptoms of depression? Can we use the answers to this question to invent more effective, faster acting antidepressant drug therapies? This review provides an overview and update of the converging human and preclinical evidence supporting the hypothesis that changes in the function of excitatory synapses impair the function of the circuits they are embedded in to give rise to the pathological changes in mood, hedonic state, and thought processes that characterize depression. The review also highlights complementary human and preclinical findings that classical and novel antidepressant drugs relieve the symptoms of depression by restoring the functions of these same synapses and circuits. These findings offer a useful path forward for designing better antidepressant compounds.

Mesolimbic dopamine adapts the rate of learning from action

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions^1-3. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction⁴; however, so far there has been little consideration of how direct policy learning might inform our understanding⁵. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning⁶.

Evaluating instrumental learning and striatal-cortical functional connectivity in adolescent alcohol and cannabis use

Adolescence is a vulnerable time for the acquisition of substance use disorders, potentially relating to ongoing development of neural circuits supporting instrumental learning. Striatal-cortical circuits undergo dynamic changes during instrumental learning and are implicated in contemporary addiction theory. Human studies have not yet investigated these dynamic changes in relation to adolescent substance use. Here, functional magnetic resonance imaging was used while 135 adolescents without (AUD-CUDLow ) and with significant alcohol (AUDHigh ) or cannabis use disorder symptoms (CUDHigh ) performed an instrumental learning task. We assessed how cumulative experience with instrumental cues altered cue selection preferences and functional connectivity strength between reward-sensitive striatal and cortical regions. Adolescents in AUDHigh and CUDHigh groups were slower in learning to select optimal instrumental cues relative to AUD-CUDLow adolescents. The relatively fast learning observed for AUD-CUDLow adolescents coincided with stronger functional connectivity between striatal and frontoparietal regions during early relative to later periods of task experience, whereas the slower learning for the CUDHigh group coincided with the opposite pattern. The AUDHigh group not only exhibited slower learning but also produced more instrumental choice errors relative to AUD-CUDLow adolescents. For the AUDHigh group, Bayesian analyses evidenced moderate support for no experience-related changes in striatal-frontoparietal connectivity strength during the task. Findings suggest that adolescent cannabis use is related to slowed instrumental learning and delays in peak functional connectivity strength between the striatal-frontoparietal regions that support this learning, whereas adolescent alcohol use may be more closely linked to broader impairments in instrumental learning and a general depression of the neural circuits supporting it.

Value-driven modulation of visual perception by visual and auditory reward cues: The role of performance-contingent delivery of reward

Perception is modulated by reward value, an effect elicited not only by stimuli that are predictive of performance-contingent delivery of reward (PC) but also by stimuli that were previously rewarded (PR). PC and PR cues may engage different mechanisms relying on goal-driven versus stimulus-driven prioritization of high value stimuli, respectively. However, these two modes of reward modulation have not been systematically compared against each other. This study employed a behavioral paradigm where participants' visual orientation discrimination was tested in the presence of task-irrelevant visual or auditory reward cues. In the first phase (PC), correct performance led to a high or low monetary reward dependent on the identity of visual or auditory cues. In the subsequent phase (PR), visual or auditory cues were not followed by reward delivery anymore. We hypothesized that PC cues have a stronger modulatory effect on visual discrimination and pupil responses compared to PR cues. We found an overall larger task-evoked pupil dilation in PC compared to PR phase. Whereas PC and PR cues both increased the accuracy of visual discrimination, value-driven acceleration of reaction times (RTs) and pupillary responses only occurred for PC cues. The modulation of pupil size by high reward PC cues was strongly correlated with the modulation of a combined measure of speed and accuracy. These results indicate that although value-driven modulation of perception can occur even when reward delivery is halted, stronger goal-driven control elicited by PC reward cues additionally results in a more efficient balance between accuracy and speed of perceptual choices.

Cholinergic activity reflects reward expectations and predicts behavioral responses

Basal forebrain cholinergic neurons (BFCNs) play an important role in associative learning, suggesting that BFCNs may participate in processing stimuli that predict future outcomes. However, the impact of outcome probabilities on BFCN activity remained elusive. Therefore, we performed bulk calcium imaging and recorded spiking of identified cholinergic neurons from the basal forebrain of mice performing a probabilistic Pavlovian cued outcome task. BFCNs responded more to sensory cues that were often paired with reward. Reward delivery also activated BFCNs, with surprising rewards eliciting a stronger response, whereas punishments evoked uniform positive-going responses. We propose that BFCNs differentially weigh predictions of positive and negative reinforcement, reflecting divergent relative salience of forecasting appetitive and aversive outcomes, partially explained by a simple reinforcement learning model of a valence-weighed unsigned prediction error. Finally, the extent of cue-driven cholinergic activation predicted subsequent decision speed, suggesting that the expectation-gated cholinergic firing is instructive to reward-seeking behaviors.

Usage of L-type calcium channel blockers to suppress drug reward and memory driving addiction: Past, present, and future

Over the past three decades, L-type Ca2+ channel (LTCC) blockers have been considered a potential therapeutic drug to alleviate the symptoms of drug addiction. This idea has been supported, in part, by 1) expression of LTCCs in the brain dopaminergic circuits that are thought to play critical roles in the development and expression of addictive behaviors and 2) common usage of LTCC blockers in treating hypertension, which may enable off-label use of these drugs with good brain penetration as therapeutics for brain disorders. Addiction can be viewed as a maladaptive form of learning where powerful memories of drug-associated stimuli and actions drive compulsive drug intake. Largely under this framework, we will focus on the dopaminergic system that is thought be critically involved in drug-associated learning and memory and provide a brief overview of the past and recent studies testing the therapeutic potential of LTCC blockers for addictive disorders in animal models and humans and offer a future perspective on the use of LTCC blockers in drug addiction and, possibly, addiction to other non-drug rewards (e.g., gambling, eating, shopping). Interested readers can refer to other related articles in this issue and a comprehensive review available elsewhere (Little, 2021) to gain further insights into the roles of LTCCs in drug addiction and withdrawal symptoms associated with dependence. This article is part of the Special Issue on 'L-type calcium channel mechanisms in neuropsychiatric disorders'.

The calcium-sensing receptor (CaSR) regulates zebrafish sensorimotor decision making via a genetically defined cluster of hindbrain neurons

Decision making is a fundamental nervous system function that ranges widely in complexity and speed of execution. We previously established larval zebrafish as a model for sensorimotor decision making and identified the G-protein-coupled calcium-sensing receptor (CaSR) to be critical for this process. Here, we report that CaSR functions in neurons to dynamically regulate the bias between two behavioral outcomes: escapes and reorientations. By employing a computational guided transgenic strategy, we identify a genetically defined neuronal cluster in the hindbrain as a key candidate site for CaSR function. Finally, we demonstrate that transgenic CaSR expression targeting this cluster consisting of a few hundred neurons shifts behavioral bias in wild-type animals and restores decision making deficits in CaSR mutants. Combined, our data provide a rare example of a G-protein-coupled receptor that biases vertebrate sensorimotor decision making via a defined neuronal cluster.

Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction

Animals both explore and avoid novel objects in the environment, but the neural mechanisms that underlie these behaviors and their dynamics remain uncharacterized. Here, we used multi-point tracking (DeepLabCut) and behavioral segmentation (MoSeq) to characterize the behavior of mice freely interacting with a novel object. Novelty elicits a characteristic sequence of behavior, starting with investigatory approach and culminating in object engagement or avoidance. Dopamine in the tail of the striatum (TS) suppresses engagement, and dopamine responses were predictive of individual variability in behavior. Behavioral dynamics and individual variability are explained by a reinforcement-learning (RL) model of threat prediction in which behavior arises from a novelty-induced initial threat prediction (akin to "shaping bonus") and a threat prediction that is learned through dopamine-mediated threat prediction errors. These results uncover an algorithmic similarity between reward- and threat-related dopamine sub-systems.

Ferroelectric Polarized in Transistor Channel Polarity Modulation for Reward-Modulated Spike-Time-Dependent Plasticity Application

Reward signals reflect the developmental tendency of reinforcement learning (RL) agents. Reward-modulated spike-time-dependent plasticity (R-STDP) is an efficient and concise information processing feature in RL. However, the physical construction of R-STDP normally demands complex circuit design engineering, resulting in large power consumption and large area. In this work, we studied the role of ferroelectric polarization in the modulation of carbon nanotube transistor channel polarity. Furthermore, we applied a modulating channel method to construct a 2T synaptic component by spin-coating technology. Based on the nonvolatility of ferroelectric polarization, the synaptic component constructed has the characteristics of reconfigurable polarity. One channel could be modulated to n-type and the other to p-type. One modulated channel was used to perform the STDP function when the reward signal arrived, and the other modulated channel was used to perform the anti-STDP function when the punishment signal arrived. Finally, R-STDP learning rules are implemented on hardware. This work provides a strategy for hardware construction of RL.

Hyperconnectivity and altered interactions of a nucleus accumbens network in post-stroke depression

Post-stroke depression is a common complication of stroke. To date, no consistent locus of injury is associated with this complication. Here, we probed network dynamics and structural alterations in post-stroke depression in four functional circuits linked to major depressive disorder and a visual network, which served as a control network. Forty-four participants with recent stroke (mean age = 69.03, standard deviation age = 8.59, age range = 51–86 and gender: female = 10) and 16 healthy volunteers (mean age = 71.53, standard deviation age = 10.62, age range = 51–84 and gender: female = 11) were imaged with 3-Tesla structural, diffusion and resting-state functional MRI. The Geriatric Depression Scale was administered to measure depression severity. Associations between depression severity and functional connectivity were investigated within networks seeded from nucleus accumbens, amygdala, dorsolateral prefrontal cortex and primary visual cortex. In addition, the default mode network was identified by connectivity with medial prefrontal cortex and posterior cingulate cortex. Circuits that exhibited altered activity associated with depression severity were further investigated by extracting within-network volumetric and microstructural measures from structural images. In the stroke group, functional connectivity within the nucleus accumbens-seeded network (left hemisphere: P = 0.001; and right hemisphere: P = 0.004) and default mode network (cluster one: P &lt; 0.001; and cluster two: P &lt; 0.001) correlated positively with depressive symptoms. Normal anticorrelations between these two networks were absent in patients with post-stroke depression. Grey matter volume of the right posterior cingulate cortex (Pearson correlation coefficient = −0.286, P = 0.03), as well as microstructural measures in the posterior cingulate cortex (right: Pearson correlation coefficient = 0.4, P = 0.024; and left: Pearson correlation coefficient = 0.3, P = 0.048), right medial prefrontal cortex (Pearson correlation coefficient = 0.312, P = 0.039) and the medial forebrain bundle (Pearson correlation coefficient = 0.450, P = 0.003), a major projection pathway interconnecting the nucleus accumbens-seeded network and linking to medial prefrontal cortex, were associated with depression severity. Depression after stroke is marked by reduced mutual inhibition between functional circuits involving nucleus accumbens and default mode network as well as volumetric and microstructural changes within these networks. Aberrant network dynamics present in patients with post-stroke depression are therefore likely to be influenced by secondary, pervasive alterations in grey and white matter, remote from the site of injury.

The neurobiological reward system and binge eating: A critical systematic review of neuroimaging studies

OBJECTIVE

Changes in reward processing are hypothesized to play a role in the onset and maintenance of binge eating (BE). However, despite an increasing number of studies investigating the neurobiological reward system in individuals who binge eat, no comprehensive systematic review exists on this topic. Therefore, this review has the following objectives: (1) identify structural and functional changes in the brain reward system, either during rest or while performing a task; and (2) formulate directions for future research.

METHODS

A search was conducted of articles published until March 31, 2022. Neuroimaging studies were eligible if they wanted to study the reward system and included a group of individuals who binge eat together with a comparator group. Their results were summarized in a narrative synthesis.

RESULTS

A total of 58 articles were included. At rest, individuals who binge eat displayed a lower striatal dopamine release, a change in the volume of the striatum, frontal cortex, and insula, as well as a lower frontostriatal connectivity. While performing a task, there was a higher activity of the brain reward system when anticipating or receiving food, more model-free reinforcement learning, and more habitual behavior. Most studies only included one patient group, used general reward-related measures, and did not evaluate the impact of comorbidities, illness duration, race, or sex.

DISCUSSION

Confirming previous hypotheses, this review finds structural and functional changes in the neurobiological reward system in BE. Future studies should compare disorders, use measures that are specific to BE, and investigate the impact of confounding factors.

PUBLIC SIGNIFICANCE STATEMENT

This systematic review finds that individuals who binge eat display structural and functional changes in the brain reward system. These changes could be related to a higher sensitivity to food, relying more on previous experiences when making decisions, and more habitual behavior. Future studies should use a task that is specific to binge eating, look across different patient groups, and investigate the impact of comorbidities, illness duration, race, and sex.

Abnormal Responses in Cognitive Impulsivity Circuits Are Associated with Glycosylated Hemoglobin Trajectories in Type 1 Diabetes Mellitus and Impaired Metabolic Control

BACKGROUND

Risky health decisions and impulse control profiles may impact on metabolic control in type 1 diabetes mellitus (T1DM). We hypothesize that the neural correlates of cognitive impulsivity and decision-making in T1DM relate to metabolic control trajectories.

METHODS

We combined functional magnetic resonance imaging (fMRI), measures of metabolic trajectories (glycosylated hemoglobin [HbA1c] over multiple time points) and behavioral assessment using a cognitive impulsivity paradigm, the Balloon Analogue Risk Task (BART), in 50 participants (25 T1DM and 25 controls).

RESULTS

Behavioral results showed that T1DM participants followed a rigid conservative risk strategy along the iterative game. Imaging group comparisons showed that patients showed larger activation of reward related, limbic regions (nucleus accumbens, amygdala) and insula (interoceptive saliency network) in initial game stages. Upon game completion differences emerged in relation to error monitoring (anterior cingulate cortex [ACC]) and inhibitory control (inferior frontal gyrus). Importantly, activity in the saliency network (ACC and insula), which monitors interoceptive states, was related with metabolic trajectories, which was also found for limbic/reward networks. Parietal and posterior cingulate regions activated both in controls and patients with adaptive decision-making, and positively associated with metabolic trajectories.

CONCLUSION

We found triple converging evidence when comparing metabolic trajectories, patients versus controls or risk averse (non-learners) versus patients who learned by trial and error. Dopaminergic reward and saliency (interoceptive and error monitoring) circuits show a tight link with impaired metabolic trajectories and cognitive impulsivity in T1DM. Activity in parietal and posterior cingulate are associated with adaptive trajectories. This link between reward-saliency-inhibition circuits suggests novel strategies for patient management.

Nanomedicine in the Face of Parkinson's Disease: From Drug Delivery Systems to Nanozymes

The complexity and overall burden of Parkinson's disease (PD) require new pharmacological approaches to counteract the symptomatology while reducing the progressive neurodegeneration of affected dopaminergic neurons. Since the pathophysiological signature of PD is characterized by the loss of physiological levels of dopamine (DA) and the misfolding and aggregation of the alpha-synuclein (α-syn) protein, new proposals seek to restore the lost DA and inhibit the progressive damage derived from pathological α-syn and its impact in terms of oxidative stress. In this line, nanomedicine (the medical application of nanotechnology) has achieved significant advances in the development of nanocarriers capable of transporting and delivering basal state DA in a controlled manner in the tissues of interest, as well as highly selective catalytic nanostructures with enzyme-like properties for the elimination of reactive oxygen species (responsible for oxidative stress) and the proteolysis of misfolded proteins. Although some of these proposals remain in their early stages, the deepening of our knowledge concerning the pathological processes of PD and the advances in nanomedicine could endow for the development of potential treatments for this still incurable condition. Therefore, in this paper, we offer: (i) a brief summary of the most recent findings concerning the physiology of motor regulation and (ii) the molecular neuropathological processes associated with PD, together with (iii) a recapitulation of the current progress in controlled DA release by nanocarriers and (iv) the design of nanozymes, catalytic nanostructures with oxidoreductase-, chaperon, and protease-like properties. Finally, we conclude by describing the prospects and knowledge gaps to overcome and consider as research into nanotherapies for PD continues, especially when clinical translations take place.

Activity of a direct VTA to ventral pallidum GABA pathway encodes unconditioned reward value and sustains motivation for reward

Dopamine signaling from the ventral tegmental area (VTA) plays critical roles in reward-related behaviors, but less is known about the functions of neighboring VTA GABAergic neurons. We show here that a primary target of VTA GABA projection neurons is the ventral pallidum (VP). Activity of VTA-to-VP-projecting GABA neurons correlates consistently with size and palatability of the reward and does not change following cue learning, providing a direct measure of reward value. Chemogenetic stimulation of this GABA projection increased activity of a subset of VP neurons that were active while mice were seeking reward. Optogenetic stimulation of this pathway improved performance in a cue-reward task and maintained motivation to work for reward over days. This VTA GABA projection provides information about reward value directly to the VP, likely distinct from the prediction error signal carried by VTA dopamine neurons.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

Midbrain dopamine neurons signal phasic and ramping reward prediction error during goal-directed navigation

Goal-directed navigation requires learning to accurately estimate location and select optimal actions in each location. Midbrain dopamine neurons are involved in reward value learning and have been linked to reward location learning. They are therefore ideally placed to provide teaching signals for goal-directed navigation. By imaging dopamine neural activity as mice learned to actively navigate a closed-loop virtual reality corridor to obtain reward, we observe phasic and pre-reward ramping dopamine activity, which are modulated by learning stage and task engagement. A Q-learning model incorporating position inference recapitulates our results, displaying prediction errors resembling phasic and ramping dopamine neural activity. The model predicts that ramping is followed by improved task performance, which we confirm in our experimental data, indicating that the dopamine ramp may have a teaching effect. Our results suggest that midbrain dopamine neurons encode phasic and ramping reward prediction error signals to improve goal-directed navigation.

An Adaptive Motivation Approach to Understanding the 'How' and 'Why' of Wellbeing

A new model provides insight into the 'how' and 'why' of wellbeing to better understand the 'what'. Informed by evolutionary psychology and neuroscience, it proposes that systems for adaptive motivation underpin experiential and reflective wellbeing. The model proposes that the brain learns to predict situations, and errors arise between the predictions and experience. These prediction errors drive emotional experience, learning, motivation, decision-making, and the formation of wellbeing-relevant memories. The model differentiates four layers of wellbeing: objective, experiential, reflective, and narrative, which relate to the model in different ways. Constituents of wellbeing, human motives, and specific emotions integrate into the model. A simple computational implementation of the model reproduced several established wellbeing phenomena, including: the greater frequency of pleasant to unpleasant emotions, the stronger emotional salience of unpleasant emotions, hedonic adaptation to changes in circumstances, heritable influences on wellbeing, and affective forecasting errors. It highlights the importance of individual differences, and implies that high wellbeing will correlate with the experience of infrequent, routine, and predictable avoidance cues and frequent, varied, and novel approach cues. The model suggests that wellbeing arises directly from a system for adaptive motivation. This system functions like a mental dashboard that calls attention to situational changes and motivates the kinds of behaviours that gave humans a relative advantage in their ancestral environment. The model offers a set of fundamental principles and processes that may underlie diverse conceptualisations of wellbeing.

Bumblebees retrieve only the ordinal ranking of foraging options when comparing memories obtained in distinct settings

Are animals' preferences determined by absolute memories for options (e.g. reward sizes) or by their remembered ranking (better/worse)? The only studies examining this question suggest humans and starlings utilise memories for both absolute and relative information. We show that bumblebees' learned preferences are based only on memories of ordinal comparisons. A series of experiments showed that after learning to discriminate pairs of different flowers by sucrose concentration, bumblebees preferred flowers (in novel pairings) with (1) higher ranking over equal absolute reward, (2) higher ranking over higher absolute reward, and (3) identical qualitative ranking but different quantitative ranking equally. Bumblebees used absolute information in order to rank different flowers. However, additional experiments revealed that, even when ranking information was absent (i.e. bees learned one flower at a time), memories for absolute information were lost or could no longer be retrieved after at most 1 hr. Our results illuminate a divergent mechanism for bees (compared to starlings and humans) of learned preferences that may have arisen from different adaptations to their natural environment.

Neuroimaging oxytocin modulation of social reward learning in schizophrenia

BACKGROUND

Conventional pharmacological approaches have limited effectiveness for schizophrenia. There is interest in the application of oxytocin, which is involved in social cognition. Clinical trials have yielded mixed results, with a gap in understanding neural mechanisms.

AIMS

To evaluate the behavioural impact of oxytocin administration on a social learning task in individuals with schizophrenia, and elucidate any differential neural activity produced.

METHOD

We recruited 20 clinically stable right-handed men diagnosed with schizophrenia or schizoaffective disorder. In a double-blind cross-over randomised controlled study, 40 IU of oxytocin or placebo were administered before functional magnetic resonance imaging of participants playing a multi-round economic exchange game of trust. Participants had the role of investors (investment trials) receiving repayment on their investments (repayment trials), playing one session against a computer and a second against a player believed to be human.

RESULTS

During investment trials, oxytocin increased neural signalling in the right lateral parietal cortex for both human and computer player trials, and attenuated signalling in the right insula for human player trials. For repayment trials, oxytocin elicited signal increases in left insula and left ventral caudate, and a signal decrease in right amygdala during the human player trials; conversely it resulted in right dorsal caudate activation during the computer player trials. We did not find a significant change in behavioural performance associated with oxytocin administration, or any associations with symptoms.

CONCLUSIONS

During a social learning task oxytocin modulates cortical and limbic substrates of the reward-processing network. These perturbations can be putatively linked to the pathoaetiology of schizophrenia.

A framework for application of consumer neuroscience in pro-environmental behavior change interventions

Most consumers are aware that climate change is a growing problem and admit that action is needed. However, research shows that consumers' behavior often does not conform to their value and orientations. This value-behavior gap is due to contextual factors such as price, product design, and social norms as well as individual factors such as personal and hedonic values, environmental beliefs, and the workload capacity an individual can handle. Because of this conflict of interest, consumers have a hard time identifying the true drivers of their behavior, as they are either unaware of or unwilling to acknowledge the processes at play. Therefore, consumer neuroscience methods might provide a valuable tool to uncover the implicit measurements of pro-environmental behavior (PEB). Several studies have already defined neurophysiological differences between green and non-green individuals; however, a behavior change intervention must be developed to motivate PEB among consumers. Motivating behavior with reward or punishment will most likely get users engaged in climate change action via brain structures related to the reward system, such as the amygdala, nucleus accumbens, and (pre)frontal cortex, where the reward information and subsequent affective responses are encoded. The intensity of the reward experience can be increased when the consumer is consciously considering the action to achieve it. This makes goal-directed behavior the potential aim of behavior change interventions. This article provides an extensive review of the neuroscientific evidence for consumer attitude, behavior, and decision-making processes in the light of sustainability incentives for behavior change interventions. Based on this review, we aim to unite the current theories and provide future research directions to exploit the power of affective conditioning and neuroscience methods for promoting PEB engagement.

Relevance of interactions between dopamine and glutamate neurotransmission in schizophrenia

Dopamine (DA) and glutamate neurotransmission are strongly implicated in schizophrenia pathophysiology. While most studies focus on contributions of neurons that release only DA or glutamate, neither DA nor glutamate models alone recapitulate the full spectrum of schizophrenia pathophysiology. Similarly, therapeutic strategies limited to either system cannot effectively treat all three major symptom domains of schizophrenia: positive, negative, and cognitive symptoms. Increasing evidence suggests extensive interactions between the DA and glutamate systems and more effective treatments may therefore require the targeting of both DA and glutamate signaling. This offers the possibility that disrupting DA-glutamate circuitry between these two systems, particularly in the striatum and forebrain, culminate in schizophrenia pathophysiology. Yet, the mechanisms behind these interactions and their contributions to schizophrenia remain unclear. In addition to circuit- or system-level interactions between neurons that solely release either DA or glutamate, here we posit that functional alterations involving a subpopulation of neurons that co-release both DA and glutamate provide a novel point of integration between DA and glutamate systems, offering a key missing link in our understanding of schizophrenia pathophysiology. Better understanding of mechanisms underlying DA/glutamate co-release from these neurons may therefore shed new light on schizophrenia pathophysiology and lead to more effective therapeutics.

"Wanting" versus "needing" related value: An fMRI meta-analysis

Consumption and its excesses are sometimes explained by imbalance of need or lack of control over "wanting." "Wanting" assigns value to cues that predict rewards, whereas "needing" assigns value to biologically significant stimuli that one is deprived of. Here we aimed at studying how the brain activation patterns related to value of "wanted" stimuli differs from that of "needed" stimuli using activation likelihood estimation neuroimaging meta-analysis approaches. We used the perception of a cue predicting a reward for "wanting" related value and the perception of food stimuli in a hungry state as a model for "needing" related value. We carried out separate, contrasts, and conjunction meta-analyses to identify differences and similarities between "wanting" and "needing" values. Our overall results for "wanting" related value show consistent activation of the ventral tegmental area, striatum, and pallidum, regions that both activate behavior and direct choice, while for "needing" related value, we found an overall consistent activation of the middle insula and to some extent the caudal-ventral putamen, regions that only direct choice. Our study suggests that wanting has more control on consumption and behavioral activation.

Multistability, perceptual value, and internal foraging

Substantial experimental, theoretical, and computational insights into sensory processing have been derived from the phenomena of perceptual multistability-when two or more percepts alternate or switch in response to a single sensory input. Here, we review a range of findings suggesting that alternations can be seen as internal choices by the brain responding to values. We discuss how elements of external, experimenter-controlled values and internal, uncertainty- and aesthetics-dependent values influence multistability. We then consider the implications for the involvement in switching of regions, such as the anterior cingulate cortex, which are more conventionally tied to value-dependent operations such as cognitive control and foraging.

Comparative Study of the Neurotoxic Effects of Pregabalin Versus Tramadol in Rats

In Egypt, both pregabalin and tramadol misuse increased in the last decade. Although many studies have confirmed the neurotoxic effects of tramadol, those of pregabalin are understudied. The aim of the study is to evaluate the neurotoxic effects of pregabalin compared with tramadol. Thirty male albino rats were included in this experimental study, and they were randomly allocated into three equal groups: group I (normal saline), group II (tramadol misuse), and group III (pregabalin misuse). All rats received the commenced drugs for 1 month. Open field tests were performed on the day of scarification, and after that, cortical samples were taken for immunohistochemical analysis and quantification of dopamine receptors' gene expression. The drug misuse groups showed a significant decrease in weight gain at the end of the study. Open field testing showed the upper hand of controls regarding all of the tested parameters. Tramadol has a more negative impact on the locomotor parameters compared with pregabalin. Both drugs induced relatively low dopamine-1 receptor (D1Rs) expression to dopamine-2 receptors (D2Rs), mimicking the schizophrenia model. Both tramadol and pregabalin were associated with neurotoxic effects in male albino rats. These effects were less noticed with pregabalin. It is suggested that long-term abuse may end in psychosis.