Dopamine reward prediction-error signalling: a two-component response

A mismatch between striatal cholinergic pauses and dopaminergic reward prediction errors

Striatal acetylcholine and dopamine critically regulate movement, motivation, and reward-related learning. Pauses in cholinergic interneuron (CIN) firing are thought to coincide with dopamine pulses encoding reward prediction errors (RPE) to jointly enable synaptic plasticity. Here, we examine the firing of identified CINs during reward-guided decision-making in freely moving rats and compare this firing to dopamine release. Relationships between CINs, dopamine, and behavior varied strongly by subregion. In the dorsal-lateral striatum, a Go! cue evoked burst-pause CIN spiking, followed by a brief dopamine pulse that was unrelated to RPE. In the dorsal-medial striatum, this cue evoked only a CIN pause, that was curtailed by a movement-selective rebound in firing. Finally, in the ventral striatum, a reward cue evoked RPE-coding increases in both dopamine and CIN firing, without a consistent pause. Our results demonstrate a spatial and temporal dissociation between CIN pauses and dopamine RPE signals and will inform future models of striatal information processing under both normal and pathological conditions.

Dopamine and deep brain stimulation accelerate the neural dynamics of volitional action in Parkinson's disease

The ability to initiate volitional action is fundamental to human behaviour. Loss of dopaminergic neurons in Parkinson's disease is associated with impaired action initiation, also termed akinesia. Both dopamine and subthalamic deep brain stimulation (DBS) can alleviate akinesia, but the underlying mechanisms are unknown. An important question is whether dopamine and DBS facilitate de novo build-up of neural dynamics for motor execution or accelerate existing cortical movement initiation signals through shared modulatory circuit effects. Answering these questions can provide the foundation for new closed-loop neurotherapies with adaptive DBS, but the objectification of neural processing delays prior to performance of volitional action remains a significant challenge. To overcome this challenge, we studied readiness potentials and trained brain signal decoders on invasive neurophysiology signals in 25 DBS patients (12 female) with Parkinson's disease during performance of self-initiated movements. Combined sensorimotor cortex electrocorticography and subthalamic local field potential recordings were performed OFF therapy (n = 22), ON dopaminergic medication (n = 18) and on subthalamic deep brain stimulation (n = 8). This allowed us to compare their therapeutic effects on neural latencies between the earliest cortical representation of movement intention as decoded by linear discriminant analysis classifiers and onset of muscle activation recorded with electromyography. In the hypodopaminergic OFF state, we observed long latencies between motor intention and motor execution for readiness potentials and machine learning classifications. Both, dopamine and DBS significantly shortened these latencies, hinting towards a shared therapeutic mechanism for alleviation of akinesia. To investigate this further, we analysed directional cortico-subthalamic oscillatory communication with multivariate granger causality. Strikingly, we found that both therapies independently shifted cortico-subthalamic oscillatory information flow from antikinetic beta (13-35 Hz) to prokinetic theta (4-10 Hz) rhythms, which was correlated with latencies in motor execution. Our study reveals a shared brain network modulation pattern of dopamine and DBS that may underlie the acceleration of neural dynamics for augmentation of movement initiation in Parkinson's disease. Instead of producing or increasing preparatory brain signals, both therapies modulate oscillatory communication. These insights provide a link between the pathophysiology of akinesia and its' therapeutic alleviation with oscillatory network changes in other non-motor and motor domains, e.g. related to hyperkinesia or effort and reward perception. In the future, our study may inspire the development of clinical brain computer interfaces based on brain signal decoders to provide temporally precise support for action initiation in patients with brain disorders.

Storytelling changes the content and perceived value of event memories

Memories are not only stored for personal recall, but also to communicate knowledge to others in service of adaptive decision-making. Prior research shows that goals to share information can change which content is communicated in memory as well as the linguistic style embedded in this communication. Yet, little is known as to how communication-related alterations in memory narration drive differences of value processing in listeners. Here, we test how memory communication alters multi-featural recall for complex events and the downstream consequence on value estimations in naïve listeners. Participants recalled a memory of playing an exploratory videogame at a 24-h delay under instructions to either share (i.e., social condition) or recall (i.e., control condition) their memory. Sharing goals systematically altered the content and linguistic style of recall, such that narrators from the social condition were biased towards recall of non-episodic details and communicated their memories with more clout, less formality, and less authenticity. Across two independent samples of naïve listeners, these features differentially influenced value estimations of the video game. We found that greater clout was associated with greater enjoyment while listening to memories (hedonic value), and that greater inclusion of non-episodic details resulted in greater willingness to purchase the video game (motivational drive). These findings indicate that sharing an experience as a story can change the content and linguistic tone of memory recall, which in turn shape perceived value in naïve listeners.

Dopamine-mediated formation of a memory module in the nucleus accumbens for goal-directed navigation

Spatial memories guide navigation efficiently toward desired destinations. However, the neuronal and circuit mechanisms underlying the encoding of goal locations and its translation into goal-directed navigation remain unclear. Here we demonstrate that mice rapidly form a spatial memory of a shelter during shelter experiences, guiding escape behavior toward the goal location-a shelter-when under threat. Dopaminergic neurons in the ventral tegmental area and their projection to the nucleus accumbens (NAc) encode safety signals associated with the shelter. Optogenetically induced phasic dopamine signals are sufficient to create a place memory that directs escape navigation. Converging dopaminergic and hippocampal glutamatergic inputs to the NAc mediate the formation of a goal-related memory within a subpopulation of NAc neurons during shelter experiences. Artificial co-activation of this goal-related NAc ensemble with neurons in the dorsal periaqueductal gray was sufficient to trigger memory-guided, rather than random, escape behavior. These findings provide causal evidence of cognitive circuit modules linking memory with goal-directed action.

Distributed midbrain responses signal the content of positive identity prediction errors

Recent work across species has shown that midbrain dopamine neurons signal not only errors in the prediction of reward value but also in the prediction of value-neutral sensory features. To support learning of associative structures in downstream areas, identity prediction errors (iPEs) should signal specific information about the mis-predicted outcome. Here, we used pattern-based analysis of functional magnetic resonance imaging (fMRI) data acquired during reversal learning to characterize the information content of iPE responses in the human midbrain. We find that fMRI responses to value-neutral identity errors contain information about the identity of the unexpectedly received reward (positive iPE+) but not about the identity of the omitted reward (negative iPE-). Exploratory analyses revealed representations of iPE- in the dorsomedial prefrontal cortex. These results demonstrate that ensemble midbrain responses to value-neutral identity errors convey information about the identity of unexpectedly received outcomes, which could shape the formation of novel stimulus-outcome associations that constitute cognitive maps.

The direct and indirect pathways of the basal ganglia antagonistically influence cortical activity and perceptual decisions

The striatum, the main input nucleus of the basal ganglia, receives topographically organized input from the cortex and gives rise to the direct and indirect output pathways, which have antagonistic effects on basal ganglia output directed to the cortex. We optogenetically stimulated the direct and indirect pathways in a visual and a working memory task in mice that responded by licking. Unilateral direct pathway stimulation increased the probability of lick responses toward the contralateral, non-stimulated side and increased cortical activity globally. In contrast, indirect pathway stimulation increased the probability of responses toward the stimulated side and decreased activity in the stimulated hemisphere. Moreover, direct pathway stimulation enhanced the neural representation of a contralateral visual stimulus during the delay of the working memory task, whereas indirect pathway stimulation had the opposite effect. Our results demonstrate how these two pathways influence perceptual decisions and working memory and modify activity in the dorsal cortex.

Gambling on others' health: risky pro-social decision-making in the era of COVID-19

Introduction

In the early days of the COVID-19 pandemic, individuals were asked to perform costly actions to reduce harm to strangers, even while the general population, including authorities and experts, grappled with the uncertainty surrounding thenovel virus. Many studies have examined health decision-making by experts, but the study of lay, non-expert, individual decision-making on a stranger's health has been left to the wayside, as ordinary citizens are usually not tasked with such decisions.

Methods

We sought to capture a snapshot of this specific choice behavior by administering two surveys to the general population in the spring of 2020, when much of the global community was subject to COVID-19-related restrictions, as well as uncertainty surrounding the virus. We presented study participants with fictitious diseases varying in severity that threatened oneself, a loved one or a stranger. Participants were asked to choose between treatment options that could either provide a sure, but mild improvement (sure option) or cure the affected person at a given probability of success (risky option).

Results

Respondents preferred gambles overall, but risk-seeking decreased progressively with higher expected severity of disease. This pattern was observed regardless of the recipient's identity. Distinctions between targets emerged however whendecisions were conditioned on a treatment's monetary cost, with participants preferring cheaper options for strangers.

Discussion

Overall, these findings provide a descriptive model of individual decision-making under risk for others; and inform on the limits of what can be asked of an individual in service to a stranger.

Dopamine neurons encode trial-by-trial subjective reward value in an auction-like task

The dopamine reward prediction error signal is known to be subjective but has so far only been assessed in aggregate choices. However, personal choices fluctuate across trials and thus reflect the instantaneous subjective reward value. In the well-established Becker-DeGroot-Marschak (BDM) auction-like mechanism, participants are encouraged to place bids that accurately reveal their instantaneous subjective reward value; inaccurate bidding results in suboptimal reward ("incentive compatibility"). In our experiment, male rhesus monkeys became experienced over several years to place accurate BDM bids for juice rewards without specific external constraints. Their bids for physically identical rewards varied trial by trial and increased overall for larger rewards. In these highly experienced animals, responses of midbrain dopamine neurons followed the trial-by-trial variations of bids despite constant, explicitly predicted reward amounts. Inversely, dopamine responses were similar with similar bids for different physical reward amounts. Support Vector Regression demonstrated accurate prediction of the animals' bids by as few as twenty dopamine neurons. Thus, the phasic dopamine reward signal reflects instantaneous subjective reward value.

A unified theoretical framework underlying the regulation of motivated behavior

To orchestrate behaviors for survival, multiple psychological components have evolved. The current theories do not clearly distinguish the distinct components. In this article, we provide a unified theoretical framework. To optimize survival, there should be four components; (1) "need", an alarm based on a predicted deficiency. (2) "motivation", a direct behavior driver. (3) "pleasure", a teacher based on immediate outcomes. (4) "utility", a teacher based on final delayed outcomes. For behavior stability, need should be accumulated into motivation to drive behavior. Based on the immediate outcome of the behavior, the pleasure should teach whether to continue the current behavior. Based on the final delay outcome, the utility should teach whether to increase future behavior by reshaping the other three components. We provide several neural substrate candidates in the food context. The proposed theoretical framework, in combination with appropriate experiments, will unravel the neural components responsible for each theoretical component.

The nucleus accumbens in reward and aversion processing: insights and implications

The nucleus accumbens (NAc), a central component of the brain's reward circuitry, has been implicated in a wide range of behaviors and emotional states. Emerging evidence, primarily drawing from recent rodent studies, suggests that the function of the NAc in reward and aversion processing is multifaceted. Prolonged stress or drug use induces maladaptive neuronal function in the NAc circuitry, which results in pathological conditions. This review aims to provide comprehensive and up-to-date insights on the role of the NAc in motivated behavior regulation and highlights areas that demand further in-depth analysis. It synthesizes the latest findings on how distinct NAc neuronal populations and pathways contribute to the processing of opposite valences. The review examines how a range of neuromodulators, especially monoamines, influence the NAc's control over various motivational states. Furthermore, it delves into the complex underlying mechanisms of psychiatric disorders such as addiction and depression and evaluates prospective interventions to restore NAc functionality.

Chronic stress deficits in reward behaviour co-occur with low nucleus accumbens dopamine activity during reward anticipation specifically

Whilst reward pathologies are major and common in stress-related neuropsychiatric disorders, their neurobiology and treatment are poorly understood. Imaging studies in human reward pathology indicate attenuated BOLD activity in nucleus accumbens (NAc) coincident with reward anticipation but not reinforcement; potentially, this is dopamine (DA) related. In mice, chronic social stress (CSS) leads to reduced reward learning and motivation. Here, DA-sensor fibre photometry is used to investigate whether these behavioural deficits co-occur with altered NAc DA activity during reward anticipation and/or reinforcement. In CSS mice relative to controls: (1) Reduced discriminative learning of the sequence, tone-on + appetitive behaviour = tone-on + sucrose reinforcement, co-occurs with attenuated NAc DA activity throughout tone-on and sucrose reinforcement. (2) Reduced motivation during the sequence, operant behaviour = tone-on + sucrose delivery + sucrose reinforcement, co-occurs with attenuated NAc DA activity at tone-on and typical activity at sucrose reinforcement. (3) Reduced motivation during the sequence, operant behaviour = appetitive behaviour + sociosexual reinforcement, co-occurs with typical NAc DA activity at female reinforcement. Therefore, in CSS mice, low NAc DA activity co-occurs with low reward anticipation and could account for deficits in learning and motivation, with important implications for understanding human reward pathology.

Reward processes in extinction learning and applications to exposure therapy

Anxiety disorders are common and highly distressing mental health conditions. Exposure therapy is a gold-standard treatment for anxiety disorders. Mechanisms of Pavlovian fear learning, and particularly fear extinction, are central to exposure therapy. A growing body of evidence suggests an important role of reward processes during Pavlovian fear extinction. Nonetheless, predominant models of exposure therapy do not currently incorporate reward processes. Herein, we present a theoretical model of reward processes in relation to Pavlovian mechanisms of exposure therapy, including a focus on dopaminergic prediction error signaling, coinciding positive emotional experiences (i.e., relief), and unexpected positive outcomes. We then highlight avenues for further research and discuss potential strategies to leverage reward processes to maximize exposure therapy response, such as pre-exposure interventions to increase reward sensitivity or post-exposure rehearsal (e.g., savoring, imaginal recounting strategies) to enhance retrieval and retention of learned associations.

Experience-dependent regulation of dopaminergic signaling in the somatosensory cortex

Dopamine critically influences reward processing, sensory perception, and motor control. Yet, the modulation of dopaminergic signaling by sensory experiences is not fully delineated. Here, by manipulating sensory experience using bilateral single-row whisker deprivation, we demonstrated that gene transcription in the dopaminergic signaling pathway (DSP) undergoes experience-dependent plasticity in both granular and supragranular layers of the primary somatosensory (barrel) cortex (S1). Sensory experience and deprivation compete for the regulation of DSP transcription across neighboring cortical columns, and sensory deprivation-induced changes in DSP are topographically constrained. These changes in DSP extend beyond cortical map plasticity and influence neuronal information processing. Pharmacological regulation of D2 receptors, a key component of DSP, revealed that D2 receptor activation suppresses excitatory neuronal excitability, hyperpolarizes the action potential threshold, and reduces the instantaneous firing rate. These findings suggest that the dopaminergic drive originating from midbrain dopaminergic neurons, targeting the sensory cortex, is subject to experience-dependent regulation and might create a regulatory feedback loop for modulating sensory processing. Finally, using topological gene network analysis and mutual information, we identify the molecular hubs of experience-dependent plasticity of DSP. These findings provide new insights into the mechanisms by which sensory experience shapes dopaminergic signaling in the brain and might help unravel the sensory deficits observed after dopamine depletion.

Distinct subcircuits within the mesolimbic dopamine system encode the salience and valence of social stimuli

The mesolimbic dopamine (DA) system (MDS) is the canonical "reward" pathway that has been studied extensively in the context of the rewarding properties of sex, food, and drugs of abuse. In contrast, very little is known about the role of the MDS in the processing of the rewarding and aversive properties of social stimuli. Social interactions can be characterized by their salience (i.e., importance) and their rewarding or aversive properties (i.e., valence). Here, we test the novel hypothesis that projections from the medial ventral tegmental area (VTA) to the nucleus accumbens (NAc) core codes for the salience of social stimuli through the phasic release of DA in response to both rewarding and aversive social stimuli. In contrast, we hypothesize that projections from the lateral VTA to the NAc shell codes for the rewarding properties of social stimuli by increasing the tonic release of DA and the aversive properties of social stimuli by reducing the tonic release of DA. Using DA amperometry, which monitors DA signaling with a high degree of temporal and anatomical resolution, we measured DA signaling in the NAc core or shell while rewarding and aversive social interactions were taking place. These findings, as well as additional anatomical and functional studies, provide strong support for the proposed neural circuitry underlying the response of the MDS to social stimuli. Together, these data provide a novel conceptualization of how the functional and anatomical heterogeneity within the MDS detect and distinguish between social salience, social reward, and social aversion.

Significance Statement

Social interactions of both positive and negative valence are highly salient stimuli that profoundly impact social behavior and social relationships. Although DA projections from the VTA to the NAc are involved in reward and aversion little is known about their role in the saliency and valence of social stimuli. Here, we report that DA projections from the mVTA to the NAc core signal the salience of social stimuli, whereas projections from the lVTA to the NAc shell signal valence of social stimuli. This work extends our current understanding of the role of DA in the MDS by characterizing its subcircuit connectivity and associated function in the processing of rewarding and aversive social stimuli.

Striosomes Target Nigral Dopamine-Containing Neurons via Direct-D1 and Indirect-D2 Pathways Paralleling Classic Direct-Indirect Basal Ganglia Systems

Balanced activity of canonical direct D1 and indirect D2 basal ganglia pathways is considered a core requirement for normal movement, and their imbalance is an etiologic factor in movement and neuropsychiatric disorders. We present evidence for a conceptually equivalent pair of direct-D1 and indirect-D2 pathways that arise from striatal projection neurons (SPNs) of the striosome compartment rather than from SPNs of the matrix, as do the canonical pathways. These S-D1 and S-D2 striosomal pathways target substantia nigra dopamine-containing neurons instead of basal ganglia motor output nuclei. They modulate movement oppositely to the modulation by the canonical pathways: S-D1 is inhibitory and S-D2 is excitatory. The S-D1 and S-D2 circuits likely influence motivation for learning and action, complementing and reorienting canonical pathway modulation. A major conceptual reformulation of the classic direct-indirect pathway model of basal ganglia function is needed, as well as reconsideration of the effects of D2-targeting therapeutic drugs.

Dopamine-mediated interactions between short- and long-term memory dynamics

In dynamic environments, animals make behavioural decisions on the basis of the innate valences of sensory cues and information learnt about these cues across multiple timescales1-3. However, it remains unclear how the innate valence of a sensory stimulus affects the acquisition of learnt valence information and subsequent memory dynamics. Here we show that in the Drosophila brain, interconnected short- and long-term memory units of the mushroom body jointly regulate memory through dopamine signals that encode innate and learnt sensory valences. By performing time-lapse in vivo voltage-imaging studies of neural spiking in more than 500 flies undergoing olfactory associative conditioning, we found that protocerebral posterior lateral 1 dopamine neurons (PPL1-DANs)4 heterogeneously and bidirectionally encode innate and learnt valences of punishment, reward and odour cues. During learning, these valence signals regulate memory storage and extinction in mushroom body output neurons (MBONs)5. During initial conditioning bouts, PPL1-γ1pedc and PPL1-γ2α'1 neurons control short-term memory formation, which weakens inhibitory feedback from MBON-γ1pedc>α/β to PPL1-α'2α2 and PPL1-α3. During further conditioning, this diminished feedback allows these two PPL1-DANs to encode the net innate plus learnt valence of the conditioned odour cue, which gates long-term memory formation. A computational model constrained by the fly connectome6,7 and our spiking data explains how dopamine signals mediate the circuit interactions between short- and long-term memory traces, yielding predictions that our experiments confirmed. Overall, the mushroom body achieves flexible learning through the integration of innate and learnt valences in parallel learning units sharing feedback interconnections. This hybrid physiological-anatomical mechanism may be a general means by which dopamine regulates memory dynamics in other species and brain structures, including the vertebrate basal ganglia.

High-Precision Optical Fiber-Based Lickometer

Quantifying and analyzing licking behavior can offer valuable insights into fundamental neurobiological mechanisms controlling animal consummatory behaviors. Lickometers are typically based on electrical properties, a strategy that comes with limitations, including susceptibility to electrical interference and generation of electrical disturbances in electrophysiological measurements. While optical lickometers offer an alternative method to measure licks and quantify fluid intake in animals, they are prone to false readings and susceptibility to outside light sources. To overcome this problem, we propose a low-cost open-source lickometer that combines a restricted infrared beam defined by optical fibers, with a poke design that allows easy access to the tongue while limiting access of other body parts and external light sources. This device also includes features for detecting nose pokes and presenting visual cues during behavioral tasks. We provide validation experiments that demonstrate the optical lickometer's reliability, high-sensitivity and precision, and its application in a behavioral task, showcasing the potential of this tool to study lick microstructure in combination with other techniques, such as imaging of neural activity, in freely moving mice.

Dopamine and acetylcholine have distinct roles in delay- and effort-based decision-making in humans

In everyday life, we encounter situations that require tradeoffs between potential rewards and associated costs, such as time and (physical) effort. The literature indicates a prominent role for dopamine in discounting of both delay and effort, with mixed findings for delay discounting in humans. Moreover, the reciprocal antagonistic interaction between dopaminergic and cholinergic transmission in the striatum suggests a potential opponent role of acetylcholine in these processes. We found opposing effects of dopamine D2 (haloperidol) and acetylcholine M1 receptor (biperiden) antagonism on specific components of effort-based decision-making in healthy humans: haloperidol decreased, whereas biperiden increased the willingness to exert physical effort. In contrast, delay discounting was reduced under haloperidol, but not affected by biperiden. Together, our data suggest that dopamine, acting at D2 receptors, modulates both effort and delay discounting, while acetylcholine, acting at M1 receptors, appears to exert a more specific influence on effort discounting only.

Pharmacological Enhancement of Dopamine Neurotransmission Does Not Affect Illusory Pattern Perception

Psychotic symptoms and delusional beliefs have been linked to dopamine transmission in both healthy and clinical samples and are assumed to result at least in part from perceiving illusory patterns in noise. However, the existing literature on the role of dopamine in detecting patterns in noise is inconclusive. To address this issue, we assessed the effect of manipulating dopaminergic neurotransmission on illusory pattern perception in healthy individuals (n = 48, n = 19 female) in a double-blind placebo-controlled within-subjects design (see preregistration at https://osf.io/a4k9j/). We predicted individuals on versus off ʟ-DOPA to be more likely to perceive illusory patterns, specifically objects in images containing only noise. Using a signal detection model, however, we found no credible evidence that ʟ-DOPA compared with placebo increased false alarm rates. Further, ʟ-DOPA did not reliably modulate measures of accuracy, discrimination sensitivity, and response bias. In all cases, Bayesian statistics revealed strong evidence in favor of the null hypothesis. The task design followed previous work on illusory pattern perception and comprised a limited number of items per condition. The results therefore need to be interpreted with caution, as power was limited. Future studies should address illusory pattern perception using more items and take into account potential dose-dependent effects and differential effects in healthy versus clinical samples.

Development of a genetically encoded sensor for probing endogenous nociceptin opioid peptide release

Nociceptin/orphanin-FQ (N/OFQ) is a recently appreciated critical opioid peptide with key regulatory functions in several central behavioral processes including motivation, stress, feeding, and sleep. The functional relevance of N/OFQ action in the mammalian brain remains unclear due to a lack of high-resolution approaches to detect this neuropeptide with appropriate spatial and temporal resolution. Here we develop and characterize NOPLight, a genetically encoded sensor that sensitively reports changes in endogenous N/OFQ release. We characterized the affinity, pharmacological profile, spectral properties, kinetics, ligand selectivity, and potential interaction with intracellular signal transducers of NOPLight in vitro. Its functionality was established in acute brain slices by exogeneous N/OFQ application and chemogenetic induction of endogenous N/OFQ release from PNOC neurons. In vivo studies with fibre photometry enabled direct recording of NOPLight binding to exogenous N/OFQ receptor ligands, as well as detection of endogenous N/OFQ release within the paranigral ventral tegmental area (pnVTA) during natural behaviors and chemogenetic activation of PNOC neurons. In summary, we show here that NOPLight can be used to detect N/OFQ opioid peptide signal dynamics in tissue and freely behaving animals.

Neural connectivity patterns explain why adolescents perceive the world as moving slow

That younger individuals perceive the world as moving slower than adults is a familiar phenomenon. Yet, it remains an open question why that is. Using event segmentation theory, electroencephalogram (EEG) beamforming and nonlinear causal relationship estimation using artificial neural network methods, we studied neural activity while adolescent and adult participants segmented a movie. We show when participants were instructed to segment a movie into meaningful units, adolescents partitioned incoming information into fewer encapsulated segments or episodes of longer duration than adults. Importantly, directed communication between medial frontal and lower-level perceptual areas and between occipito-temporal regions in specific neural oscillation spectrums explained behavioral differences between groups. Overall, the study reveals that a different organization of directed communication between brain regions and inefficient transmission of information between brain regions are key to understand why younger people perceive the world as moving slow.

Generalized cue reactivity in dopamine neurons after opioids

Cue reactivity is the maladaptive neurobiological and behavioral response upon exposure to drug cues and is a major driver of relapse. The leading hypothesis is that dopamine release by addictive drugs represents a persistently positive reward prediction error that causes runaway enhancement of dopamine responses to drug cues, leading to their pathological overvaluation compared to non-drug reward alternatives. However, this hypothesis has not been directly tested. Here we developed Pavlovian and operant procedures to measure firing responses, within the same dopamine neurons, to drug versus natural reward cues, which we found to be similarly enhanced compared to cues predicting natural rewards in drug-naïve controls. This enhancement was associated with increased behavioral reactivity to the drug cue, suggesting that dopamine release is still critical to cue reactivity, albeit not as previously hypothesized. These results challenge the prevailing hypothesis of cue reactivity, warranting new models of dopaminergic function in drug addiction, and provide critical insights into the neurobiology of cue reactivity with potential implications for relapse prevention.

A transdisciplinary view on curiosity beyond linguistic humans: animals, infants, and artificial intelligence

Curiosity is a core driver for life-long learning, problem-solving and decision-making. In a broad sense, curiosity is defined as the intrinsically motivated acquisition of novel information. Despite a decades-long history of curiosity research and the earliest human theories arising from studies of laboratory rodents, curiosity has mainly been considered in two camps: 'linguistic human' and 'other'. This is despite psychology being heritable, and there are many continuities in cognitive capacities across the animal kingdom. Boundary-pushing cross-disciplinary debates on curiosity are lacking, and the relative exclusion of pre-linguistic infants and non-human animals has led to a scientific impasse which more broadly impedes the development of artificially intelligent systems modelled on curiosity in natural agents. In this review, we synthesize literature across multiple disciplines that have studied curiosity in non-verbal systems. By highlighting how similar findings have been produced across the separate disciplines of animal behaviour, developmental psychology, neuroscience, and computational cognition, we discuss how this can be used to advance our understanding of curiosity. We propose, for the first time, how features of curiosity could be quantified and therefore studied more operationally across systems: across different species, developmental stages, and natural or artificial agents.

Dopaminergic lesions of the anterior cingulate cortex of rats increase vulnerability to salient distractors

The anterior cingulate cortex (ACC) has been shown to be critical to many aspects of executive function including filtering irrelevant information, updating response contingencies when reinforcement contingencies change and stabilizing task sets. Nonspecific lesions to this region in rats produce a vulnerability to distractors that have gained salience through prior associations with reinforcement. These lesions also exacerbate cognitive fatigue in tests of sustained attention but do not produce global attentional impairments nor do they produce distractibility to novel distractors that do not have a prior association with reinforcement. To determine the neurochemical basis of these cognitive impairments, dopaminergically selective lesions of the ACC were made in both male and female Long-Evans, hooded rats prior to assessment in two attentional tasks. Dopaminergic lesions of the ACC increase the vulnerability of subjects to previously reinforced distractors and impede formation of an attentional set. Lesioned rats were not more susceptible to the effects of novel, irrelevant stimuli in a test of sustained attention as has been previously shown. Additionally, the effects of dopaminergic lesions were found to differ based on sex. Lesioned female, but not male, rats were more vulnerable than sham-lesioned females to the effects of prolonged testing and the removal of reinforcement during a test of sustained attention. Together, these data support the hypothesis that dopamine in the ACC is critical to filtering distractors whose salience has been gained through reinforcement.

Brain Mechanisms Underlying Panic Attack and Panic Disorder

Panic disorder is a psychiatric disorder characterized by recurrent panic attacks, with a prevalence of ~ 4% in the general population, causing heavy personal and socioeconomic burdens. The similarities of animal defense responses to clinical panic attack symptoms in humans make it possible to translate neuroanatomical pathways identified in animal studies to panic disorder in humans. Therefore, in this review we first present a basic overview of panic disorder in humans including the main subtypes, models commonly used to trigger panic attacks, related hypotheses, the neurotransmitter systems that may be involved, and the current clinical treatments to give the reader a comprehensive understanding of panic disorder. The animal section introduces the models that trigger panic-like behavior in animals and the brain regions that may be involved, providing insights for future elucidation of the neural circuit mechanisms behind panic attacks.

Endogenous opioids in the olfactory tubercle and their roles in olfaction and quality of life

Olfactory dysfunctions decrease daily quality of life (QOL) in part by reducing the pleasure of eating. Olfaction plays an essential role in flavor sensation and palatability. The decreased QOL due to olfactory dysfunction is speculated to result from abnormal neural activities in the olfactory and limbic areas of the brain, as well as peripheral odorant receptor dysfunctions. However, the specific underlying neurobiological mechanisms remain unclear. As the olfactory tubercle (OT) is one of the brain's regions with high expression of endogenous opioids, we hypothesize that the mechanism underlying the decrease in QOL due to olfactory dysfunction involves the reduction of neural activity in the OT and subsequent endogenous opioid release in specialized subregions. In this review, we provide an overview and recent updates on the OT, the endogenous opioid system, and the pleasure systems in the brain and then discuss our hypothesis. To facilitate the effective treatment of olfactory dysfunctions and decreased QOL, elucidation of the neurobiological mechanisms underlying the pleasure of eating through flavor sensation is crucial.

Adaptive Responding to Stimulus-Outcome Associations Requires Noradrenergic Transmission in the Medial Prefrontal Cortex

A dynamic environment, such as the one we inhabit, requires organisms to continuously update their knowledge of the setting. While the prefrontal cortex is recognized for its pivotal role in regulating such adaptive behavior, the specific contribution of each prefrontal area remains elusive. In the current work, we investigated the direct involvement of two major prefrontal subregions, the medial prefrontal cortex (mPFC, A32D + A32V) and the orbitofrontal cortex (OFC, VO + LO), in updating pavlovian stimulus-outcome (S-O) associations following contingency degradation in male rats. Specifically, animals had to learn that a particular cue, previously fully predicting the delivery of a specific reward, was no longer a reliable predictor. First, we found that chemogenetic inhibition of mPFC, but not of OFC, neurons altered the rats' ability to adaptively respond to degraded and non-degraded cues. Next, given the growing evidence pointing at noradrenaline (NA) as a main neuromodulator of adaptive behavior, we decided to investigate the possible involvement of NA projections to the two subregions in this higher-order cognitive process. Employing a pair of novel retrograde vectors, we traced NA projections from the locus ceruleus (LC) to both structures and observed an equivalent yet relatively segregated amount of inputs. Then, we showed that chemogenetic inhibition of NA projections to the mPFC, but not to the OFC, also impaired the rats' ability to adaptively respond to the degradation procedure. Altogether, our findings provide important evidence of functional parcellation within the prefrontal cortex and point at mPFC NA as key for updating pavlovian S-O associations.

Development of a genetically encoded sensor for probing endogenous nociceptin opioid peptide release

Nociceptin/orphanin-FQ (N/OFQ) is a recently appreciated critical opioid peptide with key regulatory functions in several central behavioral processes including motivation, stress, feeding, and sleep. The functional relevance of N/OFQ action in the mammalian brain remains unclear due to a lack of high-resolution approaches to detect this neuropeptide with appropriate spatial and temporal resolution. Here we develop and characterize NOPLight, a genetically encoded sensor that sensitively reports changes in endogenous N/OFQ release. We characterized the affinity, pharmacological profile, spectral properties, kinetics, ligand selectivity, and potential interaction with intracellular signal transducers of NOPLight in vitro. Its functionality was established in acute brain slices by exogeneous N/OFQ application and chemogenetic induction of endogenous N/OFQ release from PNOC neurons. In vivo studies with fibre photometry enabled direct recording of NOPLight binding to exogenous N/OFQ receptor ligands, as well as detection of endogenous N/OFQ release within the paranigral ventral tegmental area (pnVTA) during natural behaviors and chemogenetic activation of PNOC neurons. In summary, we show here that NOPLight can be used to detect N/OFQ opioid peptide signal dynamics in tissue and freely behaving animals.

Ventral tegmental area dopamine projections to the hippocampus trigger long-term potentiation and contextual learning

In most models of neuronal plasticity and memory, dopamine is thought to promote the long-term maintenance of Long-Term Potentiation (LTP) underlying memory processes, but not the initiation of plasticity or new information storage. Here, we used optogenetic manipulation of midbrain dopamine neurons in male DAT::Cre mice, and discovered that stimulating the Schaffer collaterals - the glutamatergic axons connecting CA3 and CA1 regions - of the dorsal hippocampus concomitantly with midbrain dopamine terminals within a 200 millisecond time-window triggers LTP at glutamatergic synapses. Moreover, we showed that the stimulation of this dopaminergic pathway facilitates contextual learning in awake behaving mice, while its inhibition hinders it. Thus, activation of midbrain dopamine can operate as a teaching signal that triggers NeoHebbian LTP and promotes supervised learning.

An in vivo Dissection, and Analysis of Socio-Affective Symptoms related to Cerebellum-Midbrain Reward Circuitry in Humans

Emerging research in non-human animals implicates cerebellar projections to the ventral tegmental area (VTA) in appetitive behaviors, but these circuits have not been characterized in humans. Here, we mapped cerebello-VTA white-matter connectivity in humans using probabilistic tractography on diffusion imaging data from the Human Connectome Project. We uncovered the topographical organization of these connections by separately tracking from parcels of cerebellar lobule VI, crus I/II, vermis, paravermis, and cerebrocerebellum. Results revealed that connections from the cerebellum to the VTA predominantly originate in the right hemisphere, interposed nucleus, and paravermal cortex, and terminate mostly ipsilaterally. Paravermal crus I sends the most connections to the VTA compared to other lobules. We discovered a medial-to-lateral gradient of connectivity, such that the medial cerebellum has the highest connectivity with the VTA. Individual differences in microstructure were associated with measures of negative affect and social functioning. By splitting the tracts into quarters, we found that the socio-affective effects were driven by the third quarter of the tract, corresponding to the point at which the fibers leave the deep nuclei. Taken together, we produced detailed maps of cerebello-VTA structural connectivity for the first time in humans and established their relevance for trait differences in socio-affective regulation.

Developmental and adult striatal patterning of nociceptin ligand marks striosomal population with direct dopamine projections

Circuit influences on the midbrain dopamine system are crucial to adaptive behavior and cognition. Recent developments in the study of neuropeptide systems have enabled high-resolution investigations of the intersection of neuromodulatory signals with basal ganglia circuitry, identifying the nociceptin/orphanin FQ (N/OFQ) endogenous opioid peptide system as a prospective regulator of striatal dopamine signaling. Using a prepronociceptin-Cre reporter mouse line, we characterized highly selective striosomal patterning of Pnoc mRNA expression in mouse dorsal striatum, reflecting early developmental expression of Pnoc . In the ventral striatum, Pnoc expression was was clustered across the nucleus accumbens core and medial shell, including in adult striatum. We found that Pnoc tdTomato reporter cells largely comprise a population of dopamine receptor D1 ( Drd1 ) expressing medium spiny projection neurons localized in dorsal striosomes, known to be unique among striatal projections neurons for their direct innervation of midbrain dopamine neurons. These findings provide new understanding of the intersection of the N/OFQ system among basal ganglia circuits with particular implications for developmental regulation or wiring of striatal-nigral circuits.

Changes in the dopaminergic circuitry and adult neurogenesis linked to reinforcement learning in corvids

The caudolateral nidopallium (NCL, an analog of the prefrontal cortex) is known to be involved in learning, memory, and discrimination in corvids (a songbird), whereas the involvement of other brain regions in these phenomena is not well explored. We used house crows (Corvus splendens) to explore the neural correlates of learning and decision-making by initially training them on a shape discrimination task followed by immunohistochemistry to study the immediate early gene expression (Arc), a dopaminoceptive neuronal marker (DARPP-32, Dopamine- and cAMP-regulated phosphoprotein, Mr 32 kDa) to understand the involvement of the reward pathway and an immature neuronal marker (DCX, doublecortin) to detect learning-induced changes in adult neurogenesis. We performed neuronal counts and neuronal tracing, followed by morphometric analyses. Our present results have demonstrated that besides NCL, other parts of the caudal nidopallium (NC), avian basal ganglia, and intriguingly, vocal control regions in house crows are involved in visual discrimination. We have also found that training on the visual discrimination task can be correlated with neurite pruning in mature dopaminoceptive neurons and immature DCX-positive neurons in the NC of house crows. Furthermore, there is an increase in the incorporation of new neurons throughout NC and the medial striatum which can also be linked to learning. For the first time, our results demonstrate that a combination of structural changes in mature and immature neurons and adult neurogenesis are linked to learning in corvids.

To lie or to tell the truth? The influence of processing the opponent's feedback on the forthcoming choice

Introduction

The brain mechanisms of deceptive behavior are relatively well studied, and the key brain regions involved in its processing were established. At the same time, the brain mechanisms underlying the processes of preparation for deception are less known.

Methods

We studied BOLD-signal changes during the presentation of the opponent's feedback to a previous deceptive or honest action during the computer game. The goal of the game was to mislead the opponent either by means of deception or by means of telling the truth.

Results

As a result, it was shown that several brain regions that were previously demonstrated as involved in deception execution, such as the left anterior cingulate cortex and anterior insula, also underlie processes related to deception preparation.

Discussion

The results obtained also allowed us to suggest that brain regions responsible for performance monitoring, intention assessment, suppression of non-selected solutions, and reward processing could be involved in shaping future action selection and preparation for deception. By shedding light on the brain mechanisms underlying deception, our study contributes to a deeper understanding of this complex cognitive process. Furthermore, it emphasizes the significance of exploring brain mechanisms governing the choice between deception and truth at various stages of decision-making.

Acetylcholine demixes heterogeneous dopamine signals for learning and moving

Midbrain dopamine neurons promote reinforcement learning and movement vigor. A major outstanding question is how dopamine-recipient neurons in the striatum parse these heterogeneous signals. Here we characterized dopamine and acetylcholine release in the dorsomedial striatum (DMS) of rats performing a decision-making task. We found that dopamine acted as a reward prediction error (RPE), modulating behavior and DMS spiking on subsequent trials when coincident with pauses in cholinergic release. In contrast, at task events that elicited coincident bursts of acetylcholine and dopamine, dopamine preceded contralateral movements and predicted movement vigor without inducing plastic changes in DMS firing rates. Our findings provide a circuit-level mechanism by which cholinergic modulation allows the same dopamine signals to be used for either movement or learning depending on instantaneous behavioral context.

Paradoxical Boosting of Weak and Strong Spatial Memories by Hippocampal Dopamine Uncaging

The ability to remember changes in the surroundings is fundamental for daily life. It has been proposed that novel events producing dopamine release in the hippocampal CA1 region could modulate spatial memory formation. However, the role of hippocampal dopamine increase on weak or strong spatial memories remains unclear. We show that male mice exploring two objects located in a familiar environment for 5 min created a short-term memory (weak) that cannot be retrieved 1 d later, whereas 10 min exploration created a long-term memory (strong) that can be retrieved 1 d later. Remarkably, hippocampal dopamine elevation during the encoding of weak object location memories (OLMs) allowed their retrieval 1 d later but dopamine elevation during the encoding of strong OLMs promoted the preference for a familiar object location over a novel object location after 24 h. Moreover, dopamine uncaging after the encoding of OLMs did not have effect on weak memories whereas on strong memories diminished the exploration of the novel object location. Additionally, hippocampal dopamine elevation during the retrieval of OLMs did not allow the recovery of weak memories and did not affect the retrieval of strong memory traces. Finally, dopamine elevation increased hippocampal theta oscillations, indicating that dopamine promotes the recurrent activation of specific groups of neurons. Our experiments demonstrate that hippocampal dopaminergic modulation during the encoding of OLMs depends on memory strength indicating that hyperdopaminergic levels that enhance weak experiences could compromise the normal storage of strong memories.

Prairie voles as a model for adaptive reward remodeling following loss of a bonded partner

Loss of a loved one is a painful event that substantially elevates the risk for physical and mental illness and impaired daily function. Socially monogamous prairie voles are laboratory-amenable rodents that form life-long pair bonds and exhibit distress upon partner separation, mirroring phenotypes seen in humans. These attributes make voles an excellent model for studying the biology of loss. In this review, we highlight parallels between humans and prairie voles, focusing on reward system engagement during pair bonding and loss. As yearning is a unique feature that differentiates loss from other negative mental states, we posit a model in which the homeostatic reward mechanisms that help to maintain bonds are disrupted upon loss, resulting in yearning and other negative impacts. Finally, we synthesize studies in humans and voles that delineate the remodeling of reward systems during loss adaptation. The stalling of these processes likely contributes to prolonged grief disorder, a diagnosis recently added to the Diagnostic and Statistical Manual for Psychiatry.

Two what, two where, visual cortical streams in humans

ROLLS, E. T. Two What, Two Where, Visual Cortical Streams in Humans. NEUROSCI BIOBEHAV REV 2024. Recent cortical connectivity investigations lead to new concepts about 'What' and 'Where' visual cortical streams in humans, and how they connect to other cortical systems. A ventrolateral 'What' visual stream leads to the inferior temporal visual cortex for object and face identity, and provides 'What' information to the hippocampal episodic memory system, the anterior temporal lobe semantic system, and the orbitofrontal cortex emotion system. A superior temporal sulcus (STS) 'What' visual stream utilising connectivity from the temporal and parietal visual cortex responds to moving objects and faces, and face expression, and connects to the orbitofrontal cortex for emotion and social behaviour. A ventromedial 'Where' visual stream builds feature combinations for scenes, and provides 'Where' inputs via the parahippocampal scene area to the hippocampal episodic memory system that are also useful for landmark-based navigation. The dorsal 'Where' visual pathway to the parietal cortex provides for actions in space, but also provides coordinate transforms to provide inputs to the parahippocampal scene area for self-motion update of locations in scenes in the dark or when the view is obscured.

The lateral habenula: A hub for value-guided behavior

The habenula is an evolutionarily highly conserved diencephalic brain region divided into two major parts, medial and lateral. Over the past two decades, studies of the lateral habenula (LHb), in particular, have identified key functions in value-guided behavior in health and disease. In this review, we focus on recent insights into LHb connectivity and its functional relevance for different types of aversive and appetitive value-guided behavior. First, we give an overview of the anatomical organization of the LHb and its main cellular composition. Next, we elaborate on how distinct LHb neuronal subpopulations encode aversive and appetitive stimuli and on their involvement in more complex decision-making processes. Finally, we scrutinize the afferent and efferent connections of the LHb and discuss their functional implications for LHb-dependent behavior. A deepened understanding of distinct LHb circuit components will substantially contribute to our knowledge of value-guided behavior.

Midfrontal conflict theta and parietal P300 are linked to a latent factor of DSM externalising disorders

Psychiatric illnesses form spectra rather than categories, with symptoms varying continuously across individuals, i.e., there is no clear break between health and disorder. Dimensional measures of behaviour and brain activity are promising targets for studying biological mechanisms that are common across disorders. Here, we assessed the extent to which neural measures of the sensitivity of the three biological systems in the reinforcement sensitivity theory (RST) could account for individual differences in a latent general factor estimated from symptom counts across externalising disorders (EXTs). RST explanatory power was pitted against reduced P300, a reliable indicator of externalising per previous research. We assessed 206 participants for DSM-5 EXTs (antisocial personality disorder, conduct disorder, attention-deficit/hyperactivity disorder, intermittent explosive disorder symptoms, alcohol use disorder, and cannabis use disorder). Of the final sample, 49% met diagnostic criteria for at least one of the EXTs. Electroencephalographic measures of the sensitivities of the behavioural activation system (BAS), the fight/flight/freeze system, and the behavioural inhibition system (BIS), as well as P300 were extracted from the gold bar-lemon and stop-signal tasks. As predicted, we found that low neural BIS sensitivity and low P300 were uniquely and negatively associated with our latent factor of externalising. Contrary to prediction, neural BAS/"dopamine" sensitivity was not associated with externalising. Our results provide empirical support for low BIS sensitivity and P300 as neural mechanisms common to disorders within the externalising spectrum; but, given the low N involved, future studies should seek to assess the replicability of our findings and, in particular, the differential involvement of the three RST systems.

Distributional coding of associative learning in discrete populations of midbrain dopamine neurons

Midbrain dopamine neurons are thought to play key roles in learning by conveying the difference between expected and actual outcomes. Recent evidence suggests diversity in dopamine signaling, yet it remains poorly understood how heterogeneous signals might be organized to facilitate the role of downstream circuits mediating distinct aspects of behavior. Here, we investigated the organizational logic of dopaminergic signaling by recording and labeling individual midbrain dopamine neurons during associative behavior. Our findings show that reward information and behavioral parameters are not only heterogeneously encoded but also differentially distributed across populations of dopamine neurons. Retrograde tracing and fiber photometry suggest that populations of dopamine neurons projecting to different striatal regions convey distinct signals. These data, supported by computational modeling, indicate that such distributional coding can maximize dynamic range and tailor dopamine signals to facilitate specialized roles of different striatal regions.

Early-Life Adversities Are Associated With Lower Expected Value Signaling in the Adult Brain

BACKGROUND

Early adverse experiences are assumed to affect fundamental processes of reward learning and decision making. However, computational neuroimaging studies investigating these circuits in the context of adversity are sparse and limited to studies conducted in adolescent samples, leaving the long-term effects unexplored.

METHODS

Using data from a longitudinal birth cohort study (n = 156; 87 female), we investigated associations between adversities and computational markers of reward learning (i.e., expected value, prediction errors). At age 33 years, all participants completed a functional magnetic resonance imaging-based passive avoidance task. Psychopathology measures were collected at the time of functional magnetic resonance imaging investigation and during the COVID-19 pandemic. We applied a principal component analysis to capture common variations across 7 adversity measures. The resulting adversity factors (factor 1: postnatal psychosocial adversities and prenatal maternal smoking; factor 2: prenatal maternal stress and obstetric adversity; factor 3: lower maternal stimulation) were linked with psychopathology and neural responses in the core reward network using multiple regression analysis.

RESULTS

We found that the adversity dimension primarily informed by lower maternal stimulation was linked to lower expected value representation in the right putamen, right nucleus accumbens, and anterior cingulate cortex. Expected value encoding in the right nucleus accumbens further mediated the relationship between this adversity dimension and psychopathology and predicted higher withdrawn symptoms during the COVID-19 pandemic.

CONCLUSIONS

Our results suggested that early adverse experiences in caregiver context might have a long-term disruptive effect on reward learning in reward-related brain regions, which can be associated with suboptimal decision making and thereby may increase the vulnerability of developing psychopathology.

Characterizing the dynamic learning process: Implications of a quantitative analysis

Understanding the neural mechanisms involved in learning processes is crucial for unraveling the complexities of behavior and cognition. Sudden change from the untrained level to the fully-learned level is a pivotal feature of instrumental learning. However, the concept of change point and suitable methods to conveniently analyze the characteristics of sudden change in groups remain elusive, which might hinder a fuller understanding of the neural mechanism underlying dynamic leaning process. In the current study, we investigated the learning processes of mice that were trained in an aversive instrumental learning task, and introduced a novel strategy to analyze behavioral variations in instrumental learning, leading to improved clarity on the concept of sudden change and enabling comprehensive group analysis. By applying this novel strategy, we examined the effects of cocaine and a cannabinoid receptor agonist on instrumental learning. Intriguingly, our analysis revealed significant differences in timing and occurrence of sudden changes that were previously overlooked using traditional analysis. Overall, our research advances understanding of behavioral variation during instrumental learning and the interplay between learning behaviors and neurotransmitter systems, contributing to a deeper comprehension of learning processes and informing future investigations and therapeutic interventions.

More widespread and rigid neuronal representation of reward expectation underlies impulsive choices

Impulsive choices prioritize smaller, more immediate rewards over larger, delayed, or potentially uncertain rewards. Impulsive choices are a critical aspect of substance use disorders and maladaptive decision-making across the lifespan. Here, we sought to understand the neuronal underpinnings of expected reward and risk estimation on a trial-by-trial basis during impulsive choices. To do so, we acquired electrical recordings from the human brain while participants carried out a risky decision-making task designed to measure choice impulsivity. Behaviorally, we found a reward-accuracy tradeoff, whereby more impulsive choosers were more accurate at the task, opting for a more immediate reward while compromising overall task performance. We then examined how neuronal populations across frontal, temporal, and limbic brain regions parametrically encoded reinforcement learning model variables, namely reward and risk expectation and surprise, across trials. We found more widespread representations of reward value expectation and prediction error in more impulsive choosers, whereas less impulsive choosers preferentially represented risk expectation. A regional analysis of reward and risk encoding highlighted the anterior cingulate cortex for value expectation, the anterior insula for risk expectation and surprise, and distinct regional encoding between impulsivity groups. Beyond describing trial-by-trial population neuronal representations of reward and risk variables, these results suggest impaired inhibitory control and model-free learning underpinnings of impulsive choice. These findings shed light on neural processes underlying reinforced learning and decision-making in uncertain environments and how these processes may function in psychiatric disorders.

Dopamine projections to the basolateral amygdala drive the encoding of identity-specific reward memories

To make adaptive decisions, we build an internal model of the associative relationships in an environment and use it to make predictions and inferences about specific available outcomes. Detailed, identity-specific cue-reward memories are a core feature of such cognitive maps. Here we used fiber photometry, cell-type and pathway-specific optogenetic manipulation, Pavlovian cue-reward conditioning and decision-making tests in male and female rats, to reveal that ventral tegmental area dopamine (VTADA) projections to the basolateral amygdala (BLA) drive the encoding of identity-specific cue-reward memories. Dopamine is released in the BLA during cue-reward pairing; VTADA→BLA activity is necessary and sufficient to link the identifying features of a reward to a predictive cue but does not assign general incentive properties to the cue or mediate reinforcement. These data reveal a dopaminergic pathway for the learning that supports adaptive decision-making and help explain how VTADA neurons achieve their emerging multifaceted role in learning.

Evolutionary perspectives on substance and behavioural addictions: Distinct and shared pathways to understanding, prediction and prevention

Addiction poses significant social, health, and criminal issues. Its moderate heritability and early-life impact, affecting reproductive success, poses an evolutionary paradox: why are humans predisposed to addictive behaviours? This paper reviews biological and psychological mechanisms of substance and behavioural addictions, exploring evolutionary explanations for the origin and function of relevant systems. Ancestrally, addiction-related systems promoted fitness through reward-seeking, and possibly self-medication. Today, psychoactive substances disrupt these systems, leading individuals to neglect essential life goals for immediate satisfaction. Behavioural addictions (e.g. video games, social media) often emulate ancestrally beneficial behaviours, making them appealing yet often irrelevant to contemporary success. Evolutionary insights have implications for how addiction is criminalised and stigmatised, propose novel avenues for interventions, anticipate new sources of addiction from emerging technologies such as AI. The emerging potential of glucagon-like peptide 1 (GLP-1) agonists targeting obesity suggest the satiation system may be a natural counter to overactivation of the reward system.

Observational reinforcement learning in children and young adults

Observational learning is essential for the acquisition of new behavior in educational practices and daily life and serves as an important mechanism for human cognitive and social-emotional development. However, we know little about its underlying neurocomputational mechanisms from a developmental perspective. In this study we used model-based fMRI to investigate differences in observational learning and individual learning between children and younger adults. Prediction errors (PE), the difference between experienced and predicted outcomes, related positively to striatal and ventral medial prefrontal cortex activation during individual learning and showed no age-related differences. PE-related activation during observational learning was more pronounced when outcomes were worse than predicted. Particularly, negative PE-coding in the dorsal medial prefrontal cortex was stronger in adults compared to children and was associated with improved observational learning in children and adults. The current findings pave the way to better understand observational learning challenges across development and educational settings.

Neurovascular effects of cocaine: relevance to addiction

Cocaine is a highly addictive drug, and its use is associated with adverse medical consequences such as cerebrovascular accidents that result in debilitating neurological complications. Indeed, brain imaging studies have reported severe reductions in cerebral blood flow (CBF) in cocaine misusers when compared to the brains of healthy non-drug using controls. Such CBF deficits are likely to disrupt neuro-vascular interaction and contribute to changes in brain function. This review aims to provide an overview of cocaine-induced CBF changes and its implication to brain function and to cocaine addiction, including its effects on tissue metabolism and neuronal activity. Finally, we discuss implications for future research, including targeted pharmacological interventions and neuromodulation to limit cocaine use and mitigate the negative impacts.

NSF DARE-transforming modeling in neurorehabilitation: a patient-in-the-loop framework

In 2023, the National Science Foundation (NSF) and the National Institute of Health (NIH) brought together engineers, scientists, and clinicians by sponsoring a conference on computational modelling in neurorehabiilitation. To facilitate multidisciplinary collaborations and improve patient care, in this perspective piece we identify where and how computational modelling can support neurorehabilitation. To address the where, we developed a patient-in-the-loop framework that uses multiple and/or continual measurements to update diagnostic and treatment model parameters, treatment type, and treatment prescription, with the goal of maximizing clinically-relevant functional outcomes. This patient-in-the-loop framework has several key features: (i) it includes diagnostic and treatment models, (ii) it is clinically-grounded with the International Classification of Functioning, Disability and Health (ICF) and patient involvement, (iii) it uses multiple or continual data measurements over time, and (iv) it is applicable to a range of neurological and neurodevelopmental conditions. To address the how, we identify state-of-the-art and highlight promising avenues of future research across the realms of sensorimotor adaptation, neuroplasticity, musculoskeletal, and sensory & pain computational modelling. We also discuss both the importance of and how to perform model validation, as well as challenges to overcome when implementing computational models within a clinical setting. The patient-in-the-loop approach offers a unifying framework to guide multidisciplinary collaboration between computational and clinical stakeholders in the field of neurorehabilitation.

The role of prospective contingency in the control of behavior and dopamine signals during associative learning

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. Here we examined the dopamine activity in the ventral striatum - a signal implicated in associative learning - in a Pavlovian contingency degradation task in mice. We show that both anticipatory licking and dopamine responses to a conditioned stimulus decreased when additional rewards were delivered uncued, but remained unchanged if additional rewards were cued. These results conflict with contingency-based accounts using a traditional definition of contingency or a novel causal learning model (ANCCR), but can be explained by temporal difference (TD) learning models equipped with an appropriate inter-trial-interval (ITI) state representation. Recurrent neural networks trained within a TD framework develop state representations like our best 'handcrafted' model. Our findings suggest that the TD error can be a measure that describes both contingency and dopaminergic activity.