Reward Anticipation Is Encoded Differently by Adolescent Ventral Tegmental Area Neurons

Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making

Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.

Neural responses to reward valence and magnitude from pre- to early adolescence

BACKGROUND

Neural activation during reward processing is thought to underlie critical behavioral changes that take place during the transition to adolescence (e.g., learning, risk-taking). Though literature on the neural basis of reward processing in adolescence is booming, important gaps remain. First, more information is needed regarding changes in functional neuroanatomy in early adolescence. Another gap is understanding whether sensitivity to different aspects of the incentive (e.g., magnitude and valence) changes during the transition into adolescence. We used fMRI from a large sample of preadolescent children to characterize neural responses to incentive valence vs. magnitude during anticipation and feedback, and their change over a period of two years.

METHODS

Data were taken from the Adolescent Cognitive and Brain DevelopmentSM (ABCD®) study release 3.0. Children completed the Monetary Incentive Delay task at baseline (ages 9-10) and year 2 follow-up (ages 11-12). Based on data from two sites (N = 491), we identified activation-based Regions of Interest (ROIs; e.g., striatum, prefrontal regions, etc.) that were sensitive to trial type (win $5, win $0.20, neutral, lose $0.20, lose $5) during anticipation and feedback phases. Then, in an independent subsample (N = 1470), we examined whether these ROIs were sensitive to valence and magnitude and whether that sensitivity changed over two years.

RESULTS

Our results show that most ROIs involved in reward processing (including the striatum, prefrontal cortex, and insula) are specialized, i.e., mainly sensitive to either incentive valence or magnitude, and this sensitivity was consistent over a 2-year period. The effect sizes of time and its interactions were significantly smaller (0.002≤η2≤0.02) than the effect size of trial type (0.06≤η2≤0.30). Interestingly, specialization was moderated by reward processing phase but was stable across development. Biological sex and pubertal status differences were few and inconsistent. Developmental changes were mostly evident during success feedback, where neural reactivity increased over time.

CONCLUSIONS

Our results suggest sub-specialization to valence vs. magnitude within many ROIs of the reward circuitry. Additionally, in line with theoretical models of adolescent development, our results suggest that the ability to benefit from success increases from pre- to early adolescence. These findings can inform educators and clinicians and facilitate empirical research of typical and atypical motivational behaviors during a critical time of development.

The role of dopamine and endocannabinoid systems in prefrontal cortex development: Adolescence as a critical period

The prefrontal cortex plays a central role in the control of complex cognitive processes including action control and decision making. It also shows a specific pattern of delayed maturation related to unique behavioral changes during adolescence and allows the development of adult cognitive processes. The adolescent brain is extremely plastic and critically vulnerable to external insults. Related to this vulnerability, adolescence is also associated with the emergence of numerous neuropsychiatric disorders involving alterations of prefrontal functions. Within prefrontal microcircuits, the dopamine and the endocannabinoid systems have widespread effects on adolescent-specific ontogenetic processes. In this review, we highlight recent advances in our understanding of the maturation of the dopamine system and the endocannabinoid system in the prefrontal cortex during adolescence. We discuss how they interact with GABA and glutamate neurons to modulate prefrontal circuits and how they can be altered by different environmental events leading to long-term neurobiological and behavioral changes at adulthood. Finally, we aim to identify several future research directions to help highlight gaps in our current knowledge on the maturation of these microcircuits.

Reward Enhances Memory via Age-Varying Online and Offline Neural Mechanisms across Development

Reward motivation enhances memory through interactions between mesolimbic, hippocampal, and cortical systems, both during and after encoding. Developmental changes in these distributed neural circuits may lead to age-related differences in reward-motivated memory and the underlying neural mechanisms. Converging evidence from cross-species studies suggests that subcortical dopamine signaling is increased during adolescence, which may lead to stronger memory representations of rewarding, relative to mundane, events and changes in the contributions of underlying subcortical and cortical brain mechanisms across age. Here, we used fMRI to examine how reward motivation influences the "online" encoding and "offline" postencoding brain mechanisms that support long-term associative memory from childhood to adulthood in human participants of both sexes. We found that reward motivation led to both age-invariant enhancements and nonlinear age-related differences in associative memory after 24 h. Furthermore, reward-related memory benefits were linked to age-varying neural mechanisms. During encoding, interactions between the prefrontal cortex (PFC) and ventral tegmental area (VTA) were associated with better high-reward memory to a greater degree with increasing age. Preencoding to postencoding changes in functional connectivity between the anterior hippocampus and VTA were also associated with better high-reward memory, but more so at younger ages. Our findings suggest that there may be developmental differences in the contributions of offline subcortical and online cortical brain mechanisms supporting reward-motivated memory.SIGNIFICANCE STATEMENT A substantial body of research has examined the neural mechanisms through which reward influences memory formation in adults. However, despite extensive evidence that both reward processing and associative memory undergo dynamic change across development, few studies have examined age-related changes in these processes. We found both age-invariant and nonlinear age-related differences in reward-motivated memory. Moreover, our findings point to developmental differences in the processes through which reward modulates the prioritization of information in long-term memory, with greater early reliance on offline subcortical consolidation mechanisms and increased contribution of systems-level online encoding circuitry with increasing age. These results highlight dynamic developmental changes in the cognitive and neural mechanisms through which motivationally salient information is prioritized in memory from childhood to adulthood.

Behavior and Fos activation reveal that male and female rats differentially assess affective valence during CTA learning and expression

Females are more affected by psychiatric illnesses including eating disorders, depression, and post-traumatic stress disorder than males. However, the neural mechanisms mediating these sex differences are poorly understood. Animal models can be useful in exploring such neural mechanisms. Conditioned taste aversion (CTA) is a behavioral task that assesses how animals process the competition between associated reinforcing and aversive stimuli in subsequent task performance, a process critical to healthy behavior in many domains. The purpose of the present study was to identify sex differences in this behavior and associated neural responses. We hypothesized that females would value the rewarding stimulus (Boost®) relative to the aversive stimulus (LiCl) more than males in performing CTA. We evaluated behavior (Boost® intake, LiCl-induced behaviors, ultrasonic vocalizations (USVs), CTA performance) and Fos activation in relevant brain regions after the acute stimuli [acute Boost® (AB), acute LiCl (AL)] and the context-only task control (COT), Boost® only task (BOT) and Boost®-LiCl task (BLT). Acutely, females drank more Boost® than males but showed similar aversive behaviors after LiCl. Females and males performed CTA similarly. Both sexes produced 55 kHz USVs anticipating BOT and inhibited these calls in the BLT. However, more females emitted both 22 kHz and 55 kHz USVs in the BLT than males: the latter correlated with less CTA. Estrous cycle stage also influenced 55 kHz USVs. Fos responses were similar in males and females after AB or AL. Females engaged the gustatory cortex and ventral tegmental area (VTA) more than males during the BOT and males engaged the amygdala more than females in both the BOT and BLT. Network analysis of correlated Fos responses across brain regions identified two unique networks characterizing the BOT and BLT, in both of which the VTA played a central role. In situ hybridization with RNAscope identified a population of D1-receptor expressing cells in the CeA that responded to Boost® and D2 receptor-expressing cells that responded to LiCl. The present study suggests that males and females differentially process the affective valence of a stimulus to produce the same goal-directed behavior.

Adolescent Dopamine Neurons Represent Reward Differently during Action and State Guided Learning

Neuronal underpinning of learning cause-and-effect associations in the adolescent brain remains poorly understood. Two fundamental forms of associative learning are Pavlovian (classical) conditioning, where a stimulus is followed by an outcome, and operant (instrumental) conditioning, where outcome is contingent on action execution. Both forms of learning, when associated with a rewarding outcome, rely on midbrain dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SN). We find that, in adolescent male rats, reward-guided associative learning is encoded differently by midbrain dopamine neurons in each conditioning paradigm. Whereas simultaneously recorded VTA and SN adult neurons have a similar phasic response to reward delivery during both forms of conditioning, adolescent neurons display a muted reward response during operant but a profoundly larger reward response during Pavlovian conditioning. These results suggest that adolescent neurons assign a different value to reward when it is not gated by action. The learning rate of adolescents and adults during both forms of conditioning was similar, supporting the notion that differences in reward response in each paradigm may be because of differences in motivation and independent of state versus action value learning. Static characteristics of dopamine neurons, such as dopamine cell number and size, were similar in the VTA and SN of both ages, but there were age-related differences in stimulated dopamine release and correlated spike activity, suggesting that differences in reward responsiveness by adolescent dopamine neurons are not because of differences in intrinsic properties of these neurons but engagement of different dopaminergic networks.SIGNIFICANCE STATEMENT Reckless behavior and impulsive decision-making by adolescents suggest that motivated behavioral states are encoded differently by the adolescent brain. Motivated behavior, which is dependent on the function of the dopamine system, follows learning of cause-and-effect associations in the environment. We find that dopamine neurons in adolescents encode reward differently depending on the cause-and-effect relationship of the means to receive that reward. Compared with adults, reward contingent on action led to a muted response, whereas reward that followed a cue but was not gated by action produced an augmented phasic response. These data demonstrate an age-related difference in dopamine neuron response to reward that is not uniform and is guided by processes that differentiate between state and action values.

Behavioral, Physiological, and Neural Signatures of Surprise during Naturalistic Sports Viewing

Surprise signals a discrepancy between past and current beliefs. It is theorized to be linked to affective experiences, the creation of particularly resilient memories, and segmentation of the flow of experience into discrete perceived events. However, the ability to precisely measure naturalistic surprise has remained elusive. We used advanced basketball analytics to derive a quantitative measure of surprise and characterized its behavioral, physiological, and neural correlates in human subjects observing basketball games. We found that surprise was associated with segmentation of ongoing experiences, as reflected by subjectively perceived event boundaries and shifts in neocortical patterns underlying belief states. Interestingly, these effects differed by whether surprising moments contradicted or bolstered current predominant beliefs. Surprise also positively correlated with pupil dilation, activation in subcortical regions associated with dopamine, game enjoyment, and long-term memory. These investigations support key predictions from event segmentation theory and extend theoretical conceptualizations of surprise to real-world contexts.

Sex and strain differences in dynamic and static properties of the mesolimbic dopamine system

Sex is a biological variable that contributes to the incidence, clinical course, and treatment outcome of brain disorders. Chief among these are disorders associated with the dopamine system. These include Parkinson's disease, ADHD, schizophrenia, and mood disorders, which show stark differences in prevalence and outcome between men and women. In order to reveal the influence of biological sex as a risk factor in these disorders, there is a critical need to collect fundamental information about basic properties of the dopamine system in males and females. In Long Evans rats, we measured dynamic and static properties related to the mesolimbic dopamine system. Static measures included assessing ventral tegmental area (VTA) dopamine cell number and volume and expression of tyrosine hydroxylase and dopamine transporter. Dynamic measures in behaving animals included assessing (1) VTA neuronal encoding during learning of a cue-action-reward instrumental task and (2) dopamine release in the nucleus accumbens in response to electrical stimulation of the VTA, vesicular depletion of dopamine, and amphetamine. We found little or no sex difference in these measures, suggesting sexual congruency in fundamental static and dynamic properties of dopamine neurons. Thus, dopamine related sex-differences are likely mediated by secondary mechanisms that flexibly influence the function of the dopamine cells and circuits. Finally, we noted that most behavioral sex differences had been reported in Sprague-Dawley rats and repeated some of the above measures in that strain. We found some sex differences in those animals highlighting the importance of considering strain differences in experimental design and result interpretation.

Reinforcement Learning during Adolescence in Rats

The most dynamic period of postnatal brain development occurs during adolescence, the period between childhood and adulthood. Neuroimaging studies have observed morphologic and functional changes during adolescence, and it is believed that these changes serve to improve the functions of circuits that underlie decision-making. Direct evidence in support of this hypothesis, however, has been limited because most preclinical decision-making paradigms are not readily translated to humans. Here, we developed a reversal-learning protocol for the rapid assessment of adaptive choice behavior in dynamic environments in rats as young as postnatal day 30. A computational framework was used to elucidate the reinforcement-learning mechanisms that change in adolescence and into adulthood. Using a cross-sectional and longitudinal design, we provide the first evidence that value-based choice behavior in a reversal-learning task improves during adolescence in male and female Long-Evans rats and demonstrate that the increase in reversal performance is due to alterations in value updating for positive outcomes. Furthermore, we report that reversal-learning trajectories in adolescence reliably predicted reversal performance in adulthood. This novel behavioral protocol provides a unique platform for conducting biological and systems-level analyses of the neurodevelopmental mechanisms of decision-making.SIGNIFICANCE STATEMENT The neurodevelopmental adaptations that occur during adolescence are hypothesized to underlie age-related improvements in decision-making, but evidence to support this hypothesis has been limited. Here, we describe a novel behavioral protocol for rapidly assessing adaptive choice behavior in adolescent rats with a reversal-learning paradigm. Using a computational approach, we demonstrate that age-related changes in reversal-learning performance in male and female Long-Evans rats are linked to specific reinforcement-learning mechanisms and are predictive of reversal-learning performance in adulthood. Our behavioral protocol provides a unique platform for elucidating key components of adolescent brain function.

Dopamine Modulation of Prefrontal Cortex Activity Is Manifold and Operates at Multiple Temporal and Spatial Scales

Although the function of dopamine in subcortical structures is largely limited to reward and movement, dopamine neurotransmission in the prefrontal cortex (PFC) is critical to a multitude of temporally and functionally diverse processes, such as attention, working memory, behavioral flexibility, action planning, and sustained motivational and affective states. How does dopamine influence computation of these temporally complex functions? We find causative links between sustained and burst patterns of phasic dopamine neuron activation and modulation of medial PFC neuronal activity at multiple spatiotemporal scales. These include a multidirectional and weak impact on individual neuron rate activity but a robust influence on coordinated ensemble activity, gamma oscillations, and gamma-theta coupling that persisted for minutes. In addition, PFC network responses to burst pattern of dopamine firing were selectively strengthened in behaviorally active states. This multiplex mode of modulation by dopamine input may enable PFC to compute and generate spatiotemporally diverse and specialized outputs.

Effects of amphetamine exposure during adolescence on behavior and prelimbic cortex neuron activity in adulthood

Repeated exposure to psychostimulants during adolescence produces long-lasting changes in behavior that may be mediated by disrupted development of the mesocorticolimbic dopamine system. Here, we tested this hypothesis by assessing the effects of amphetamine (AMPH) and dopamine receptor-selective drugs on behavior and neuron activity in the prelimbic region of the medial prefrontal cortex (PFC). Adolescent male, Sprague-Dawley rats were given saline or 3 mg/kg AMPH between postnatal day (P) 27 and P45. In Experiment 1, locomotor behavior was assessed during adulthood following challenges with a dopamine D1 (SKF 82958) or D2 (quinpirole) receptor-selective agonist. In Experiment 2, pre-exposed rats were challenged during adulthood with AMPH and a D1 (SKF 83566) or D2 (eticlopride) receptor-selective antagonist. In Experiment 3, the activity of putative pyramidal cells in the prelimbic cortex was recorded as rats behaved in an open-field arena before and after challenge injections with AMPH and one of the antagonists. We found that compared to controls, adolescent pre-exposed rats were more sensitive to the stimulant effects of AMPH and the dopamine receptor agonists, as well as to the ability of the antagonists to reverse AMPH-induced stereotypy. Prelimbic neurons from AMPH pre-exposed rats were also more likely to respond to an AMPH challenge in adulthood, primarily by reducing their activity, and the antagonists reversed these effects. Our results suggest that exposure to AMPH during adolescence leads to enduring adaptations in the mesocorticolimbic dopamine system that likely mediate heightened response to the drug during adulthood.

Age and sex differences in behavioral flexibility, sensitivity to reward value, and risky decision-making

Compared with adults, adolescent behavior is often characterized by reduced behavioral flexibility, increased sensitivity to reward, and increased likelihood to take risks. These traits, which have been hypothesized to confer heightened vulnerability to psychopathologies such as substance use disorders (SUDs), have been the focus of studies in laboratory animal models that seek to understand their neural underpinnings. However, rodent studies to date have typically used only males and have adopted standard methodological practices (e.g., weight loss inducing food restriction) that are likely to have a disparate impact on adolescents compared with adults. Here, we used adolescent and adult Sprague-Dawley rats of both sexes to study instrumental behavior tasks that assess behavioral flexibility (strategy shifting and reversal learning; Experiment 1), sensitivity to reward value (outcome devaluation; Experiment 2), and risky decision making (probability discounting; Experiment 3). In Experiment 1, we found that adolescents were faster to acquire reversal learning than adults but there were no differences in strategy shifting. In Experiments 2 and 3, adolescents and adults were equally sensitive to changes in reward value and exhibited similar reductions in preference for a large reward when reinforcement probability was decreased. However, adolescents responded more efficiently and earned reinforcers at a higher rate than their same-sex, adult counterparts. Together, these findings provide only limited support for the existence of an "adolescent-typical" phenotype in Sprague-Dawley rats and instead suggest that age differences in the expression of these behaviors may depend on conditions such as pubertal status and motivational state. (PsycINFO Database Record

Risk of punishment influences discrete and coordinated encoding of reward-guided actions by prefrontal cortex and VTA neurons

Actions motivated by rewards are often associated with risk of punishment. Little is known about the neural representation of punishment risk during reward-seeking behavior. We modeled this circumstance in rats by designing a task where actions were consistently rewarded but probabilistically punished. Spike activity and local field potentials were recorded during task performance simultaneously from VTA and mPFC, two reciprocally connected regions implicated in reward-seeking and aversive behaviors. At the single unit level, we found that ensembles of putative dopamine and non-dopamine VTA neurons and mPFC neurons encode the relationship between action and punishment. At the network level, we found that coherent theta oscillations synchronize VTA and mPFC in a bottom-up direction, effectively phase-modulating the neuronal spike activity in the two regions during punishment-free actions. This synchrony declined as a function of punishment probability, suggesting that during reward-seeking actions, risk of punishment diminishes VTA-driven neural synchrony between the two regions.

Networks of VTA Neurons Encode Real-Time Information about Uncertain Numbers of Actions Executed to Earn a Reward

Multiple and unpredictable numbers of actions are often required to achieve a goal. In order to organize behavior and allocate effort so that optimal behavioral policies can be selected, it is necessary to continually monitor ongoing actions. Real-time processing of information related to actions and outcomes is typically assigned to the prefrontal cortex and basal ganglia, but also depends on midbrain regions, especially the ventral tegmental area (VTA). We were interested in how individual VTA neurons, as well as networks within the VTA, encode salient events when an unpredictable number of serial actions are required to obtain a reward. We recorded from ensembles of putative dopamine and non-dopamine neurons in the VTA as animals performed multiple cued trials in a recording session where, in each trial, serial actions were randomly rewarded. While averaging population activity did not reveal a response pattern, we observed that different neurons were selectively tuned to low, medium, or high numbered actions in a trial. This preferential tuning of putative dopamine and non-dopamine VTA neurons to different subsets of actions in a trial allowed information about binned action number to be decoded from the ensemble activity. At the network level, tuning curve similarity was positively associated with action-evoked noise correlations, suggesting that action number selectivity reflects functional connectivity within these networks. Analysis of phasic responses to cue and reward revealed that the requirement to execute multiple and uncertain numbers of actions weakens both cue-evoked responses and cue-reward response correlation. The functional connectivity and ensemble coding scheme that we observe here may allow VTA neurons to cooperatively provide a real-time account of ongoing behavior. These computations may be critical to cognitive and motivational functions that have long been associated with VTA dopamine neurons.

Methylphenidate has nonlinear dose effects on cued response inhibition in adults but not adolescents

Ongoing development of the dopamine system during adolescence may provide a partial mechanism for behavioral and psychiatric vulnerabilities. Despite early evidence for a hyperactive adolescent dopaminergic system, recent data suggest that adolescent dopamine may be functionally hypoactive compared to in adults. While this distinction has been established in response to dopaminergic drugs and natural rewards, little is known about age-related differences in cognitive efficacy of dopaminergic drugs. Using a recently established Cued Response Inhibition Task, we tested the effects of acute systemic methylphenidate, commonly known as Ritalin, on response inhibition and response initiation in adolescent and adults rats. First, we replicated previous data that adolescents are able to inhibit a response to a cue on par with adults, but are slower to produce a rewarded response after a stop cue. Next, we observed that methylphenidate modulated response inhibition in adult rats, with low dose (0.3mg/kg) improving inhibition, and high dose (3mg/kg) impairing performance. This dose-response pattern is commonly observed with psychostimulant cognitive modulation. In adolescents, however, methylphenidate had no effect on response inhibition at any dose. Latency of response initiation after the stop cue was not affected by methylphenidate in either adult or adolescent rats. These data establish that dose-response of a commonly prescribed psychostimulant medication is different in adolescents and adults. They further demonstrate that healthy adolescent response inhibition is not as sensitive to psychostimulants as in adults, supporting the idea that the dopamine system is hypoactive in adolescence. This article is part of a Special Issue entitled SI: Adolescent plasticity.